Wall Texture Tutorial

Originally written by Corth on http://forums.thedarkmod.com/topic/1490

Ok, Renzatic asked me (or I volunteered... I can't remember...) to create a tutorial about how I did the normal map with those big stones. So I did:

Find a Reference

Find a reference (online, or your own) and load it into photoshop. You'll need your image to adhere to the size limits, example: (X)x(X) where (X)= pixel size. It can be either 1, 4, 8, 16, 32, 64, 128, 256, 512, 1024 or 2048. The most common is 512x512. These numbers can be mixed around for a size of 256x512 or any other combination.

There are many ways to go about making the image tilable. One way is to use the photoshop filter 'offset...' found under Filter>other>Offset... it looks like this:

After the image has been offset, you'll notice seams (area's where the texture doesn't line up). Here is where you can use the clone stamp tool (shortcut: S) to try and fix it up, or you can use the regular brush (shortcut: cool.gif to fix the image up. Another way to do this, is to select parts of your reference image, and paste them onto your texture sheet, cut the pasted part in half and put them on opposite sides - quick and easy tiling.

For this tutorial I'm going to use a texture that BlackThief setup:

I saved this as a black and white image as well (shift+ctrl+U), and took out some of the shadows:

While I still had this image, I saved it again to make a specularity map from it using Levels (ctrl+L):

with the settings above, I got an image like this:

Going into Maya

Now that you have your color map (or diffuse map), specular map and a black and white image, it's time to open up maya to get yourself a normal map.

In either your front or side view, adding an image plane will speed up your process:

if at any time you want to delete the image plane, go back to this menu and select your image plane's attributes, then just hit backspace or delete.

Select your color map, or the black and white version (whichever will allow you to follow it more closely) as your image plane. Start with a polygon plane and cut and extrude wherever you need to get BASIC details down. We'll save the more difficult details for maya to figure out. One thing to keep in mind while adding your basic hight, is to keep thinking about tiling! Shading>Shade Options>XRay is also helpful during this phase.

The white line is straight up and down, you want to make sure that on either end of your modeled wall, that they will still line up.

After you have completed the basic shape of the object, it's time to UV map it, don't worry it's very simple. First, open your UV texture editor (Window>Saved Layouts>Persp/UV Texture Editor) now select your shaped wall.

Click on the box beside Planar Mapping, it will bring up a window in which the only thing that matters for now, is the mapping direction. you want to choose the axis that forms a T intersection with your wall. Check your UV Texture Editor to make sure your object fills out the whole space, even if your not using a square texture map, as maya will stretch the UV's to cover the rectangular shape.

Now it's time to setup a displacement map! that's where all the fine details come from. Now you have to open your HyperShade (Window>Rendering Editors>Hypershade) create another lambert material, name it anything you want, or leave it alone, for this part, I named it "wall_displacement". now bring up your attribute editor for the material (ctrl+A) and in the color attribute (checker box) add a file, select the black and white version of your wall texture. Back in your hypershade, select the material again. Just above the Materials list, you'll see 3 boxes with triangles, they show you the inputs or outputs of the selected item, the middle button shows both, so click it, and your 'Work Area' will be updated. now select the line that connects your black and white file to the shader. You don't actually want it to be connected to the shaders color attribute, but rather as a displacement map. Delete the connection. Middle Mouse click on your black and white file, and drag your mouse over top of the shader, a menu will pop up asking you what input to connect it to, select "displacement map"

Now is where this process becomes intense on your computer, so I suggest saving your file so you don't have to do it all over again.

Select your wall again, then Modify>Convert>Displacement to Polygons. You might have to wait a while for this to finish, it depends on the size of the map your creating, and how much it has to displace. A mesh that originally has 2800 polygons, ends up with more than 400k polygons with default settings. I ended up getting more than 600k polygons on the one I created, here it is:

Height Map, or Straight to Normal?

Now that the detail has been added in, it's time to go one of two ways, if you have maya6 and not too too many polygons on your final mesh, you can use the Transfer Surface Information tool:

F2-F5 changes your maya menu's to Animation, Modeling, Dynamics and Rendering respectively

Then there's the other way, a hight map. This one takes a little longer to do, so if you have an earlier version of maya, or transfer surface information takes too long (or crash's maya), you can do it this way.

You will need to map the UV's of the highpoly wall a different way than the lower poly version. You still need to use a planar map but this time your not trying to project it like a T. This time you want to go along an axis which makes you look THROUGH it. EG- if you had a plane that was flat across the grid, You'll want to project along either the X or Z axis. After the UV's, you'll want to go back into your hypershade and create another lambert, then for the color, you'll want to click on the checker box again, but you wont be applying a file texture. Instead click on Ramp. In your Ramp attributes (Ramp type should be either V Ramp, or U Ramp). You then need to change the color values

The circles select the color, and the boxes remove the color points

Apply the Lambert you just created on the highpoly model, you should see the ramp showing the height already. It doesn't matter which point the colors start at, you can always invert it in photoshop. Now go into your hypershade again, and select your lambert. Show it's attributes and where it says 'Ambient Color' you need to crank that thing up so it's completely white. Here's what I got:

As I'm sure you know, a viewport screenshot like the one above wont work too well. Now what you have to do is a high quality render from an orthographic view. So switch over to the view that has/had your image plane. Go into your RenderGlobals and change a few settings. Under the 'Common' tab, go all the way to the bottom and uncheck the 'turn on default lighting' option, it will mess with your render and you wont get a height map that relates that well to your actual mesh. Depending on what resolution you want your map to be, will determine the size you should render at. For a 512x512 map, I'd suggest doing a render with at least 800 to 1000 pixels (square), and resize it in photoshop. Then you need to click on the maya software tab, and change your settings to match here:

Here is an example of the map I got:

Save the image you get, this is your Height Map.

Back to Photoshop

Now you should have your height map saved to your HD somewhere. Before we continue, you'll need a plugin for photoshop from NVidia (note: you don't need an nvidia card to use them, I'm an ATI boy myself.) you can download them Here. You'll want the Adobe Photoshop Normal Map and DDS Authoring Plug-ins. It's one file that is less than 3mb. Now to restart Photoshop. Load up your height map and start the new filter, it'll be found under the filter menu, a new "nvTools" should be in your list. Under that heading, will be "NormalMapFilter" set it up like I have shown:

If your doing a texture for Doom3, your normal map will have to look backwards, so check the "3D Preview" button, and if it looks like it's going the wrong way your set, if it looks like it's showing the lighting info the way you want it to look inside the game, you'll need to put a check mark by "Invert Y" (This is another reason a hight map is an advantage over Transfer Surface Information). Scale is another important part of this, the scale can make the normal map appear to have more, or less depth, play around with it until it looks right. It's a good idea to save 2 versions of the normal map, so you can view one in maya that will look correct, and have the other one for the doom engine.

To view your results in Maya (if you have Maya6), create a single poly plane (if your texture is square, you can leave it as is, if it's rectangular, set your scale X to 2).

Open your hypershade and create a lambert that will display your normal map. In the lamberts attributes, connect the normal map to the color channel and display it's input/outputs in your hypershades Work Area. Delete the color connection, and middle mouse drag the normal map back to the lambert. Select "other". The Connection editor should pop up.

Attach the "outColor" of your normal map, to the " normalCamera" of the lambert like shown.

Close the connection editor and the hypershade now. While in your perspective viewport hit the "6" button (above TandY) this will set your viewport to show textures, which is needed, then you need to set the viewport to "High Quality Rendering"

You should see your normal map affecting the polygon now, here is a normal map with a specularity map added in the maya viewport:

You can now add a light to your scene to see how it reacts to light. Create>Lights>Point Light... works the best I've found. After you add the light, make sure you hit "7" so the viewport uses lights from the scene instead of default lighting.

This is the result of the color, specularity and normal map in the doom3 Engine:

And there is your height and normal map.