Chapter 12

Normal Mapping
In Chapter 7, we introduced t e x t u r e mapping, w h i c h enabled us to map fine
details from an image onto our triangles. However, our n o r m a l vectors are
s t i l l defined at the coarser v e r t e x level and interpolated over the triangle.
In this chapter, we study a popular method for specifying surface normals at
a higher resolution.
T h e N o r m a l M a p demo for this chapter is available in the download
files.

Objectives:
• To understand w h y we need n o r m a l mapping.
• To discover how n o r m a l maps are stored.
• To learn how n o r m a l maps can be created.
• To d e t e r m i n e the coordinate system the n o r m a l vectors in n o r m a l
maps are stored relative to, and how it relates to the object space
coordinate system of a 3D triangle.
• To learn how to i m p l e m e n t normal mapping in a v e r t e x and p i x e l
shader.

12.1 Motivation
Consider Figure 12.1 f r o m the Cube M a p demo of the preceding chapter
(see the download files). T h e specular highlights on the cone-shaped col-
umns do not look r i g h t — they look unnaturally smooth compared to the
bumpiness of the b r i c k t e x t u r e . T h i s is because the u n d e r l y i n g m e s h g e o m -
e t r y is smooth, and we have m e r e l y applied the image of bumpy bricks over
t h e smooth cylindrical surface. However, the l i g h t i n g calculations are p e r -
f o r m e d based on the m e s h g e o m e t r y (in particular, the interpolated v e r t e x
normals), and not the t e x t u r e image. T h u s the l i g h t i n g is not completely
consistent w i t h the t e x t u r e .

321
 Part III: DirectSD Topics

Figure 12.1: Smooth

specular highlights.

Ideally, we w o u l d tessellate the mesh g e o m e t r y so m u c h that the actual

bumps and crevices of the bricks could be modeled by the u n d e r l y i n g g e o m -
etry. T h e n the lighting and texture could be made consistent. However,
tessellating a mesh to such a resolution is not practical due to the huge
increase in v e r t e x and triangle count.
Another possible solution w o u l d be to bake the l i g h t i n g details directly
into the textures. However, this w i l l not w o r k if the lights are allowed to
move, as the texel colors w i l l remain fixed as the lights move.
Thus our goal is to find a way to i m p l e m e n t dynamic l i g h t i n g such that
the fine details that show up in the t e x t u r e map also show up in the l i g h t i n g .
Since textures provide us w i t h the fine details to begin w i t h , it is natural to
look for a t e x t u r e mapping solution to this problem. Figure 12.2 shows the
same scene w i t h normal mapping; we can see now that the dynamic l i g h t i n g
is m u c h more consistent w i t h the brick t e x t u r e .

Figure 12.2: Bumpy

specular highlights.
 Chapter 12: Normal Mapping 323

12.2 N o r m a l Maps
A normal map is a t e x t u r e , but instead of storing R G B data at each texel, we
store a compressed x-eoordinate,y-coordinate, and z-coordinate in the red
component, green component, and blue component, respectively. These
coordinates define a n o r m a l vector; thus a n o r m a l map stores a normal vec-
t o r at each pixel. Figure 12.3 shows an example of how to visualize a normal
map.

Ni Figure 12.3: Normals stored in

a normal map relative to a
texture space coordinate
system defined by the vectors
T (x-axis), B (y-axis), and N
(z-axis). The T vector runs
right horizontally to the texture
image, the B vector runs down
vertically to the texture image,
and N is orthogonal to the
texture plane.

For i l l u s t r a t i o n , we w i l l assume a 24-bit image format, w h i c h reserves a

byte for each color component, and therefore, each color component can
range from 0 to 255. (A 32-bit format could be employed w h e r e the alpha
component goes unused or stores some other scalar value. Also, a float-
ing-point format could be used in w h i c h no compression is necessary, but
this requires more memory, of course.)

Note: As Figure 12.3 shows, the vectors are generally mostly

aligned with the z-axis. That is, the z-coordinate has the largest mag-
nitude. Consequently, normal maps usually appear mostly blue when
viewed as a color image. This is because the z-coordinate is stored
in the blue channel and since it has the largest magnitude, this color
dominates.

So how do we compress a. unit vector into this format? F i r s t note that, for a
u n i t vector, each coordinate always lies in the range [ - 1 , 1], If we shift, and
scale this range to [0, 1] and m u l t i p l y by 255 and truncate the decimal, the
result w i l l be an integer in the range 0 to 255. T h a t is, if x is a coordinate in
the range [ - 1 , 1], t h e n the integer part of f(x) is an integer in the range 0
to 255, w h e r e / is defined by

/ ( x ) = (0.5x+0.5)-255
 Part III: Direct3D Topics

So to store a u n i t vector in a 24-bit image, we j u s t a p p l y / t o each coordinate

and w r i t e the coordinate to the corresponding color channel in the t e x t u r e
map.
T h e next question is h o w to reverse t h e compression process; that is,
g i v e n a compressed t e x t u r e coordinate in the range 0 to 255, h o w can we
recover its t r u e value in the i n t e r v a l [ - 1 , 1]? T h e answer is to simply i n v e r t
the f u n c t i o n / w h i c h after a l i t t l e thought, can be seen to be:

/" W = Zoo
1
^ : - 1

T h a t is, if x is an integer in the range 0 to 255, t h e n f~ (x) is a floating-

l

point n u m b e r i n t h e range [ - 1 , 1].

We w i l l not have to do the compression process ourselves, as w e w i l l
use a Photoshop p l u g - i n to c o n v e r t images to n o r m a l maps. H o w e v e r , w h e n
we sample a n o r m a l map in a pixel shader, we w i l l have to do part of the
inverse process to uncompress i t . W h e n we sample a n o r m a l map in a
shader like this:

floats normalT = gNormalMap.Sample( gTriLinearSam, pln.texC );

the color vector normal T will have normalized components (>;g, b) such that
0<>;g,b< 1. T h u s , the method has already done part of the uncompressing
w o r k for us (namely the divide by 255, w h i c h transforms an integer in t h e
range 0 to 255 to the floating-point i n t e r v a l [0, 1]). We complete the t r a n s -
formation by s h i f t i n g and scaling each component in [0, lj to [ - 1 , 1] w i t h
the function£: [0, 1] -» [ - 1 , 1] defined by:

g(x) = 2x-1

In code, we apply t h i s function to each color component like t h i s :

// Uncompress each component from [0,1] to [-1,1].

normalT - 2.0f*normalT - l.Of;

T h i s w o r k s because the scalar 1.0 is augmented to the vector ( 1 , 1, 1) so

that the expression makes sense and is done componentwise.

Note: The Photoshop plug-in is available at http://developer.

nvidia.com/object/nv_texturejools.html. There are other tools
available for generating normal maps such as http://www.crazybump.
com/. Also, there are tools that can generate normal maps from
high-resolution meshes (see http://developer.nvidia .com/object/
melody__home.html).
 Chapter 12: Normal Mapping 325

12.3 Texture/Tangent S p a c e
Consider a 3D t e x t u r e mapped triangle. For the sake of discussion, suppose
that there is no distortion in the t e x t u r e mapping; in other words, mapping
the t e x t u r e triangle onto the 3D triangle requires only a r i g i d body transfor-
mation (translation and rotation). Now, suppose that the t e x t u r e is like a
decal. So we pick up the decal, translate i t , and rotate it onto the 3D t r i a n -
gle. Figure 12.4 shows how the t e x t u r e space axes relate to the 3D t r i a n g l e :
T h e y are tangent to the triangle and lie in the plane of the triangle. T h e t e x -
t u r e coordinates of the triangle are, of course, relative to the t e x t u r e space
coordinate system. Incorporating the triangle face n o r m a l N, we obtain a 3D
TBN-hasis in the plane of the triangle that we call texture space or tangent
space. Note that the tangent space generally varies f r o m triangle to triangle
(see Figure 12.5).

Figure 12.4: The relationship

between the texture space of a
triangle and the object space.

Figure 12.5: The texture space

is different for each face of the
box.
326 Part III: DirectSD Topics

Now, as Figure 12.3 shows, the n o r m a l vectors in a n o r m a l map are defined

relative to the t e x t u r e space. B u t our lights are defined in w o r l d space. In
order to do l i g h t i n g , the n o r m a l vectors and lights need to be in the same
space. So our f i r s t step is to relate t h e tangent space coordinate system
w i t h the object space coordinate system the triangle v e r t i c e s are relative
to. Once we are in object space, we can use the w o r l d m a t r i x to get f r o m
object space to w o r l d space (the details of this are covered in the n e x t
section). L e t v,„ v„ and v define the three vertices of a 3D triangle w i t h
2

corresponding t e x t u r e coordinates («,„ v„), (u , v-J, and (u , t> ) that define s 2 2

a triangle in the t e x t u r e plane relative to the t e x t u r e space axes (i.e., T

and B ) . L e t e = v - v and e = v - v„ be t w o edge vectors of the 3D
0 0 l 2

triangle w i t h corresponding t e x t u r e triangle edge vectors (Aw , At;,,) = (1

( « ! - M,„ v - v ) and (Au Av ) = (u - u , v - v ). F r o m Figure 12.4, it is

Y n lt x 2 v 2 0

clear that

e () = A;/ T + ( I Av(lB
e } = Au T + A t ' j B
L

Representing the vectors w i t h coordinates r e l a t i v e to object space, we get

the m a t r i x equation:

e
0,x e
0,y e
0 . :
Au 0 Av {)
X T
y
T z

e
l,x e
l,y e
\,z Au L Af j Br Bz

N o t e that we k n o w the object space coordinates of the triangle vertices;

hence we k n o w the object space coordinates of the edge vectors, so t h e
matrix

7
'0,x e
0,y e
0,z

' l,x -i,y 'l.z

is k n o w n . L i k e w i s e , we k n o w the t e x t u r e coordinates, so the m a t r i x

Au 0 AV 0

AMJ AV 1

is k n o w n . Solving for t h e T and B object space coordinates we get:

T T T' z "A//„ Av ' - l -

t>
' e
0.y
x y
B
x B
y B_ z Au 1 Av {
l,x
e
l,y

1 Av l
'0,x e
0,y e
0,z

Au Av 0 1 — A i ' y A w j — Alt! AUQ '\,x 'l,v

 Chapter 12: Normal Mapping 327

a b
In t h e above, we used the fact that the inverse of a m a t r i x A= is
c d
given by:

1 d -b
A - 1
-
ad—be —c a

N o t e that the vectors T and B are generally not unit length in object space,
and if there is t e x t u r e d i s t o r t i o n , they w i l l not be o r t h o n o r m a l either.
T h e T, B, and N vectors are commonly referred to as the tangent,
binomial (or bitangent), and normal vectors, respectively.

12.4 Vertex Tangent S p a c e

In the previous section, we derived a tangent space per triangle. However,
if w e use this t e x t u r e space for normal mapping, we w i l l get a triangulated
appearance since the tangent space is constant over the face of the triangle.
Therefore, we specify tangent vectors per v e r t e x , and we do the same
averaging t r i c k that we did w i t h v e r t e x normals to approximate a smooth
surface:
1. T h e tangent vector T for an a r b i t r a r y v e r t e x v in a mesh is found by
averaging the tangent vectors of e v e r y triangle in the mesh that shares
the v e r t e x v.
2. T h e bitangent vector B for an a r b i t r a r y v e r t e x v in a mesh is found by
averaging the bitangent vectors of every triangle in the mesh that
shares t h e v e r t e x v.

A f t e r averaging, the TBN-bases w i l l generally need to be orthonormalized,

so that the vectors are m u t u a l l y orthogonal and of u n i t length. T h i s is u s u -
ally done using the Gram-Schmidt procedure. Code is available on the web
for building a p e r - v e r t e x tangent space for an a r b i t r a r y triangle mesh:
http://www.terathon.com/code/tangent.php.
In our system, we w i l l not store the bitangent vector B directly in
memory. Instead, we w i l l compute B = N x T w h e n we need B, w h e r e N is
the usual averaged v e r t e x n o r m a l . Hence, our v e r t e x s t r u c t u r e looks like
this:

struct Vertex

D3DXVEGT0R3 pos;
D3DXVECTOR3 tangent; :

D3DXVECT0R3 normal;
D3DXVECT0R2 texC;
h
For our N o r m a l M a p demo, w e w i l l continue to use the Quad, Box, Cyl i nder,
and Sphere classes. We have updated these classes t o include a tangent
328 Part III: Direct3D Topics

v e c t o r per v e r t e x . T h e object space coordinates of the tangent v e c t o r T is

easily specified at each v e r t e x for t h e Quad and Box (see F i g u r e 12.5). For
the Cyl i nder and Sphere, the tangent vector T at each v e r t e x can be found
by f o r m i n g the vector-valued function of t w o variables P(u, v) of the c y l i n -
der/sphere and c o m p u t i n g dP / du, w h e r e the parameter u is also used as
the M - t e x t u r e coordinate.

12.5 T r a n s f o r m i n g b e t w e e n Tangent
Space a n d O b j e c t Space
At t h i s point, we have an o r t h o n o r m a l T B N - b a s i s at each v e r t e x in a m e s h .
Moreover, w e have the coordinates o f the T B N vectors relative t o the
object, space of the mesh. So n o w that we have the coordinates of t h e
T B N - b a s i s relative to the object space coordinate system, we can t r a n s f o r m
coordinates f r o m tangent space to object space w i t h the m a t r i x :

T T Y

M 'bject B X
B Y B Z

N , N .

Since this m a t r i x is orthogonal, its inverse is its transpose. T h u s , t h e

change of coordinate m a t r i x f r o m object space to tangent space is:

T R B V AT,

^h.iiigmt — ^-object ~ ^object ~ B X N ,

B,

In our shader program, we w i l l actually want to t r a n s f o r m the n o r m a l

v e c t o r f r o m tangent space to w o r l d space for l i g h t i n g . One way w o u l d be to
t r a n s f o r m the n o r m a l f r o m tangent space to object space f i r s t , and t h e n use
the w o r l d m a t r i x to t r a n s f o r m f r o m object space to w o r l d space:

1 1
world ~ (n tcmgmt Mobject ) M m r U

However, since m a t r i x m u l t i p l i c a t i o n is associative, we can do it l i k e t h i s :

n, 'Hd tangent (M M ) ohhTt wor!d

A n d note that

'[' — r r; r; x

^ object ^ ,rld w< ~~ *- B -* M — — B — = B ' X B ; B \

_«-N-» _«-N'-» iv; N : N :

where T = T M . ,, B = B M uw h m M , and N' = N • M l f v r l l l . So to go f r o m

tangent space d i r e c t l y to w o r l d space, we j u s t have to describe the tangent
 Chapter 12: Normal Mapping 329

basis in w o r l d coordinates, which can be done by t r a n s f o r m i n g the

T B N - b a s i s from object space coordinates to w o r l d space coordinates.

Note: We will only be interested in transforming vectors (not

points). Thus, we only need a 3 x 3 matrix. Recall t h a t the fourth row
of an affine matrix is for translation, but we do not translate vectors.

12.6 Shader Code

We summarize the general process for normal mapping:
1. Create the desired n o r m a l maps from some art or u t i l i t y program and
store t h e m in an image file. Create 2D textures from these files w h e n
the program is initialized.
2. For each triangle, compute the tangent vector T. Obtain a p e r - v e r t e x
tangent vector for each v e r t e x v in a m e s h by averaging the tangent
vectors of every triangle in the mesh that shares the v e r t e x v. (In our
demo, we use simple g e o m e t r y and are able to specify the tangent vec-
tors directly, but this averaging process w o u l d need to be done if using
a r b i t r a r y triangle meshes made in a 3D modeling program.)
3. In the v e r t e x shader, transform the vertex n o r m a l and tangent vector
to w o r l d space and output the results to the pixel shader.
4. U s i n g the interpolated tangent vector and n o r m a l vector, we build the
TBN-basis at each pixel point on the surface of the triangle. We use
this basis to transform the sampled n o r m a l vector from the normal map
from tangent space to w o r l d space. We then have a w o r l d space n o r m a l
vector f r o m the normal map to use for our usual l i g h t i n g calculations.

T h e entire n o r m a l mapping effect is shown below for completeness, w i t h

the parts relevant to n o r m a l mapping in bold.

linclude "lighthelper.fx"

cbuffer cbPerFrame
{ ' . • ; • . ' ;• . y ; \ » ;
: ..-
Light gLight; :

floats gEyePosW;
. }; ; ' .• •. • • • • • ••• • •••• • ' , .'

cbuffer cbPerObject
{ . : '. • . •: • :
•• • .

float4x4 gWorld;
f 1 oat4x4 gWVP;
float4x4 gTexMtx;
float4 gReflectMtrl;
bool gCubeMapEnabT ed;
};
Part III: DirectSD Topics

II Nonnumeric values cannot be added to a cbuffer.

Texture2D gDiffuseMap;
Texture2D gSpecMap;
Texture2D gNormalMap;
TextureCube gCubeMap;

SamplerState gTriLinearSam
{
Filter = MIN_MAG_MIP_LIN EAR;
Addressll = Wrap; ,
AddressV = Wrap;

struct VS IN

float:3 post POSITION;

f l o a t 3 tangentL TANGENT;
float3 normalL NORMAL;
float2 texC TEXCOORD;

struct VS OUT

float.4 posH : SVJOSITION;

floats posW : POSITION;
floats tangentW : TANGENT;
float3 normalW : NORMAL;
float2 texC : TEXCOORD;

VS_OUT VS(VS_IN vln)

VS_0UT vOut;

// Transform to world space space.

vOut.posW = mul(float4(vIn.posL, l.Of), gWorld);
vOut.tangentW = mul (f1oat4(vIn.tangentL, O.Of), gWorld);
vOut.normalW = mul(float4(vIn.normalL, O.Of), gWorld);

// Transform to homogeneous clip space.

vOut.posH = mul(float4(vIn.post, l.Of), gWVP);

// Output vertex attributes for interpolation across triangle.

vOut.texC = mul(float4(vIn.texC, O.Of, l.Of), gTexMtx);

return vOut;
1
float4 PS(VS_0UT pin) : SVJarget
{
float4 diffuse = gDiffuseMap.Sample( gTriLinearSam, pln.texC );

// Kill transparent pixels.

 Chapter 12: Normal Mapping 3

clip(diffuse.a - 0.15f);

float4 spec = gSpecMap.Sample{ gTriLinearSam, pln.texC );

float3 normalT = gNormalMap.Sample( gTriLinearSam, pln.texC ) ;

// Map [0,1] -> [0,2563 :

spec.a *= 256.Of;

// Uncompress each component from [0,1] to [-1,1].

normalT = 2.Of*normalT - l.Of;

// build orthonormal b a s i s
f l o a t s N = normalize(pin.normalW);
f l o a t s T = norma Iize(pin.tangentW - dot(pIn.tangentW, N)*N);
f l o a t s B = cross(N,T);

float3x3 TBN = f l o a t 3 x 3 ( I , B, N);

// Transform from tangent space to world space.

f l o a t 3 bumpedNormalW - normalize(mul(normalT, TBN));

// Compute the l i t color for this pixel using normal from normal map
Surf acelnfo v * {pln.posW, bumpedNormalW, diffuse,: spee); :

floats litColor = ParaTlelLight(v, gLight, gEyePosW);

[branch]
iff gCubeMapEnabled )
{ , • ' /• • : ' '

floats incident pln.posW * .gEyePosW* v

s

floats refW - refiect(incident, bumpedNormalW):;

float4 reflectedColor - gCubeMap.SamplefgTriLinearSam, refW);
litColor +- (gReflectMtrl*reflectedCOlor).rgb;
} ' • ,•" • \ \ :" .

return float4(litColor, diffuse,*);

techniquelO NormalMapTech
{
pass' PO • •

SetVertexShader( CompileShaderf vs_4_0, VS() ) );

Set6eometryShader( NULL );
SetPixelShader( CompileShader( ps_4_0, PS() ) );

Two lines that might not be clear are these:

floats N = normalize(pIn.normalW);
float3 T = normalize(pln.tangentW - dot(pin.tangentW, N)*N);
332 Part lit: Direct3D Topics

A f t e r the interpolation, the tangent vector and n o r m a l vector may not be

o r t h o n o r m a l . T h i s code makes sure T is o r t h o n o r m a l to N by subtracting
off any component of T along the direction N (see Figure 12.6).

N Figure 12.6: Since ||N| = 1 ,

proj (T) =(T-N)Ni The vector
N

T - p r o j ( T ) is the portion of T
N

orthogonal to N.

T - (N • T ) N

12.7 S u m m a r y
• T h e strategy of normal mapping is to t e x t u r e our polygons w i t h n o r m a l
maps. We then have per-pixel normals, w h i c h capture the fine details of
a surface like bumps, scratches, and crevices. We t h e n use these
per-pixel normals from the n o r m a l map in our l i g h t i n g calculations,
instead of the interpolated v e r t e x normals.
• A normal map is a t e x t u r e , but instead of storing RGB data at each
texel, we store a compressed ^-coordinate, y-coordinate, and
z-coordinate in the r e d component, green component, and blue
component, respectively. We use various tools to generate n o r m a l
maps, such as the ones located at http://developer.nvidia.conV
objecVnv_textirre_tools.html, http://www.crazybump.com/, and
http://developer.nvidia.com/object/melody_home.html.
• T h e coordinates of the normals in a normal map are relative to the
t e x t u r e space coordinate system. Consequently, to do l i g h t i n g
calculations, we need to transform the n o r m a l from the t e x t u r e space
to the w o r l d space so that the lights and normals are in the same
coordinate system. T h e TBN-bases built at each v e r t e x facilitate the
transformation from t e x t u r e space to w o r l d space.
 Chapter 12: Normal Mapping 333

12.8 E x e r c i s e s
1. Download the N V I D I A n o r m a l map p l u g - i n Chttpr/'/devel-
oper.nvidia.comA)bject/nv_texture_tools.html) and experiment w i t h
m a k i n g different n o r m a ! maps w i t h i t . T r y y o u r n o r m a l maps out i n this
chapter's demo application.
2. Investigate D3DXlOComputeNormalMap and use it to create a normal map
f r o m a heightmap. Essentially, a heightmap is a grayscale image that
defines the heights of a surface (see §16.1). Use D3DXlQSaveTexture-
ToFi 1 e to export y o u r normal map to file as a Windows bitmap image
(D3DX10_IFF_BMP).