[Solved] Issue with lighting/transforming normals. Using custom Effect/Shader (Basic 3D question)


So basically I’ve been tearing my hair out getting some really basic lighting in 3D to work. I don’t think this is specific to Monogame, as I remember having similar issues when I tried this an old OpenGL project.

In a nutshell, I’m having (I think) issues with translating my normals on models.

The Image below shows what I mean. I have a light positioned at the camera. The cube in the middle is rotating. Looks mostly fine.
The other cube is orbiting the central one and you can see the front face is dark where it should be lit.

I’ve been tearing my hair out trying to work out why this is happening. Its just a simple shader with ambient and diffuse lighting, no specular.

I’m using the Monogame Crossplatform template. My drawing code is as follows:

public  void Draw(GraphicsDevice g, Effect effect)

            Matrix world = Matrix.Identity;

            world *= Matrix.CreateFromQuaternion(mRotation);           
            world *= Matrix.CreateScale(mScale);
            world *= Matrix.CreateTranslation(mPosition);



            foreach (EffectPass pass in effect.CurrentTechnique.Passes)
                g.DrawPrimitives(PrimitiveType.TriangleList, 0, mShape.VertexCount);

The shader code I adapted from Riener’s tutorial:

float4x4 World;
float4x4 View;
float4x4 Projection;      
float3x3 NormalMat; //if a 4x4 only uses 3x3 Monogame will error due to HLSL optimizer removing final row

float3 LightSourcePosition;

bool IsLightSource;

struct VertexShaderInput
    float4 Position : POSITION;    
    float4 Color : COLOR0;
    float3 Normal : NORMAL0;

struct VertexShaderOutput
    float4 Position : POSITION0; //cannot be passed pixel shader in ps_2_0    
    float4 PosOut : TEXCOORD1; //Label as texture to pass position to pixel shader.
    float4 Color : COLOR0;
    float3 Normal : TEXCOORD0;

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
    VertexShaderOutput output;
    float4 worldPosition = mul(input.Position, World);
    float4 viewPosition = mul(worldPosition, View);
    output.Position = mul(viewPosition, Projection);
    output.PosOut = output.Position;

    output.Color = input.Color;
    output.Normal = normalize(mul(input.Normal, NormalMat));
    return output;

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
    float3 ambientColor = input.Color.xyz * 0.1f;

    if (IsLightSource == true)
        return input.Color;

    float3 lightDirection = normalize(LightSourcePosition - input.PosOut.xyz);

    float diffuseIntensity = max(dot(input.Normal, lightDirection), 0);

    float3 diffuseColor = input.Color.xyz * diffuseIntensity;     

    return float4(ambientColor + diffuseColor,1);

technique Ambient
    pass Pass1
        VertexShader = compile vs_2_0 VertexShaderFunction();
        PixelShader = compile ps_2_0 PixelShaderFunction();

As I understand it with normals, you multiply them by a transposed, inverse of the world matrix to remove the scaling. My understanding is sketchy (as the results show) so any insight into what I’m doing wrong would be great.

This isn’t an issue with Monogame (I don’t think, but just in case I’m running Win7x64 with a gtx950).

Any help or replies are very much appreciated.

(As an aside): I made an account with my email previously but it won’t let me sign in, will only let me make a new account, hence this new account. Any ideas on that would be appreciated too.

Where did you see that you need to multiply by the inverse transpose? In Riemer’s tutorial, other tutorials, and my own working code, I see multiplying by the 3x3 world matrix. This makes sense, because normals start in object space, just like vertex position, and you want to get them into world space.

It was from an Opengl tutorial, just above the section on specular lighting.
I must’ve misunderstood.
I have tried the above code with passing the world matrix to a float3x3 and yielding identical results.

Oh I forgot about that. I think you only need to do that if you use non-uniform scaling, as it mentions in the previous paragraphs. If you follow that link (to this article), it gives an in-depth explanation.

I’ll take another look.

Yeah you’re right. In this case I am using uniform scaling so it would make sense that I’m getting the same results with either method.

I’ll have a read through the article. I read through the other one you linked as well, thank you.

I’ve attached another image, in this image the light is directly below the camera. I moved the camera up (Y axis) and all the lighting changed. I’m only showing as it might give a clue to what I’m doing wrong. The lighting should not have changed. edit: it looks like specular lighting but it isn’t. Its still only doing calculation for diffuse.

Your LightDirection is calculated by subtracting PosOut (in clip space) from LightSourcePosition (presumably world space). If you look at the tutorial you linked, they use FragPos (in world space) instead of PosOut.

At least for this simple shader, you shouldn’t need the clip space position again, so maybe just try setting output.PosOut = worldPosition; instead.

EDIT: This would also explain why lighting changes when you move the camera, because the shader is confusing world coordinates with screen coordinates.

1 Like

Yeah, I think that is it. I’ll have a twiddle and a read, then post back. Thank you.

Edit: I am such a doofus. You nailed it and it of course makes complete sense as to why it was completely wrong. Also explains why ( regardless of graphics API) that I had the same problem.
Can’t thank you enough.

Edit2: Fixed

1 Like