Hardware Instance adding an extra Matrix to the custom VertexDeclaration

Morgan_Sundin · December 16, 2017, 7:13pm

Hi again, I thought I had this one but nope… so I’m updating my question with new information here.

I have a vertex declaration, looks like this

static VertexDeclaration instanceVertexDeclaration = new VertexDeclaration
    (
        new VertexElement(0, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 0),
        new VertexElement(sizeof(float) * 4, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 1),
        new VertexElement(sizeof(float) * 8, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 2),
        new VertexElement(sizeof(float) * 12, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 3)
    );

These four BlendWeight 0-3 are used as a Matrix…
but, what if I would like to add one more, I can’t do this…

static VertexDeclaration instanceVertexDeclaration = new VertexDeclaration
    (
        new VertexElement(0, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 0),
        new VertexElement(sizeof(float) * 4, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 1),
        new VertexElement(sizeof(float) * 8, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 2),
        new VertexElement(sizeof(float) * 12, VertexElementFormat.Vector4, VertexElementUsage.BlendWeight, 3),
         new VertexElement(sizeof(float) * 16, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 0),
        new VertexElement(sizeof(float) * 20, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 1),
        new VertexElement(sizeof(float) * 24, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 2),
        new VertexElement(sizeof(float) * 28, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 3)
    );

And in the Shader

VertexShaderOutput HardwareInstancingVertexShader(VertexShaderInput input,
float4x4 instanceTransform : BLENDWEIGHT,
float4x4 instanceTransform2 : TEXCOORD)
{

But nothings working it’s always zero, instanceTransform is working but not the Matrice Nr 2!?
And yes I have tried to name it TEXCOORD0 but no success.

I’ve asked about this before and I thought I had it but I was wrong because I’ve noticed now that it is always zero. Isn’t this supposed to work sending two Matrices into the Shader??

I’m not using DirectX so I’m trying to find out if there’s a bug in the OpenGL part, or am

Regards, Morgan

EDIT EDIT EDIT EDIT EDIT
UPDATE UPDATE UPDATE
But still not working.

Well here’s everything

My HLSL code…

// Matrices
float4x4 World;					// The world matrix, where in 3D space the object will be placed.
float4x4 View;					// The camera view
float4x4 Projection;			// The frustum
float xLightPower;				// The power of the light
float xAmbient;					// The ambient surrounding light

// For testing
float4x4 xLightsViewProjection1;// The view of the light and its frustum, already multiplic together in the app
float3 xLightPos1;				// The position of the light
float4x4 xLightsViewProjection2;// The view of the light and its frustum, already multiplic together in the app
float3 xLightPos2;				// The position of the light

// The texture sampler that will hold
// the information about the Texture2D
// that we send from the Application
Texture xTexture;
sampler TextureSampler = sampler_state { texture = <xTexture>; magfilter = LINEAR; minfilter = LINEAR; 
mipfilter = LINEAR; AddressU = mirror; AddressV = mirror; };

// The texture of the light
// like if we want the light to cast
// an image like if it's lit through
// a churchwindow etc
Texture xLightShape;
sampler LightShapeSampler = sampler_state { texture = <xLightShape>; magfilter = LINEAR; minfilter = 
LINEAR; mipfilter = LINEAR; AddressU = clamp; AddressV = clamp; };

// -------------------------------------
// Let's create the scene
// -------------------------------------

// The input object that will hold
// all the data needed.
struct VertexShaderInput
{
float4 Position : POSITION0;
float3 Normal : NORMAL0;
float2 TextureCoordinate : TEXCOORD0;
};

struct VertexShaderOutput
{
float4 Position				: POSITION0;	// The vertex position again
float4 Position3D           : TEXCOORD2;
float4 Position2DInLight	: TEXCOORD1;
float2 TextureCoordinate	: TEXCOORD0;
float3 Normal               : TEXCOORD3;
};

struct SScenePixelToFrame
{
float4 Color : COLOR0;
};

// Hardware instancing reads the per-instance world transform from a secondary vertex stream.
VertexShaderOutput HardwareInstancingVertexShader(VertexShaderInput input,
float4x4 instanceTransform : TEXCOORD0, float4x4 lightMatrices : TEXCOORD4)
{	
VertexShaderOutput output = (VertexShaderOutput)0;

// To the world we go!
instanceTransform = mul(World, transpose(instanceTransform));

// Apply the objects translation in the world
// to the input.Position that contain the
// X and Y values of the screen coordinate of the current pixel
float4 worldPosition = mul(input.Position, instanceTransform);

// Apply the camera view to it
float4 viewPosition = mul(worldPosition, View);

// And the projection frustum to become the camera screen position
output.Position = mul(viewPosition, Projection);

// And do the same for the light screen pixels
output.Position2DInLight = mul(worldPosition, lightMatrices);

// Calculate the objects in the world vertex normals
output.Normal = normalize(mul(input.Normal, (float3x3)instanceTransform));

// The objects 3D positions is stored
output.Position3D = worldPosition;

// Copy across the input texture coordinate.
output.TextureCoordinate = input.TextureCoordinate;

return output;
}

float DotProduct(float3 lightPos, float3 pos3D, float3 normal)
{
float3 lightDir = normalize(pos3D - lightPos);
return dot(-lightDir, normal);
}

SScenePixelToFrame PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
// Create the output object that will hold the data
SScenePixelToFrame Output = (SScenePixelToFrame)0;

// Texture coordinates have to be between the [0, 1] region,
// so we need a simple remap of point (-1,-1) which has
// to become (0,0), while point (1,1) has to stay (1,1)
float2 ProjectedTexCoords;
ProjectedTexCoords[0] = input.Position2DInLight.x / input.Position2DInLight.w / 2.0f + 0.5f;
ProjectedTexCoords[1] = -input.Position2DInLight.y / input.Position2DInLight.w / 2.0f + 0.5f;

float diffuseLightingFactor = 0;
if ((saturate(ProjectedTexCoords).x == ProjectedTexCoords.x) && (saturate(ProjectedTexCoords).y == ProjectedTexCoords.y))
{
	float realDistance = input.Position2DInLight.z / input.Position2DInLight.w;
	if ((realDistance - 1.0f / 100.0f) <= 0.99f)
	{
		diffuseLightingFactor = DotProduct(xLightPos1, input.Position3D, input.Normal);
		diffuseLightingFactor = saturate(diffuseLightingFactor);
		diffuseLightingFactor *= xLightPower;

		// The light texture that will be projected onto the objects
		float lightTextureFactor = tex2D(LightShapeSampler, ProjectedTexCoords).r;
		diffuseLightingFactor *= lightTextureFactor;
	}
}

// Apply the objects textures
// and add the lightning color and ambient power to the pixel
float4 baseColor = tex2D(TextureSampler, input.TextureCoordinate);
Output.Color = baseColor * (diffuseLightingFactor + xAmbient);

return Output;
}

// Hardware instancing technique.
technique HardwareInstancing
{
pass Pass1
{
	VertexShader = compile vs_3_0 HardwareInstancingVertexShader();
	PixelShader = compile ps_3_0 PixelShaderFunction();
}
}

The Custom Vertex thingy..

public struct CustomVertex1 : IVertexType
{
    Matrix instanceTransform;
    Matrix lightMatrices;

    // To store instance transform matrices in a vertex buffer, we use this custom
    // vertex type which encodes 4x4 matrices as a set of four Vector4 values.
    static VertexDeclaration instanceVertexDeclaration = new VertexDeclaration
    (
        new VertexElement(0, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 0),
        new VertexElement(sizeof(float) * 4, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 1),
        new VertexElement(sizeof(float) * 8, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 2),
        new VertexElement(sizeof(float) * 12, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 3),
        new VertexElement(sizeof(float) * 16, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 4),
        new VertexElement(sizeof(float) * 20, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 5),
        new VertexElement(sizeof(float) * 24, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 6),
        new VertexElement(sizeof(float) * 28, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 7)
    );

    public CustomVertex1(Matrix instanceTransform, Matrix lightMatrices)
    {
        this.lightMatrices = lightMatrices;
        this.instanceTransform = instanceTransform;
    }
    //Public methods for accessing the components of the custom vertex.
    public Matrix InstaceTransform
    {
        get { return this.instanceTransform; }
        set { this.instanceTransform = value; }
    }
    public Matrix LightMatrices
    {
        get { return this.lightMatrices; }
        set { this.lightMatrices = value; }
    }
    VertexDeclaration IVertexType.VertexDeclaration
    {
        get { return instanceVertexDeclaration; }
    }
}

And the code for drawing my little entities on screen

public void drawEntitiesHarware(GraphicsDevice gD, string technique, Vector3 worldCalibrationPoint, Matrix viewMatrix, Matrix projectionMatrix,
        Matrix lightsViewProjectionMatrix, Vector3 lightSourcePosition, Texture2D lightTexture,
        float ambientLightPower, float lightSourcePower)
    {
        // Gather instance transform matrices into a single array.
        if (null == this.mCV || this.mCV.Length != this.mEntities.Count)
            Array.Resize(ref this.mCV, this.mEntities.Count);

        if (this.mEntitiesAreDirty)
        {
            for (int i = 0; i < this.mEntities.Count; i++)
            {
                this.mCV[i].InstaceTransform = Matrix.CreateScale(this.mEntities[i].getScale()) * this.mEntities[i].entityTranslationMatrix * (Matrix.CreateTranslation(worldCalibrationPoint + this.mEntities[i].planetsLocalWorldPosition));
                this.mCV[i].LightMatrices = lightsViewProjectionMatrix;
            }
            this.mEntitiesAreDirty = false;
        }

        if (this.mCV.Length == 0)
            return;

        // If we have more instances than room in our vertex buffer, grow it to the neccessary size.
        if ((this.mInstanceVertexBuffer == null) ||
            (this.mCV.Length > this.mInstanceVertexBuffer.VertexCount))
        {
            if (this.mInstanceVertexBuffer != null)
                this.mInstanceVertexBuffer.Dispose();

            this.mInstanceVertexBuffer = new DynamicVertexBuffer(gD, typeof(CustomVertex1),
                                                           this.mCV.Length, BufferUsage.WriteOnly);
        }

        // Transfer the latest instance transform matrices into the instanceVertexBuffer.
        this.mInstanceVertexBuffer.SetData<CustomVertex1>(this.mCV);

        int tt = 0;
        foreach (ModelMesh mesh in this.mModel.Meshes)
        {
            foreach (ModelMeshPart meshPart in mesh.MeshParts)
            {
                // Tell the GPU to read from both the model vertex buffer plus our instanceVertexBuffer.
                gD.SetVertexBuffers(
                    new VertexBufferBinding(meshPart.VertexBuffer, meshPart.VertexOffset, 0),
                    new VertexBufferBinding(this.mInstanceVertexBuffer, 0, 1)
                );

                gD.Indices = meshPart.IndexBuffer;

                // Set up the instance rendering effect.
                Effect effect = meshPart.Effect;
                Matrix worldMatrix = this.mModelTransforms[mesh.ParentBone.Index];
                effect.CurrentTechnique = effect.Techniques[technique];
                effect.Parameters["World"].SetValue(worldMatrix);
                effect.Parameters["View"].SetValue(viewMatrix);
                effect.Parameters["Projection"].SetValue(projectionMatrix);
                effect.Parameters["xTexture"].SetValue(this.mModelTextures[tt++]);
                effect.Parameters["xLightPos1"].SetValue(lightSourcePosition);
                effect.Parameters["xLightPower"].SetValue(lightSourcePower);
                effect.Parameters["xLightShape"].SetValue(lightTexture);
                effect.Parameters["xAmbient"].SetValue(ambientLightPower);
                
                // Draw all the instance copies in a single call.
                foreach (EffectPass pass in effect.CurrentTechnique.Passes)
                {
                    pass.Apply();

                    gD.DrawInstancedPrimitives(PrimitiveType.TriangleList, 0,
                                                   meshPart.StartIndex,
                                                   meshPart.PrimitiveCount, this.mCV.Length);
                }
            }
        }
    }

And in the Draw() function I call

.drawEntitiesHarware(GraphicsDevice, technique, worldCalibrationPoint, viewMatrix, projectionMatrix, lightsViewProjectionMatrix, lightPosTemp, lamppostTextures, ambientPower, lightPower);

If I add the lightsViewProjectionMatrix as a Parameter to the Shader and change this row in the Shader…
output.Position2DInLight = mul(worldPosition, lightMatrices);
To…
output.Position2DInLight = mul(worldPosition, xLightsViewProjection1);

Then everything is looking good with a working frustum, but as long as I’m using my lightMatrices the light’s frustum Matrix is gone.

EDIT: And I’ve changed the BLENDWEIGHT to TEXCOORD instead, even though it was working from the beginning.

And here’s the GLSL code generated for the Vertex Shader…

#ifdef GL_ES
precision highp float;
precision mediump int;
#endif

uniform vec4 vs_uniforms_vec4[12];
uniform vec4 posFixup;
vec4 vs_r0;
vec4 vs_r1;
vec4 vs_r2;
#define vs_c0 vs_uniforms_vec4[0]
#define vs_c1 vs_uniforms_vec4[1]
#define vs_c2 vs_uniforms_vec4[2]
#define vs_c3 vs_uniforms_vec4[3]
#define vs_c4 vs_uniforms_vec4[4]
#define vs_c5 vs_uniforms_vec4[5]
#define vs_c6 vs_uniforms_vec4[6]
#define vs_c7 vs_uniforms_vec4[7]
#define vs_c8 vs_uniforms_vec4[8]
#define vs_c9 vs_uniforms_vec4[9]
#define vs_c10 vs_uniforms_vec4[10]
#define vs_c11 vs_uniforms_vec4[11]
#define vs_o0 gl_Position
attribute vec4 vs_v1;
varying vec4 vTexCoord2;
#define vs_o1 vTexCoord2
attribute vec4 vs_v2;
varying vec4 vTexCoord1;
#define vs_o2 vTexCoord1
attribute vec4 vs_v3;
varying vec4 vTexCoord0;
#define vs_o3 vTexCoord0
attribute vec4 vs_v4;
varying vec4 vTexCoord3;
#define vs_o4 vTexCoord3
attribute vec4 vs_v5;
attribute vec4 vs_v6;
attribute vec4 vs_v7;
attribute vec4 vs_v8;
attribute vec4 vs_v9;
attribute vec4 vs_v10;

void main()
{
vs_r0 = vs_c1 * vs_v1.wwww;
vs_r0 = (vs_c0 * vs_v10.wwww) + vs_r0;
vs_r0 = (vs_c2 * vs_v2.wwww) + vs_r0;
vs_r0 = (vs_c3 * vs_v3.wwww) + vs_r0;
vs_r0.w = dot(vs_v8, vs_r0);
vs_r1 = vs_c1 * vs_v1.xxxx;
vs_r1 = (vs_c0 * vs_v10.xxxx) + vs_r1;
vs_r1 = (vs_c2 * vs_v2.xxxx) + vs_r1;
vs_r1 = (vs_c3 * vs_v3.xxxx) + vs_r1;
vs_r0.x = dot(vs_v8, vs_r1);
vs_r1.x = dot(vs_v9.xyz, vs_r1.xyz);
vs_r2 = vs_c1 * vs_v1.yyyy;
vs_r2 = (vs_c0 * vs_v10.yyyy) + vs_r2;
vs_r2 = (vs_c2 * vs_v2.yyyy) + vs_r2;
vs_r2 = (vs_c3 * vs_v3.yyyy) + vs_r2;
vs_r0.y = dot(vs_v8, vs_r2);
vs_r1.y = dot(vs_v9.xyz, vs_r2.xyz);
vs_r2 = vs_c1 * vs_v1.zzzz;
vs_r2 = (vs_c0 * vs_v10.zzzz) + vs_r2;
vs_r2 = (vs_c2 * vs_v2.zzzz) + vs_r2;
vs_r2 = (vs_c3 * vs_v3.zzzz) + vs_r2;
vs_r0.z = dot(vs_v8, vs_r2);
vs_r1.z = dot(vs_v9.xyz, vs_r2.xyz);
vs_r2.x = dot(vs_r0, vs_c4);
vs_r2.y = dot(vs_r0, vs_c5);
vs_r2.z = dot(vs_r0, vs_c6);
vs_r2.w = dot(vs_r0, vs_c7);
vs_o0.x = dot(vs_r2, vs_c8);
vs_o0.y = dot(vs_r2, vs_c9);
vs_o0.z = dot(vs_r2, vs_c10);
vs_o0.w = dot(vs_r2, vs_c11);
vs_o2.x = dot(vs_r0, vs_v4);
vs_o2.y = dot(vs_r0, vs_v5);
vs_o2.z = dot(vs_r0, vs_v6);
vs_o2.w = dot(vs_r0, vs_v7);
vs_o1 = vs_r0;
vs_r0.x = dot(vs_r1.xyz, vs_r1.xyz);
vs_r0.x = inversesqrt(vs_r0.x);
vs_o4.xyz = vs_r0.xxx * vs_r1.xyz;
vs_o3.xy = vs_v10.xy;
gl_Position.y = gl_Position.y * posFixup.y;
gl_Position.xy += posFixup.zw * gl_Position.ww;
gl_Position.z = gl_Position.z * 2.0 - gl_Position.w;
}

And the Pixel Shader…

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform vec4 ps_uniforms_vec4[3];
const vec4 ps_c3 = vec4(0.5, -0.01, 0.99, 0.0);
const vec4 ps_c4 = vec4(-1.0, -0.0, 0.0, 0.0);
vec4 ps_r0;
vec4 ps_r1;
vec4 ps_r2;
#define ps_c0 ps_uniforms_vec4[0]
#define ps_c1 ps_uniforms_vec4[1]
#define ps_c2 ps_uniforms_vec4[2]
uniform sampler2D ps_s0;
uniform sampler2D ps_s1;
varying vec4 vTexCoord2;
#define ps_v0 vTexCoord2
#define ps_oC0 gl_FragColor
varying vec4 vTexCoord1;
#define ps_v1 vTexCoord1
varying vec4 vTexCoord0;
#define ps_v2 vTexCoord0
varying vec4 vTexCoord3;
#define ps_v3 vTexCoord3

void main()
{
ps_r0.xyz = -ps_c0.xyz + ps_v0.xyz;
ps_r1.xyz = normalize(ps_r0.xyz);
ps_r0.x = clamp(dot(-ps_r1.xyz, ps_v3.xyz), 0.0, 1.0);
ps_r0.x = ps_r0.x * ps_c1.x;
ps_r0.y = 1.0 / ps_v1.w;
ps_r0.z = ps_r0.y * ps_v1.x;
ps_r1.x = (ps_r0.z * ps_c3.x) + ps_c3.x;
ps_r0.z = ps_r0.y * -ps_v1.y;
ps_r0.y = (ps_v1.z * ps_r0.y) + ps_c3.y;
ps_r0.y = -ps_r0.y + ps_c3.z;
ps_r1.y = (ps_r0.z * ps_c3.x) + ps_c3.x;
ps_r2 = texture2D(ps_s1, ps_r1.xy);
ps_r0.x = ps_r0.x * ps_r2.x;
ps_r0.x = ((ps_r0.y >= 0.0) ? ps_r0.x : ps_c3.w);
ps_r0.yz = clamp(ps_r1.xy, vec2(0.0), vec2(1.0));
ps_r0.yz = -ps_r1.xy + ps_r0.yz;
ps_r0.z = ((-abs(ps_r0.z) >= 0.0) ? ps_c4.x : ps_c4.y);
ps_r0.y = ((-abs(ps_r0.y) >= 0.0) ? ps_r0.z : ps_c3.w);
ps_r0.x = ((ps_r0.y >= 0.0) ? ps_c3.w : ps_r0.x);
ps_r0.x = ps_r0.x + ps_c2.x;
ps_r1 = texture2D(ps_s0, ps_v2.xy);
ps_oC0 = ps_r0.xxxx * ps_r1;
}

jnoyola · December 16, 2017, 9:07pm

According to your VertexDeclaration, it looks like instanceColor in your shader should be a float3.

Morgan_Sundin · December 16, 2017, 10:13pm

Thanks for telling me, but I can’t change it because the Output.Color is a float4 returning from the Pixel Shader function, it complains when building that it can’t convert float3 to float4. Therefore I decided to leave it float4 even if the last 4th part in the vector isn’t used.

jnoyola · December 16, 2017, 11:43pm

But even if it’s not being used, I think it would still screw up the offsets of the other elements that follow, e.g. your second matrix. If you can’t change it in your shader, can you change the color in your CustomVertex1 to be a Vector4, and increment the following offsets accordingly?

markus · December 17, 2017, 9:27pm

You can convert from float3 to float4 manually in the shader
float4 someFloat4 = float4(someFloat3, 0);

Morgan_Sundin · December 17, 2017, 10:33pm

Thanks guys, it’s working!
Regards, Morgan

Morgan_Sundin · December 22, 2017, 9:57am

I’ll have to bump this thread up.

EDIT: I was wrong it’s not working
I’ve edited the question with a new point of view.

Regards, Morgan

jnoyola · December 22, 2017, 10:15am

What platform are you building for? I’ve gotten geometry instancing working with DesktopGL but not iOS.

Also, can you share your definition of VertexShaderInput and VertexShaderOutput? And your pixel shader takes just a VertexShaderOutput as input, right?

Morgan_Sundin · December 22, 2017, 10:33am

I’m building for PC (DesktopGL) only for the moment and it’s not working half ways.

Here are my structs.

struct VertexShaderInput
{
float4 Position : POSITION0;
float3 Normal : NORMAL0;
float2 TextureCoordinate : TEXCOORD0;
};

struct VertexShaderOutput
{
float4 Position				: POSITION0;	// The vertex position again
float4 Position3D           : TEXCOORD2;
float4 Pos2DAsSeenByLight1   : TEXCOORD1;
float4 Pos2DAsSeenByLight2   : TEXCOORD4;
float2 TextureCoordinate	: TEXCOORD0;
float3 Normal               : TEXCOORD3;
};

Yes, my Pixel Shader takes the VertexShaderOutput as input only.

jnoyola · December 22, 2017, 11:09am

Nothing really stands out to my tired eyes, but I have had really strange shader issues recently if there are any variables that aren’t being used, which includes individual columns of matrices since they’re separated in glsl. For example if you do something like this float4 p = mul(v, M); float2 q = p.xy / p.w; then it causes issues because you’re not using p.z which means you’re not using the third column of M. MojoShader optimizes by removing that column variable, screwing up the rest of your variables. Maybe double check that you’re using all columns of your matrices.

Morgan_Sundin · December 22, 2017, 4:30pm

That was a great point/hint!
And now I’m back from testing testing things to see if that could be the case, but I couldn’t find anything solving the issue sadly. I still can’t debunk the idea though, maybe I’m missing something.

Something doesn’t feel right, it’s a little bit harsh but I’ve started to blame Monogame for this
Should I change to DirX to see if that will make any difference? Long shot and I really don’t want to do that.

Regards, Morgan

There’s nothing logical about Shaders, it’s just magical…

jnoyola · December 22, 2017, 5:56pm

I’ve never tried MG with DirectX so I couldn’t tell you if it might help, but here are some other things

Your geometry vertex type definitely matches the struct? Position, Normal, TextureCoordinate
No errors when building the shader with 2MGFX for OpenGL manually?
It definitely worked with the single matrix before? Because BlendWeight is intended for single floats, while Texcoord is intended for float4
How are you checking that the values are 0 when they enter the shader? Is it possible there’s an issue with the shader that’s screwing up the output? Could you post the complete hlsl, and then go in and debug the effect to find the glsl strings inside the vertex shader and pixel shader objects in your pass, in your technique. For example I think it could be CurrentTechnique.Pass[0].pixelShader.glsl or something like that.

Morgan_Sundin · December 22, 2017, 7:19pm

Well here’s everything

My HLSL code…

// Matrices
float4x4 World;					// The world matrix, where in 3D space the object will be placed.
float4x4 View;					// The camera view
float4x4 Projection;			// The frustum
float xLightPower;				// The power of the light
float xAmbient;					// The ambient surrounding light

// For testing
float4x4 xLightsViewProjection1;// The view of the light and its frustum, already multiplic together in the app
float3 xLightPos1;				// The position of the light
float4x4 xLightsViewProjection2;// The view of the light and its frustum, already multiplic together in the app
float3 xLightPos2;				// The position of the light

// The texture sampler that will hold
// the information about the Texture2D
// that we send from the Application
Texture xTexture;
sampler TextureSampler = sampler_state { texture = <xTexture>; magfilter = LINEAR; minfilter = LINEAR; 
mipfilter = LINEAR; AddressU = mirror; AddressV = mirror; };

// The texture of the light
// like if we want the light to cast
// an image like if it's lit through
// a churchwindow etc
Texture xLightShape;
sampler LightShapeSampler = sampler_state { texture = <xLightShape>; magfilter = LINEAR; minfilter = 
LINEAR; mipfilter = LINEAR; AddressU = clamp; AddressV = clamp; };

// -------------------------------------
// Let's create the scene
// -------------------------------------

// The input object that will hold
// all the data needed.
struct VertexShaderInput
{
float4 Position : POSITION0;
float3 Normal : NORMAL0;
float2 TextureCoordinate : TEXCOORD0;
};

struct VertexShaderOutput
{
float4 Position				: POSITION0;	// The vertex position again
float4 Position3D           : TEXCOORD2;
float4 Position2DInLight	: TEXCOORD1;
float2 TextureCoordinate	: TEXCOORD0;
float3 Normal               : TEXCOORD3;
};

struct SScenePixelToFrame
{
float4 Color : COLOR0;
};

// Hardware instancing reads the per-instance world transform from a secondary vertex stream.
VertexShaderOutput HardwareInstancingVertexShader(VertexShaderInput input,
float4x4 instanceTransform : TEXCOORD0, float4x4 lightMatrices : TEXCOORD4)
{	
VertexShaderOutput output = (VertexShaderOutput)0;

// To the world we go!
instanceTransform = mul(World, transpose(instanceTransform));

// Apply the objects translation in the world
// to the input.Position that contain the
// X and Y values of the screen coordinate of the current pixel
float4 worldPosition = mul(input.Position, instanceTransform);

// Apply the camera view to it
float4 viewPosition = mul(worldPosition, View);

// And the projection frustum to become the camera screen position
output.Position = mul(viewPosition, Projection);

// And do the same for the light screen pixels
output.Position2DInLight = mul(worldPosition, lightMatrices);

// Calculate the objects in the world vertex normals
output.Normal = normalize(mul(input.Normal, (float3x3)instanceTransform));

// The objects 3D positions is stored
output.Position3D = worldPosition;

// Copy across the input texture coordinate.
output.TextureCoordinate = input.TextureCoordinate;

return output;
}

float DotProduct(float3 lightPos, float3 pos3D, float3 normal)
{
float3 lightDir = normalize(pos3D - lightPos);
return dot(-lightDir, normal);
}

SScenePixelToFrame PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
// Create the output object that will hold the data
SScenePixelToFrame Output = (SScenePixelToFrame)0;

// Texture coordinates have to be between the [0, 1] region,
// so we need a simple remap of point (-1,-1) which has
// to become (0,0), while point (1,1) has to stay (1,1)
float2 ProjectedTexCoords;
ProjectedTexCoords[0] = input.Position2DInLight.x / input.Position2DInLight.w / 2.0f + 0.5f;
ProjectedTexCoords[1] = -input.Position2DInLight.y / input.Position2DInLight.w / 2.0f + 0.5f;

float diffuseLightingFactor = 0;
if ((saturate(ProjectedTexCoords).x == ProjectedTexCoords.x) && (saturate(ProjectedTexCoords).y == ProjectedTexCoords.y))
{
	float realDistance = input.Position2DInLight.z / input.Position2DInLight.w;
	if ((realDistance - 1.0f / 100.0f) <= 0.99f)
	{
		diffuseLightingFactor = DotProduct(xLightPos1, input.Position3D, input.Normal);
		diffuseLightingFactor = saturate(diffuseLightingFactor);
		diffuseLightingFactor *= xLightPower;

		// The light texture that will be projected onto the objects
		float lightTextureFactor = tex2D(LightShapeSampler, ProjectedTexCoords).r;
		diffuseLightingFactor *= lightTextureFactor;
	}
}

// Apply the objects textures
// and add the lightning color and ambient power to the pixel
float4 baseColor = tex2D(TextureSampler, input.TextureCoordinate);
Output.Color = baseColor * (diffuseLightingFactor + xAmbient);

return Output;
}

// Hardware instancing technique.
technique HardwareInstancing
{
pass Pass1
{
	VertexShader = compile vs_3_0 HardwareInstancingVertexShader();
	PixelShader = compile ps_3_0 PixelShaderFunction();
}
}

The Custom Vertex thingy..

public struct CustomVertex1 : IVertexType
{
    Matrix instanceTransform;
    Matrix lightMatrices;

    // To store instance transform matrices in a vertex buffer, we use this custom
    // vertex type which encodes 4x4 matrices as a set of four Vector4 values.
    static VertexDeclaration instanceVertexDeclaration = new VertexDeclaration
    (
        new VertexElement(0, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 0),
        new VertexElement(sizeof(float) * 4, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 1),
        new VertexElement(sizeof(float) * 8, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 2),
        new VertexElement(sizeof(float) * 12, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 3),
        new VertexElement(sizeof(float) * 16, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 4),
        new VertexElement(sizeof(float) * 20, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 5),
        new VertexElement(sizeof(float) * 24, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 6),
        new VertexElement(sizeof(float) * 28, VertexElementFormat.Vector4, VertexElementUsage.TextureCoordinate, 7)
    );

    public CustomVertex1(Matrix instanceTransform, Matrix lightMatrices)
    {
        this.lightMatrices = lightMatrices;
        this.instanceTransform = instanceTransform;
    }
    //Public methods for accessing the components of the custom vertex.
    public Matrix InstaceTransform
    {
        get { return this.instanceTransform; }
        set { this.instanceTransform = value; }
    }
    public Matrix LightMatrices
    {
        get { return this.lightMatrices; }
        set { this.lightMatrices = value; }
    }
    VertexDeclaration IVertexType.VertexDeclaration
    {
        get { return instanceVertexDeclaration; }
    }
}

And the code for drawing my little entities on screen

public void drawEntitiesHarware(GraphicsDevice gD, string technique, Vector3 worldCalibrationPoint, Matrix viewMatrix, Matrix projectionMatrix,
        Matrix lightsViewProjectionMatrix, Vector3 lightSourcePosition, Texture2D lightTexture,
        float ambientLightPower, float lightSourcePower)
    {
        // Gather instance transform matrices into a single array.
        if (null == this.mCV || this.mCV.Length != this.mEntities.Count)
            Array.Resize(ref this.mCV, this.mEntities.Count);

        if (this.mEntitiesAreDirty)
        {
            for (int i = 0; i < this.mEntities.Count; i++)
            {
                this.mCV[i].InstaceTransform = Matrix.CreateScale(this.mEntities[i].getScale()) * this.mEntities[i].entityTranslationMatrix * (Matrix.CreateTranslation(worldCalibrationPoint + this.mEntities[i].planetsLocalWorldPosition));
                this.mCV[i].LightMatrices = lightsViewProjectionMatrix;
            }
            this.mEntitiesAreDirty = false;
        }

        if (this.mCV.Length == 0)
            return;

        // If we have more instances than room in our vertex buffer, grow it to the neccessary size.
        if ((this.mInstanceVertexBuffer == null) ||
            (this.mCV.Length > this.mInstanceVertexBuffer.VertexCount))
        {
            if (this.mInstanceVertexBuffer != null)
                this.mInstanceVertexBuffer.Dispose();

            this.mInstanceVertexBuffer = new DynamicVertexBuffer(gD, typeof(CustomVertex1),
                                                           this.mCV.Length, BufferUsage.WriteOnly);
        }

        // Transfer the latest instance transform matrices into the instanceVertexBuffer.
        this.mInstanceVertexBuffer.SetData<CustomVertex1>(this.mCV);

        int tt = 0;
        foreach (ModelMesh mesh in this.mModel.Meshes)
        {
            foreach (ModelMeshPart meshPart in mesh.MeshParts)
            {
                // Tell the GPU to read from both the model vertex buffer plus our instanceVertexBuffer.
                gD.SetVertexBuffers(
                    new VertexBufferBinding(meshPart.VertexBuffer, meshPart.VertexOffset, 0),
                    new VertexBufferBinding(this.mInstanceVertexBuffer, 0, 1)
                );

                gD.Indices = meshPart.IndexBuffer;

                // Set up the instance rendering effect.
                Effect effect = meshPart.Effect;
                Matrix worldMatrix = this.mModelTransforms[mesh.ParentBone.Index];
                effect.CurrentTechnique = effect.Techniques[technique];
                effect.Parameters["World"].SetValue(worldMatrix);
                effect.Parameters["View"].SetValue(viewMatrix);
                effect.Parameters["Projection"].SetValue(projectionMatrix);
                effect.Parameters["xTexture"].SetValue(this.mModelTextures[tt++]);
                effect.Parameters["xLightPos1"].SetValue(lightSourcePosition);
                effect.Parameters["xLightPower"].SetValue(lightSourcePower);
                effect.Parameters["xLightShape"].SetValue(lightTexture);
                effect.Parameters["xAmbient"].SetValue(ambientLightPower);
                
                // Draw all the instance copies in a single call.
                foreach (EffectPass pass in effect.CurrentTechnique.Passes)
                {
                    pass.Apply();

                    gD.DrawInstancedPrimitives(PrimitiveType.TriangleList, 0,
                                                   meshPart.StartIndex,
                                                   meshPart.PrimitiveCount, this.mCV.Length);
                }
            }
        }
    }

And in the Draw() function I call

.drawEntitiesHarware(GraphicsDevice, technique, worldCalibrationPoint, viewMatrix, projectionMatrix, lightsViewProjectionMatrix, lightPosTemp, lamppostTextures, ambientPower, lightPower);

If I add the lightsViewProjectionMatrix as a Parameter to the Shader and change this row in the Shader…
output.Position2DInLight = mul(worldPosition, lightMatrices);
To…
output.Position2DInLight = mul(worldPosition, xLightsViewProjection1);

Then everything is looking good with a working frustum, but as long as I’m using my lightMatrices the light’s frustum Matrix is gone.

EDIT: And I’ve changed the BLENDWEIGHT to TEXCOORD instead, even though it was working from the beginning.

And here’s the GLSL code generated for the Vertex Shader…

#ifdef GL_ES
precision highp float;
precision mediump int;
#endif

uniform vec4 vs_uniforms_vec4[12];
uniform vec4 posFixup;
vec4 vs_r0;
vec4 vs_r1;
vec4 vs_r2;
#define vs_c0 vs_uniforms_vec4[0]
#define vs_c1 vs_uniforms_vec4[1]
#define vs_c2 vs_uniforms_vec4[2]
#define vs_c3 vs_uniforms_vec4[3]
#define vs_c4 vs_uniforms_vec4[4]
#define vs_c5 vs_uniforms_vec4[5]
#define vs_c6 vs_uniforms_vec4[6]
#define vs_c7 vs_uniforms_vec4[7]
#define vs_c8 vs_uniforms_vec4[8]
#define vs_c9 vs_uniforms_vec4[9]
#define vs_c10 vs_uniforms_vec4[10]
#define vs_c11 vs_uniforms_vec4[11]
#define vs_o0 gl_Position
attribute vec4 vs_v1;
varying vec4 vTexCoord2;
#define vs_o1 vTexCoord2
attribute vec4 vs_v2;
varying vec4 vTexCoord1;
#define vs_o2 vTexCoord1
attribute vec4 vs_v3;
varying vec4 vTexCoord0;
#define vs_o3 vTexCoord0
attribute vec4 vs_v4;
varying vec4 vTexCoord3;
#define vs_o4 vTexCoord3
attribute vec4 vs_v5;
attribute vec4 vs_v6;
attribute vec4 vs_v7;
attribute vec4 vs_v8;
attribute vec4 vs_v9;
attribute vec4 vs_v10;

void main()
{
vs_r0 = vs_c1 * vs_v1.wwww;
vs_r0 = (vs_c0 * vs_v10.wwww) + vs_r0;
vs_r0 = (vs_c2 * vs_v2.wwww) + vs_r0;
vs_r0 = (vs_c3 * vs_v3.wwww) + vs_r0;
vs_r0.w = dot(vs_v8, vs_r0);
vs_r1 = vs_c1 * vs_v1.xxxx;
vs_r1 = (vs_c0 * vs_v10.xxxx) + vs_r1;
vs_r1 = (vs_c2 * vs_v2.xxxx) + vs_r1;
vs_r1 = (vs_c3 * vs_v3.xxxx) + vs_r1;
vs_r0.x = dot(vs_v8, vs_r1);
vs_r1.x = dot(vs_v9.xyz, vs_r1.xyz);
vs_r2 = vs_c1 * vs_v1.yyyy;
vs_r2 = (vs_c0 * vs_v10.yyyy) + vs_r2;
vs_r2 = (vs_c2 * vs_v2.yyyy) + vs_r2;
vs_r2 = (vs_c3 * vs_v3.yyyy) + vs_r2;
vs_r0.y = dot(vs_v8, vs_r2);
vs_r1.y = dot(vs_v9.xyz, vs_r2.xyz);
vs_r2 = vs_c1 * vs_v1.zzzz;
vs_r2 = (vs_c0 * vs_v10.zzzz) + vs_r2;
vs_r2 = (vs_c2 * vs_v2.zzzz) + vs_r2;
vs_r2 = (vs_c3 * vs_v3.zzzz) + vs_r2;
vs_r0.z = dot(vs_v8, vs_r2);
vs_r1.z = dot(vs_v9.xyz, vs_r2.xyz);
vs_r2.x = dot(vs_r0, vs_c4);
vs_r2.y = dot(vs_r0, vs_c5);
vs_r2.z = dot(vs_r0, vs_c6);
vs_r2.w = dot(vs_r0, vs_c7);
vs_o0.x = dot(vs_r2, vs_c8);
vs_o0.y = dot(vs_r2, vs_c9);
vs_o0.z = dot(vs_r2, vs_c10);
vs_o0.w = dot(vs_r2, vs_c11);
vs_o2.x = dot(vs_r0, vs_v4);
vs_o2.y = dot(vs_r0, vs_v5);
vs_o2.z = dot(vs_r0, vs_v6);
vs_o2.w = dot(vs_r0, vs_v7);
vs_o1 = vs_r0;
vs_r0.x = dot(vs_r1.xyz, vs_r1.xyz);
vs_r0.x = inversesqrt(vs_r0.x);
vs_o4.xyz = vs_r0.xxx * vs_r1.xyz;
vs_o3.xy = vs_v10.xy;
gl_Position.y = gl_Position.y * posFixup.y;
gl_Position.xy += posFixup.zw * gl_Position.ww;
gl_Position.z = gl_Position.z * 2.0 - gl_Position.w;
}

And the Pixel Shader…

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform vec4 ps_uniforms_vec4[3];
const vec4 ps_c3 = vec4(0.5, -0.01, 0.99, 0.0);
const vec4 ps_c4 = vec4(-1.0, -0.0, 0.0, 0.0);
vec4 ps_r0;
vec4 ps_r1;
vec4 ps_r2;
#define ps_c0 ps_uniforms_vec4[0]
#define ps_c1 ps_uniforms_vec4[1]
#define ps_c2 ps_uniforms_vec4[2]
uniform sampler2D ps_s0;
uniform sampler2D ps_s1;
varying vec4 vTexCoord2;
#define ps_v0 vTexCoord2
#define ps_oC0 gl_FragColor
varying vec4 vTexCoord1;
#define ps_v1 vTexCoord1
varying vec4 vTexCoord0;
#define ps_v2 vTexCoord0
varying vec4 vTexCoord3;
#define ps_v3 vTexCoord3

void main()
{
ps_r0.xyz = -ps_c0.xyz + ps_v0.xyz;
ps_r1.xyz = normalize(ps_r0.xyz);
ps_r0.x = clamp(dot(-ps_r1.xyz, ps_v3.xyz), 0.0, 1.0);
ps_r0.x = ps_r0.x * ps_c1.x;
ps_r0.y = 1.0 / ps_v1.w;
ps_r0.z = ps_r0.y * ps_v1.x;
ps_r1.x = (ps_r0.z * ps_c3.x) + ps_c3.x;
ps_r0.z = ps_r0.y * -ps_v1.y;
ps_r0.y = (ps_v1.z * ps_r0.y) + ps_c3.y;
ps_r0.y = -ps_r0.y + ps_c3.z;
ps_r1.y = (ps_r0.z * ps_c3.x) + ps_c3.x;
ps_r2 = texture2D(ps_s1, ps_r1.xy);
ps_r0.x = ps_r0.x * ps_r2.x;
ps_r0.x = ((ps_r0.y >= 0.0) ? ps_r0.x : ps_c3.w);
ps_r0.yz = clamp(ps_r1.xy, vec2(0.0), vec2(1.0));
ps_r0.yz = -ps_r1.xy + ps_r0.yz;
ps_r0.z = ((-abs(ps_r0.z) >= 0.0) ? ps_c4.x : ps_c4.y);
ps_r0.y = ((-abs(ps_r0.y) >= 0.0) ? ps_r0.z : ps_c3.w);
ps_r0.x = ((ps_r0.y >= 0.0) ? ps_c3.w : ps_r0.x);
ps_r0.x = ps_r0.x + ps_c2.x;
ps_r1 = texture2D(ps_s0, ps_v2.xy);
ps_oC0 = ps_r0.xxxx * ps_r1;
}

jnoyola · December 22, 2017, 9:03pm

I haven’t seen anything that seems like it would cause this problem yet, but I did notice that your pixel shader output has semantics twice. I guess the compiler allows that. I’ll try looking some more, but it might be helpful to start from the beginning with a minimal shader that uses 2 matrices for each instance and see if it works without all the extra fluff. If so, then add back one thing at a time til it breaks. That’s what I’ve done to debug shader issues in the past.

Morgan_Sundin · December 22, 2017, 9:06pm

Thanks for taking your time helping.

I’ll try that, minimizing my shader and start from the beginning like you said, good idea.

Thanks

Morgan_Sundin · December 22, 2017, 10:03pm

Aaaaaaaaaah such a silly man made ugly thing…

I forgot to transpose(lightMatrices)!!

Now it’s working

Thank you all for the helping hand

Regards, Morgan

jnoyola · December 23, 2017, 12:04am

Aw haha I feel like I led you one a goose chase. Glad you found it.

Morgan_Sundin · December 23, 2017, 10:22am

No don’t see it like that, all you did was making me look harder in new angles.
I’ve been trying to work this out for days before turning to you guys.

Kind regards, Morgan

willmotil · December 25, 2017, 6:42am

I was just looking at this and noticed your using reimers shadow shader. I was just referencing a old copy of it i had, because i forgot how you use the homogenous coordinates to get the correct texel depth. Im putting shadows in my shader atm.

Is that working code you have there for instancing with it ?

Morgan_Sundin · December 25, 2017, 3:44pm

Hello, yes you’re right I’m using Reimers shadow map shader, had some difficulties implementing it with my hardware instancing but it is working. I removed it from the code I’m working on now but I have a saved copy of the working version both shadow and instancing Shader here somewhere, you want it?

Regards, Morgan