Instancing shader instance matrix translation gets lost

Graphics
1

Hi,

I hope someone can help me with a shader problem that I cannot solve. I solved the issue already by doing some calculations on the CPU (solution at the bottom) but I don’t get it to work on the GPU.

I created a simple instancing shader which receives an InstanceMatrix for each particle containing scalation, rotation and translation information. Additionally the shader has a global variable describing the parent world matrix also containing scalation, rotation and translation information.

The first issue is that the instance translation does not get added to the parent (World) translation. Scalation and rotation are applied correctly. I don’t understand why the translation of the instance matrix gets lost.

My shader currently looks like this.

//Camera view matrix.
float4x4 View;

//Camera projection matrix.
float4x4 Projection;

//Particle effect position (parent of each particle instance)
float4x4 World;

//Indicates whether the particle needs to face the camera or not.
bool IsBillboardRequired;

sampler2D Texture : register(s0);

struct ParticleGeometryVSInput
{
    float4 Position : SV_POSITION0;
    float2 TexCoord : TEXCOORD0;  
};

struct ParticleVSinput
{
    float4x4 WorldMatrix : BINORMAL1;
    float4 Color : COLOR0;
};

struct ParticleVSoutput
{
    float4 Position : SV_POSITION0;
    float2 TexCoord : TEXCOORD0;
	float4 Color : COLOR0;
};


ParticleVSoutput ParticleVS(ParticleGeometryVSInput geometryInput, ParticleVSinput instanceInput)
{
    ParticleVSoutput output;

    //TODO translation of instanceInput.WorldMatrix seems to get lost somewhere between here and setting the output.Position.
    float4x4 instancePosition = mul(World, transpose(instanceInput.WorldMatrix));
    if (IsBillboardRequired == true)
    {
        //Solved billboard and take rotation & scaling into account
    }
    else
    {
        float4 worldPosition = mul(geometryInput.Position, instancePosition);
        float4 viewPosition = mul(worldPosition, View);
        output.Position = mul(viewPosition, Projection);
    }

    output.TexCoord = geometryInput.TexCoord;
    output.Color = instanceInput.Color;
    return output;
}

float4 ParticlePS(ParticleVSoutput input) : COLOR0
{
    float4 color = tex2D(Texture, input.TexCoord);
    return color * input.Color;
}

technique ParticleEffect
{
    pass Pass0
    {
        VertexShader = compile VS_SHADERMODEL ParticleVS();
        PixelShader = compile PS_SHADERMODEL ParticlePS();
    }
}
Solved Billboarding issue and solution

Additionally I want to make the model (plane) face the camera when I set a global shader parameter (IsBillboardRequired) to true. I cannot figure out how to calculate it within the shader. Especially the rotation of the instance and parent matrix always get lost. I want that these rotations are relative applied to a plane that already faces the camera. I aware of the fact that rotations greater 90° don’t make sense.

Solution:
Everything seems to work correctly by applying the changes below but I want to do archive the same in the shader code and I don’t want to lose the instance translation when not using Billboarding.

    // Same shader as above but with the following code added. It is the same code as in the else block but with the calculations on the cpu side everything works including the translation.
    if (IsBillboardRequired == true)
    {
        float4 worldPosition = mul(geometryInput.Position, instancePosition);
        float4 viewPosition = mul(worldPosition, View);
        output.Position = mul(viewPosition, Projection);
    }

//Calculation of each instance before setting the instance buffer
var instanceMatrix = Matrix.CreateScale(instanceScale) * Matrix.CreateRotationY(MathHelper.ToRadians(instanceYRotation))) * Matrix.CreateBillboard(instancePosition camera.Position, Vector3.Up, null);

Edit: I actually was able to move the calculations to the shader by coping the behavior of CreateBillboard.

//Based on the monogame Matrix.CreateBillboard code
float4x4 Billboarding(float3 position)
{
    float3 viewDirection = CameraPosition - position;
    //float num = length(viewDirection); Length seems to be broken
    float num = (viewDirection.x * viewDirection.x) + (viewDirection.y * viewDirection.y) + (viewDirection.z * viewDirection.z);
    
    if (num < 0.0001)
    {
        viewDirection = CameraForward;
    }
    else
    {
        viewDirection = mul(viewDirection, 1.0 / sqrt(num));
    }

    float3 vector3 = normalize(cross(CameraUp, viewDirection.xyz));
    float3 vector2 = cross(viewDirection.xyz, vector3.xyz);
    float4x4 result = float4x4(float4(vector3, 0.0), float4(vector2, 0.0), float4(viewDirection,0.0), float4(position, 1.0));
    return result;
}

    if (IsBillboardRequired == true)
    {
        float3 pos = instanceInput.WorldMatrix[3].xyz;
        instanceInput.WorldMatrix[3].xyz = float3(0,0,0);
        float4x4 billboardInstanceMatrix = mul(instanceInput.WorldMatrix, Billboarding(pos));
        
        float4x4 instancePosition = mul(World, transpose(billboardInstanceMatrix));
        float4 worldPosition = mul(geometryInput.Position, instancePosition);
        float4 viewPosition = mul(worldPosition, View);
        output.Position = mul(viewPosition, Projection);
    }

I hope some one can explain what I’m doing wrong with the translation.

Thank you for your help.

2

Hi,

It seems like you’re dealing with a translation issue in your instancing shader. While I’m not a shader expert, have you considered checking the order of matrix multiplication for translations and transposing?

Additionally, for more advanced solutions, you might explore AI translation models. They can handle complex tasks, but it depends on the specifics of your project.

Hope this helps!

3

This is a bot that was reported few times, still here…

1 Like