Why is my displacement effect not working? (based on code from deferred engine)

Graphics
1

Hi all,

I tried to copy some shader code from the awesome deferred engine here:

To add displacement to my own custom effects, but for some reason it doesn’t work and I just can’t figure out why…
Here’s my vertex shader:

VertexShaderOutput VertexShaderMain(VertexShaderInput input)
{
// Initialize Output  
VertexShaderOutput output;

// Transform Position  
float4 worldPosition = mul(input.Position, World);
float4 viewPosition = mul(worldPosition, View);
float4 fragPosition = mul(viewPosition, Projection);
output.Position = fragPosition;
output.FragPosition = fragPosition;

// Pass Depth  
output.Depth.x = output.Position.z;
output.Depth.y = output.Position.w;
output.Depth.z = viewPosition.z;

// Build TBN Matrix (Tangent, Bitangent, Normals)
float3x3 TBN;
TBN[0] = normalize(mul(input.Tangent, (float3x3)WorldViewIT));
TBN[1] = normalize(mul(input.BiTangent, (float3x3)WorldViewIT));
TBN[2] = normalize(mul(input.Normal, (float3x3)WorldViewIT));
output.TBN = TBN;

// Pass UV  
output.UV = TextureOffset + input.UV * TexturesTiling;

// Return Output  
return output;
}

and here’s the part that do displacement in the pixel shader:

	// get pixel height from depth map
	float pixelHeight = tex2D(NormalSampler, UV).a;

	// get tangent pos and tangent camera
	float3 tangentPos = mul(input.FragPosition.xyz, input.TBN);
	float3 tangentCamera = mul(CameraPosition, input.TBN);

	// calculate view direction
	float3 viewDir = normalize(tangentPos - tangentCamera);

	// calculate new UV
	float2 p = viewDir.xy / viewDir.z * pixelHeight * 0.01f;
	UV = UV - p;

Which should do the same as the code from the deferred engine. However, the result I’m getting looks like this:

error.png1287×912 1.05 MB

Where the distort line “moves” along the surface as I change the camera angle / position.

BTW before I tried this code I had the much more primitive displacement code:

float2 offset = EyeTangent.xy * (pixelHeight * 0.04f * DisplacementScale - 0.01f) + DisplacementBias;
return uv + offset;

Which was sort-of working but looks pretty ugly and basic.

@kosmonautgames hope you see this :slight_smile:

2

Not sure on how @kosmonautgames does it, but here’s the depth-step calculation from my POM:

// n, t, b = tangent basis
// pointToCameraDirWS = normalize(cameraPos - pixelWorldPos)
float a = dot(n, -pointToCameraDirWS);
float3 s = float3(-dot(pointToCameraDirWS, t), dot(pointToCameraDirWS, b), a);
s *= -Depth / a * 0.1; // <--- pop-up not, sink down
float2 ds = s.xy; //  <--- step direction
float d = HeightFieldRaycast(uv, ds); // finds distance along ds via linear + binary search
return uv + (ds * d);
3

Could you please post the content of ‘HeightFieldRaycast’?
Thanks :slight_smile:

4

Sure can:

float HeightFieldRaycast(float2 dp, float2 ds)
{
    const int linearSteps = 10;
    const int binarySteps = 10;

    float size = 1.0 / float(linearSteps);
    float depth = 0.0;
    for (int i = 0; i < linearSteps; i++)
    {
        float t = tex2D(HeightMapSampler, dp + ds*depth).r;
        if (depth < (1.0 - t))
            depth += size;
    }
    
    for (int ii = 0; ii < binarySteps; ii++)
    {
        size *= 0.5;
        float t = tex2D(HeightMapSampler, dp + ds * depth).r;
        if (depth < (1.0 - t))
            depth += (2.0 * size);
        depth -= size;
    }
    return depth;
}
5

Thanks!
And to be sure, t, b, and n are tangent, binormal and normal?

6

Correct, they’re the vectors from the vertex data.

7

The call to updateUv looks like:

return updateUV(pixelToEye, input.Normal, normalize(input.Tangent), normalize(input.Bitangent), 0.4, input.SampleUV);

8

Will it be greedy to ask you to upload the whole effect file? I tried to add your method but it looks all distort and buggy and I’m beginning to think maybe I have a problem with the tangent calculation or something else…

Thanks! :slight_smile:

9

No problem: https://gist.github.com/JSandusky/fdf97852fc9c84e35dd776f6a39998ab

Disp.inc is the file. See DoHeightMap in PBREffect.fx.

Edit: turns out I wasn’t using handed tangents here.

1 Like