MGFX weirdness when adjusting texCoord

I’m getting really weird behavior when trying to do a refraction post process effect. TextureSampler is my main game, RefractSampler just contains a bunch of green clouds for heat haze, etc.

With this code, what gets returned is just whatever’s on RefractSampler: basically a black screen with some green clouds.

float4 Refract(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
{	
	float2 tex = texCoord;
	
	float offset = tex2D(RefractSampler, texCoord).g;
	
	tex.x += toffset * 0.01f;

	float4 col = tex2D(TextureSampler, tex);
	
	return col; 
}

If I switch the next to last line like below, I get just the regular game screen (of course, no heat haze)

float4 Refract(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0//(PS_INPUT Input) : COLOR0
{	
	float2 tex = texCoord;
	
	float offset = tex2D(RefractSampler, texCoord).g;
	
	tex.x += offset * 0.01f;

	float4 col = tex2D(TextureSampler, texCoord);
	
	return col; 
}

Any ideas? Neither snippet ever explicitly grabs the RefractSampler to return, but somehow as soon as I let the texture to return’s coordinate be affected by RefractSampler’s green value, it returns RefractSampler. I’ve tried just saying tex.x += 0.5f, and that produced the desired effect: TextureSampler offset to the left with a bunch of horizontal streaks halfway across. So I can adjust the TextureSampler tex, but as soon as that number’s affected by RefractSampler’s green channel, it all goes funny.

Thanks!

I got it working, and it’s pretty definitely an issue with the MGFX compiler. Important: TextureSampler is the main game screen, RefractSampler is a black screen with some green glows on it to indicate where we’re refracting. Notice that we’re returning RefractSampler at the end to get the correct effect.

    //refract.fx

    sampler TextureSampler : register(s0);
    sampler RefractSampler : register(s1);

    float amt = 0.0f;

    struct PS_INPUT
    {
    	float2 TexCoord : TEXCOORD0;
    };

    float4 Refract(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0//(PS_INPUT Input) : COLOR0
    {	
    	float2 tex = texCoord;
    	float2 ttex = texCoord;

    	float offset = tex2D(RefractSampler, texCoord).g; //Take offset from Refract Sampler
    	float toffset = tex2D(TextureSampler, texCoord).g; //Take offset from Texture Sampler, actually reads from Refract Sampler
	
    	tex.x += offset * 0.01f;	//If I leave this line out, it throws everything off
        	ttex.x += toffset * 0.01f;
    
        	float4 col = tex2D(RefractSampler, ttex);	//Get adjusted texture from Refract Sampler, actually reads from Texture Sampler
    	
    	return col; 
    }
    
    technique RefractTechnique
    {
        pass P0
        {
            PixelShader = compile ps_4_0_level_9_3 Refract();
        }
    }

So first question is where is the vertex shader coming from? Are you using this custom effect with SpriteBatch?

Yep, using this custom effect with SpriteBatch.

First note the MGFX uses the DirectX HLSL compiler under the hood, so we’re limited by what it wants to do at times.

What is missing in this shader is defining the textures and associating the texture to the sampler.

Textures and samplers are different elements… both are needed to read a texture. In this case the HLSL compiler is automatically adding textures to your shader for you… but it is getting the order wrong.

So while you are doing this:

sampler TextureSampler : register(s0);
sampler RefractSampler : register(s1);

The HLSL compiler is adding textures for you and associating them to your samplers under the hood for you:

texture2d RefractTex : register(t0);
texture2d TextureTex : register(t1);

Why is it backwards? I guess it is because the code references the refract sampler first… so it creates the texture for you and starts with register t0 because it is the first available texture slot.

The right fix is to use the correct SM 4 way of defining textures and doing sampling. Look at our texture definition macro we use in the stock effects…

https://github.com/mono/MonoGame/blob/develop/MonoGame.Framework/Graphics/Effect/Resources/Macros.fxh#L23

Notice it creates the texture and the sampler. Also there is no magic connection between t0 and s0… the compiler needs to be told what texture is associated to what sampler. The old tex2D() method doesn’t do this… it leaves it up to the compiler to guess. Instead use the SM 4 method like in our macro…

https://github.com/mono/MonoGame/blob/develop/MonoGame.Framework/Graphics/Effect/Resources/Macros.fxh#L31

This is just part of moving forward to the new shader model.

Thanks, Tom!

Here’s the working shader now, works totally fine:

//refract.fx

#define DECLARE_TEXTURE(Name, index) \
    Texture2D<float4> Name : register(t##index); \
    sampler Name##Sampler : register(s##index)

#define SAMPLE_TEXTURE(Name, texCoord)  Name.Sample(Name##Sampler, texCoord)

DECLARE_TEXTURE(TextureSampler, 0);
DECLARE_TEXTURE(RefractSampler, 1);

float amt = 0.0f;

struct PS_INPUT
{
	float2 TexCoord : TEXCOORD0;
};

float4 Refract(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
{	
	float2 tex = texCoord;
	float offset = SAMPLE_TEXTURE(RefractSampler, texCoord).g;
	tex.x += offset * 0.01f;
	tex.y += offset * 0.01f;

	return SAMPLE_TEXTURE(TextureSampler, tex);
}

technique RefractTechnique
{
    pass P0
    {
        PixelShader = compile ps_4_0_level_9_3 Refract();
    }
}