passing mask texture to shader. Works in XNA but not monogame.

[bangclash1]

I’m new shader. I’m passing a mask texture to a shader. want to manipulate the gameplay screen based on the mask. This code works in XNA but not monogame. It seems the mask sampler is just the same as the gameplay screen. I’m using DirectX template. Sample code below.

#if OPENGL
#define SV_POSITION POSITION
#define VS_SHADERMODEL vs_4_0
#define PS_SHADERMODEL ps_4_0
#else
#define VS_SHADERMODEL vs_4_0_level_9_1
#define PS_SHADERMODEL ps_4_0_level_9_1
#endif

sampler2D screen : register(s0);
sampler2D mask : register(s1);

struct VertexShaderInput
{
	float4 Position : POSITION0;
	float4 Color : COLOR0;
};

struct VertexShaderOutput
{
	float4 Position : SV_POSITION;
	float4 Color : COLOR0;
	float2 TextureCoordinates : TEXCOORD0;
};


float4 MainPS(VertexShaderOutput input) : COLOR
{
	float2 uv = input.TextureCoordinates;
	float4 colorM = tex2D(mask, uv);
	float4 colorGameplay = tex2D(screen, uv);
// Do masking logic below to combine the colors and return that color.
}

technique BasicColorDrawing
{
	pass P0
	{
		PixelShader = compile PS_SHADERMODEL MainPS();
	}
};


// my draw method.
            GraphicsDevice.Clear(Color.CornflowerBlue);
            GraphicsDevice.Textures[1] = mask;
            GraphicsDevice.SamplerStates[1] = SamplerState.PointClamp;

            spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, 
                                         SamplerState.PointClamp, null, null, effect, null);
            spriteBatch.Draw(gamplayscreens, new Rectangle(0, 0, 800, 600), Color.White);
            spriteBatch.End();

[reiti.net]

while I have no definite answer why it’s not working that way, you could just use regular parameters instead and it should work:

effect.Parameters["mask"].SetValue(mask);

not sure if monogame does something different than XNA here, but at least it keeps you going

(be aware, this is just setting a texture, so you may want to setup the sampler in in the shader code)

[bangclash1]

I also tried that too, but it doesn’t work either. Below is my code. This is driving me crazy. I know this should be simple.

Texture2D mask;
sampler2D maskSampler = sampler_state
{
	Texture = <mask>;
};

/// draw method.
            GraphicsDevice.Clear(Color.CornflowerBlue);
            effect.Parameters["mask"].SetValue(mask);
            spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.AlphaBlend, SamplerState.PointClamp, null, null, effect, null);
            spriteBatch.Draw(gamplayscreens, new Rectangle(0, 0, 800, 600), Color.White);
            spriteBatch.End();

[Ravendarke]

If you are using DirectX (which you said you are) you can go modern and clean.

Texture2D mask : register(t1); //t0 is used directly by spritebatch and that slot binds texture specified in .Draw(..)
SamplerState sampler1 : register(s1);

...
colorM = mask.Sample(sampler1, uv);

[bangclash1]

I kind of combined both suggestions and now it works. I still don’t know why it worked. Can someone explain? Using s0 seems to be an issue but seems to work in XNA.

#if OPENGL
	#define SV_POSITION POSITION
	#define VS_SHADERMODEL vs_4_0
	#define PS_SHADERMODEL ps_4_0
#else
	#define VS_SHADERMODEL vs_4_0
	#define PS_SHADERMODEL ps_4_0
#endif

SamplerState screen : register(s1);
Texture2D mask : register(t2);
SamplerState sampler1 : register(s2);

struct VertexShaderOutput
{
	float4 Position : SV_POSITION;
	float4 Color : COLOR0;
	float2 TextureCoordinates : TEXCOORD0;
};


float4 MainPS(VertexShaderOutput input) : COLOR
{
	float4 color = tex2D(screen, input.TextureCoordinates);
	float4 colorM = mask.Sample(sampler1, input.TextureCoordinates);
	// test code just to see if I can display both textures.
	return float4(colorM.rg, color.b, 1);
}

technique SpriteDrawing
{
	pass P0
	{
		PixelShader = compile PS_SHADERMODEL MainPS();
	}
};

/// draw code.
            GraphicsDevice.Clear(Color.Yellow);
            GraphicsDevice.SamplerStates[1] = SamplerState.PointClamp;
            spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, SamplerState.PointClamp, null, null, effect);
            effect.Parameters["mask"].SetValue(mask);
            spriteBatch.Draw(gameplayScreen, new Rectangle(0, 0, 800, 600), Color.White);
            spriteBatch.End();

[Damian_Eaglestein]

i can do this santity testing part , marked “solution,” , but I did you get actual clip to work ?
could you share the cliping shader logic? I only see code to test the sampling works, i confirm that works.

is it the using the compute fork.? i tried the MG 3.8.1 release nuget it on android, opengl ,windows net 7 , and directx .I found that s0 also works now. its better and i can sample the registers, but w spirtebatch using a simple shader that actually CLIPs doesnt work, and its nt shown in the solution. this alwasy fails at the clipping branch or any attempt i made to clip if the mask isnt float4(1,1,1,1) white?
the sampling works, and i can i still pass the mask and the sprite as textures to the shader, via parameters, the combine those to a big sprite with mipmaps via to a rendertarget, is ok but for vectors i wanna just clip without making a huge 4k sprite, its way too expensive.

? shouldnt the float4 be 0 to 255 = 0 to 1 ( float) … with surface.Color
.the simple ifdoes work when passing in two tectures and making a sprite with transparency , in rendertarget which passing back a clippedsprite.
but uses tons of memory…

float4 MainPS(VertexShaderOutput input) : COLOR
{
float4 color = tex2D(screen, input.TextureCoordinates);
float4 colorM = mask.Sample(sampler1, input.TextureCoordinates);
//return color; // test code just to see if I can display both textures. (fine this works on 4 platforms i rigged , but its not CLIPPING anything off the screen sprite)… registers work and i pass only the mask… its better but its not doing the clip.

/// return colorM ; works…

if (all(colorM) == float4(1, 1, 1, 1)); //draw spiritewhite part of mask transparent background
{
return color**; //Never goes here this, worked if if use rendder target, should i use a nother shading branchless way some bitwise AND ? sorry im new to shaders or ive set my device surface wrong?**
}
return float4(0 ,0, 0, 0); // transparent.
}

i tried all ths too:

float4 color = tex2D(screen, input.TextureCoordinates);
float4 colorM = mask.Sample(sampler1, input.TextureCoordinates);


//clip(colorM.a -0.05); //tried all thsi branchless stufff.. doesnt do anything expected..

// return color;
// return ret;
// ret.a = ret.a * colorM.a;
// ret.a = colorM.a;
// return ret;
// return color.rgba * colorM.r;
// return color;
//if (colorM.b ==1) //there are built in stencis, depths, and other clipping stuff in Basic effect but I couldnt get them to work. the samples are from XNA times

  //if (colorM.r == 1)
    if (all(colorM) == all(float4(1, 1, 1, 1))) //this is what  i did w render targets..  that did work

{return color;
}
   return float4(0, 0,0,0); //    transparent.     

//tired branchless clip and operators to achieve clip , i think somehing is being culled as this is 3d graphics territory…I just cant see any differnce, and im new to stencils and all the settings, also in hsl if the a mask has a is zero, a neg clip(-1) should discard the pixel and thus clip… im using Surface.Color // it doesnt work on dx or openGL ive looked at shader forums, totally stuck … Please advise…i dont want to use render targets and pass 64 meg of image data to clip, i want to draw through a clip raster at 4k res, on the main device, no blending, should be just a simple thing iike if its 255,255,255,255 then drw otherwise dont draw.

I can’t find ia sample i want to add it to my shared x platform demo to help show how to quickly deploy and test single linked content and shared core game code, on 5 projects in one solution… ios, androind, mac linux, and windowsgl , winodwsdx, without maintaing links for new files like the current samples do for some reason… trimming and AOT must be off though for now on relese deployments for now. with net 7 you can get SIMD gaints with JITing thasts arent possible with AOT anyways.