[SOLVED] Stencil not working

Graphics
1

My stencil is not working. It just draw both textures as normal.

graphics = new GraphicsDeviceManager(this)
{
PreferredDepthStencilFormat = DepthFormat.Depth24Stencil8
};

void Draw(GameTime gameTime){
// GraphicsDevice.Clear(Color.CornflowerBlue);
GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.Stencil, Color.CornflowerBlue, 0, 0);

        // TODO: Add your drawing code here

        // Start drawing
        spriteBatch.Begin(transformMatrix:GameResolution.Instance.GetScaleMatrix()

,blendState: BlendState.AlphaBlend);
-----Draw other stuf ------
DrawStencil(…);
-----Draw other stuf ------

spriteBatch.End();
}

> public void DrawStencil(Vector2 start, Vector2 end,Texture2D texture,Rectangle source, Vector2 position)

    {

        this.SpriteBatch.End();

        var m = Matrix.CreateOrthographicOffCenter(0,
            Graphics.GraphicsDevice.PresentationParameters.BackBufferWidth,
            Graphics.GraphicsDevice.PresentationParameters.BackBufferHeight,
            0, 0, 1
        );

        var a = new AlphaTestEffect(Graphics.GraphicsDevice)
        {
            Projection = m
        };

        var s1 = new DepthStencilState
        {
            StencilEnable = true,
            StencilFunction = CompareFunction.Always,
            StencilPass = StencilOperation.Replace,
            ReferenceStencil = 1,
            DepthBufferEnable = false,
        };

        var s2 = new DepthStencilState
        {
            StencilEnable = true,
            StencilFunction = CompareFunction.Equal,
            StencilPass = StencilOperation.Keep,
            ReferenceStencil = 1,
            DepthBufferEnable = false,
        };
        Graphics.GraphicsDevice.Clear(ClearOptions.Stencil, Color.Gray, 0, 0);

        this.SpriteBatch.Begin(SpriteSortMode.Immediate, depthStencilState: s1, effect: a , transformMatrix: GameResolution.Instance.GetScaleMatrix());
        //DrawRectangle(start, end, Color.White);
        this.SpriteBatch.Draw(pixel3x7, position, Color.White); //The mask  
        this.SpriteBatch.End();

        this.SpriteBatch.Begin(SpriteSortMode.Immediate, depthStencilState: s2, effect: a, transformMatrix: GameResolution.Instance.GetScaleMatrix());
        this.SpriteBatch.Draw(texture, position, sourceRectangle: source, color: Color.White); //The background  
        this.SpriteBatch.End();

        //Graphics.GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.Stencil, Color.Blue, 0, 0);
        this.SpriteBatch.Begin(transformMatrix: GameResolution.Instance.GetScaleMatrix(), blendState: BlendState.AlphaBlend);

    }
2

Hmm let’s check

In s1 you Always Replace the current stencil with ReferenceStencil = 1

In s2 you compare the stencil value with ReferenceStencil = 1 and check if the value is LessEqual.

Let’s assume you clean your stencil before drawing, so it’s 0.
You draw texture 1 with s1 and all affected pixels replace the stencil with value 1.
You draw texture 2 with s2 and check if the stencil value is less or equal to 1. It always is, so you draw the second texture.

3

Let’s assume you clean your stencil before drawing, so it’s 0.
Yep

GraphicsDevice.Clear(ClearOptions.Target | ClearOptions.Stencil, Color.CornflowerBlue, 0, 0);

I changed s2 for:

var s2 = new DepthStencilState
{
StencilEnable = true,
StencilFunction = CompareFunction.Equal,
StencilPass = StencilOperation.Keep,
ReferenceStencil = 1,
DepthBufferEnable = false,
};

Still not working.

4

Here are my old stencil functions, where one texture should draw on the other, try it out and tell me if it works

_stencilWriteOnly = new DepthStencilState()
            {
                DepthBufferEnable = false,
                DepthBufferWriteEnable = false,
                CounterClockwiseStencilFunction = CompareFunction.Always,
                StencilFunction = CompareFunction.Always,
                StencilFail = StencilOperation.IncrementSaturation,
                StencilPass = StencilOperation.IncrementSaturation,
                CounterClockwiseStencilFail = StencilOperation.IncrementSaturation,
                CounterClockwiseStencilPass = StencilOperation.IncrementSaturation,
                ReferenceStencil = 0,
                StencilEnable = true,
                StencilMask = 0,
            };

        _stencilReadOnly = new DepthStencilState()
        {
            DepthBufferEnable = false,
            DepthBufferWriteEnable = false,
            CounterClockwiseStencilFunction = CompareFunction.Less,
            StencilFunction = CompareFunction.Less,
            StencilFail = StencilOperation.Keep,
            StencilPass = StencilOperation.Keep,
            CounterClockwiseStencilFail = StencilOperation.Keep,
            CounterClockwiseStencilPass = StencilOperation.Keep,
            ReferenceStencil = 0,
            StencilEnable = true
        };
5

No.
I have create a new project with only that code, and do not work. Draw both textures.
if i do not draw anything in the firts stencil, the second texture is still been drawed.

spriteBatch.Begin(SpriteSortMode.Immediate, depthStencilState: s1, effect: a);
//spriteBatch.Draw(pixel3x7, position, Color.White); //The mask
spriteBatch.End();

spriteBatch.Begin(SpriteSortMode.Immediate, depthStencilState: s2, effect: a);
spriteBatch.Draw(texture, position, sourceRectangle: source,
color: Color.White, effects: inverse ? SpriteEffects.FlipHorizontally : SpriteEffects.None); //The background
spriteBatch.End();

6

I just made a new solution and it works straight away.

spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Opaque, SamplerState.LinearClamp, _stencilWriteOnly);
            spriteBatch.Draw(_plainWhite, new Rectangle(10,10,200,200),Color.White);
            spriteBatch.End();

            spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Opaque, SamplerState.LinearClamp, _stencilReadOnly);
            spriteBatch.Draw(_plainWhite, new Rectangle(100, 100, 200, 200), Color.Red);
            spriteBatch.End();

stencil.png802×512 2.31 KB 

//spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Opaque, SamplerState.LinearClamp, _stencilWriteOnly);
            //spriteBatch.Draw(_plainWhite, new Rectangle(10,10,200,200),Color.White);
            //spriteBatch.End();

            spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Opaque, SamplerState.LinearClamp, _stencilReadOnly);
            spriteBatch.Draw(_plainWhite, new Rectangle(100, 100, 200, 200), Color.Red);
            spriteBatch.End();

stencil2.png802×512 2.29 KB

7

Here is the solution (64kb)

https://we.tl/lIoB0Vi5vh

8 
  in project configuration file (.csproj) I had this:

$(MonoGameInstallDirectory)\MonoGame\v3.0\Assemblies\DesktopGL\MonoGame.Framework.dll

Changed for this and now work:
> $(MonoGameInstallDirectory)\MonoGame\v3.0\Assemblies\Windows\MonoGame.Framework.dll

I am working on windows but i choosed Cross Platform Desktop Project.

PD: Thank.

9

So it doesn’t work on OpenGL? This should be fixed then

I checked and can confirm. I’ll open an issue if not open yet