Hi,
I’m trying to draw using masking with DepthStencilState as described here but I cannot get this to work on iOS or Android.
Whatever I attempt to pass to SpriteBatch Begin() just seems to render as normal, ignoring any DepthStencilState settings.
Has this been implemented in MG yet - and if so has anyone actually got it to work!?
cheers
It is totally busted on Mac so I assume that means its probably busted on iOS/Android. I can’t even take a stab at fixing it due to an issue compiling from source with both MonoGame 3.4 and 3.5 crashing on launch.
It is implemented… has been for years now.
What version of MonoGame are you using? Are you on 3.4 or using a build from the develop branch?
We fixed some matrix issues that was causing odd behavior under sprite batch since 3.4, but other than that the DepthStencilState should be fully working.
Hi Tom,
The iOS version is from develop from about Nov 2015. The Android version says it’s version 3.1.2.0.
Maybe I’ll take another look later - it’s something that’s totally new to me so I can’t be sure I’m doing things 100% correct but I have tried all sorts and everything just seems to render as if I’d passed a null DepthStencilState.
I have seen a few posts around about it not working correctly such as this one and @prime31 also seems to be having problems above!
I finally found the issue on Mac last night! I was using the PCL MonoGame DLL and it has incorrect stencil enum values that don’t match reality. Hardwiring them to the proper values did the trick for my particular case.
Ok - mea culpa - it wasn’t working because I was being a complete tit! I should know by now to always presume that as the first option! 
So the DepthStencilState values that I’m passing to the SpriteBatch are now doing something but I have run into another issue which I’m finding hard to resolve. First I draw a bitmap (which I want to use as a mask) using the following settings:
mask = new DepthStencilState {
StencilEnable = true,
StencilFunction = CompareFunction.Always,
StencilPass = StencilOperation.Replace,
ReferenceStencil = 1,
DepthBufferEnable = false,
};
Then I draw an overlay (which I want to appear only where the non-transparent pixels of the mask are) using the following settings:
overlay = new DepthStencilState {
StencilEnable = true,
StencilFunction = CompareFunction.LessEqual,
StencilPass = StencilOperation.Keep,
ReferenceStencil = 1,
DepthBufferEnable = false,
};
The problem is that there seems to be a value of 1 in the stencil for every pixel in the mask, whether it is transparent or not. Consequently my overlay bitmap just gets drawn as a square the size of the mask.
How can I get the pixels in the mask that have a zero alpha value not to write to the stencil?
cheers
The stencil will indeed have a value for all pixels, transparent or not. The way you work around that is to either use AlphaTestEffect or just make your own shader that discards pixels that are less than some threshold (say alpha < 0.2).
Hi,
Thanks for the input here. I gave up on DepthStencilState in the end and achieved what I wanted using a custom shader. Even on my tests results were inconsistent on iOS and Android so I figured there may still be something screwy going on in one of the MG implementations at least.
I would very much like to get it to work at some point though. When you say ‘discard pixels’ in a shader what exactly do you mean? The textures I were using were 1bit alpha so the ‘empty’ pixels should have returned a value of zero for colour and alpha - is there anything else I can do?
cheers
A transparent pixel is still a pixel so the stencil will still get written to. In your shader you can use ‘discard’ to completely throw away the result and write nothing. See the AlphaTestEffect shader for an example.
OK - thanks - get it now ; )