HLSL problem - solved

I’m using Monogame 3.01 for Windows DirectX and am working with 2D sprites, however need to apply effects to each sprite. PixelShaders seem ideal but I’ve hit a brick wall trying to use them. My HLSL file is as follows;

    float brightnessEffect;
float colorEffect;
// Our texture sampler
texture Texture;
sampler TextureSampler = sampler_state
{
	Texture = <Texture>;
};

struct PixelInput
{
float2 TexCoord: TEXCOORD0;
};

float RGBCVtoHUE(in float3 RGB, in float C, in float V)
  {
      float3 Delta = (V - RGB) / C;
      Delta.rgb -= Delta.brg;
      Delta.rgb += float3(2,4,6);
      Delta.brg = step(V, RGB) * Delta.brg;
      float H;
      H = max(Delta.r, max(Delta.g, Delta.b));
      return frac(H / 6);
  }
 
float3 RGBtoHSL(in float3 RGB)
  {
    float3 HSL = 0;
    float U, V;
    U = -min(RGB.r, min(RGB.g, RGB.b));
    V = max(RGB.r, max(RGB.g, RGB.b));
    HSL.z = (V - U) * 0.5;
    float C = V + U;
    if (C != 0)
    {
      HSL.x = RGBCVtoHUE(RGB, C, V);
      HSL.y = C / (1 - abs(2 * HSL.z - 1));
    }
    return HSL;
  }

  float3 HUEtoRGB(in float H)
  {
    float R = abs(H * 6 - 3) - 1;
    float G = 2 - abs(H * 6 - 2);
    float B = 2 - abs(H * 6 - 4);
    return saturate(float3(R,G,B));
  }

  float3 HSLtoRGB(in float3 HSL)
  {
    float3 RGB = HUEtoRGB(HSL.x);
    float C = (1 - abs(2 * HSL.z - 1)) * HSL.y;
    return (RGB - 0.5) * C + HSL.z;
  }

float4 PixelShaderFunction(PixelInput input) : COLOR0
{
	float4 colour = tex2D(TextureSampler, input.TexCoord);
	float3 hslColour = RGBtoHSL(float3(colour.r,colour.g,colour.b));
	hslColour.z +=brightnessEffect;
	hslColour.z = saturate(hslColour.z);
	hslColour.x += colorEffect;
	if (hslColour.x > 1)
	{
		hslColour.x -=1;
	}
	hslColour.x = saturate(hslColour.x);
	float3 newrgb = HSLtoRGB(hslColour);
    return float4(newrgb.r, newrgb.g, newrgb.b, colour.a);
}
 
technique Technique1
{
    pass Pass1
    {
        PixelShader = compile ps_4_0_level_9_1 PixelShaderFunction();
    }
}

The pixelshader is supposed to apply a colour and brightness effect to the sprite based on the HSL colour space. The fx file compile ok using the /DX11 switch. In the draw method I have

   textureE.SetValue(cr.texture);
    brightnessE.SetValue(Stage.Effects.Brightness / 100f);
    colourE.SetValue(Stage.Effects.Color / 200f);
    _effect.CurrentTechnique.Passes[0].Apply();
    sb.Draw(cr.texture, new Microsoft.Xna.Framework.Rectangle(0, 0, 480, 360), Color.White);

which loops through all the sprites where texture,brightnessE, colourE are Effect Parameters.
The spritebatch runs in immediate mode, but although the brightness can be changed I never see a texture on the screen. It seems odd that I have to pass a texture as an effect parameter when the texture is part of the sprite batch draw.

Any idea what I’m doing wrong? Thank in advance.

Finally got it working from this article http://www.software7.com/blog/pitfalls-when-developing-hlsl-shader/ and so using the complete pixelshader signature.

This happens a lot when you first get into shaders. It isn’t a MonoGame thing… it is a DX10/11 thing. I was thinking we might be able to add some debug code into MonoGame to assert if the vertex output and the pixel shader inputs don’t match up correctly.

I’m still somewhat clueless as to how the registers get populated and what get’s passed where. My current issue is that having put all my spritebatchs into immediate mode to cater for different effect parameters per draw have been hit badly performance wise. So am returning to deferred mode and to set effects am hijacking the tint color to use as a parameter for each draw. I’m hoping that is a good plan but am currently trying to work out whether the pixelshader color1 parameter is the tint color passed into the spritebatch draw or whether I have to add in a vertex shader to pass that color info onto the pixelshader correctly. It really does seem like going down the rabbit hole as Alice might say! :wink:

It seems the performance problem isn’t because of running in immediate mode. I’ve got the new pixelshader working which takes its parameters from the tint color passed to the draw function and so can run everything in deferred mode. However the performance is still bad so the only real difference is the pixelshader. The new pixelshader is this

// Texture passed in from spritebatch
sampler s0;

// Fn’s for brightness and colour effects

float RGBCVtoHUE(in float3 RGB, in float C, in float V)
{
float3 Delta = (V - RGB) / C;
Delta.rgb -= Delta.brg;
Delta.rgb += float3(2,4,6);
Delta.brg = step(V, RGB) * Delta.brg;
float H;
H = max(Delta.r, max(Delta.g, Delta.b));
return frac(H / 6);
}

float3 RGBtoHSL(in float3 RGB)
{
float3 HSL = 0;
float U, V;
U = -min(RGB.r, min(RGB.g, RGB.b));
V = max(RGB.r, max(RGB.g, RGB.b));
HSL.z = (V - U) * 0.5;
float C = V + U;
if (C != 0)
{
HSL.x = RGBCVtoHUE(RGB, C, V);
HSL.y = C / (1 - abs(2 * HSL.z - 1));
}
return HSL;
}

float3 HUEtoRGB(in float H)
{
float R = abs(H * 6 - 3) - 1;
float G = 2 - abs(H * 6 - 2);
float B = 2 - abs(H * 6 - 4);
return saturate(float3(R,G,B));
}

float3 HSLtoRGB(in float3 HSL)
{
float3 RGB = HUEtoRGB(HSL.x);
float C = (1 - abs(2 * HSL.z - 1)) * HSL.y;
return (RGB - 0.5) * C + HSL.z;
}

float4 ColourAndBrightness(in float4 colour, in float colorEffect, in float brightnessEffect)
{
float3 hslColour = RGBtoHSL(float3(colour.r,colour.g,colour.b));
hslColour.z +=brightnessEffect;
hslColour.z = saturate(hslColour.z);
hslColour.x += colorEffect;
if (hslColour.x > 1)
{
hslColour.x -=1;
}
hslColour.x = saturate(hslColour.x);
float3 newrgb = HSLtoRGB(hslColour);
return float4(newrgb.r,newrgb.g,newrgb.b,colour.a);
}

// The Pixel Shader
// Note: tint color color1 is uses to pass color in r, brightness in g and ghost in a.
float4 PixelShaderFunction(float4 pos : SV_POSITION, float4 color1 : COLOR0, float2 Tex : TEXCOORD0) : SV_TARGET0
{
float4 colour = tex2D(s0, Tex);
if (color1.b ==0)
{
float brightness = (color1.g - 0.5)*2;
if (color1.r != 0 || brightness !=0)
{
colour = ColourAndBrightness(colour, color1.r,brightness );
}
}
colour.a *= color1.a;
colour.a = saturate(colour.a);
return colour;
}

technique Technique1
{
pass Pass1
{
PixelShader = compile ps_4_0_level_9_1 PixelShaderFunction();
}
}

If color.b is 0 then it doesn’t do all the hsl conversions so should be quick I’d’ve thought. However my framerate has gone from 60fps to 1 fps on one test when no hsl conversion should be performed as the tint is Color.White. What’s causing the slowdown?

There is nothing there I could see that would cause it going from 60fps to 1fps. You have some other issue causing this.

Thank you for the feedback. I shall start trying to make small incremental changes to see where the problem is but it really seems to be the difference between

            _SpriteBatch.Begin();

and

        _SpriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, null, null, null, scratchEffect);

in 3 places that cause the difference.

I’ve whittled it down to the main spritebatch that I use and the difference is this;

                _spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, null, null, null, null, screenScale); // = 42 fps
            _spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, null, null, null, scratchEffect, screenScale); // = 4 fps

So the only difference is the addition of the pixeslhader as opposed to the built in one but there’s a 10x difference in framerate between the 2.

Could it be that the new pixelshader because it’s being compiled with /DX11 and/or ps_4_0_level_9_1 is creating a pixelshader which cannot be run on my old laptop and so is having to run things differently instead? Just a complete noob guess but went the DX11 switch route as that was the one route that enabled the pixelshader to be compiled in the end despite not needing any DX11 features! Remembered why I used the DX11 switch now because if you use ps_2_0 then get an exception explained here

Well your shader has quite a bit of if blocks which generally are not considered fast.

But I really think this is more of a system issue. Have you installed the latest drivers on your system? Have you run any other 3D games on your system?

Well as mentioned it’s a laptop so not really built for 3D games. It’s as up to date as a GMA945 can be! I think the chipset does tend to offload things onto the cpu so was also one of the reasons I was thinking did the pixelshader generate code which would be offloaded onto the cpu instead of gpu.

Even a GMA945 shouldn’t be that bad. There is more going on there I think.

Thank you again. I’m not really sure how I can investigate further unfortunately. If you’ve seen my other post about nesting sprite batches then I’m going to keep the main sprite batch as the default pixelshader and then have another sprite batch create textures for the other sprite batch with the pixelshader effects applied (but not nested!)

Not much more I can do to help you here.

There is nothing special or different about MonoGame Effects… they are compiled HLSL or GLSL effects like on any other engine/platform. If the pixel shader is truly the performance bottleneck then it is simply a case where your video card doesn’t have enough fillrate to keep up.

Just thinking about your comment about “if”'s not being fast in a pixelshader. I have to apply a number of effects and I though that if I could do it all in one pass that would be more efficient so just one pixelshader and in the pixelshader have if effect is applicable then apply it. Would it be better to have a pixelshader for each effect and if an effect is needed then apply that effect then, using the texture from that, pass it into the next spritebatch to apply the next effect to it and so on? So I won’t have so many if’s but have to set up a series of spritebatchs applying different effects.

Just as an update have now got 7 graphical effects working via 2 pixelshaders. Video here of the effects if interested http://youtu.be/7ZMUuey-I4U no issues with performance so thank you once again monogame!

Cool… glad to see you worked your issue out!