How to use a custom shader with a dynamically generated mesh

I’m trying to use a custom shader effetct on a generated mesh and unfortunately all that I get is a blank screen.
HLSL is something new to me and I’m not familiar with, so I’ve tried reusing some simple examples with no luck.
I’ve tried to use a couple of simple shaders, just to get them running with my code.
This is an ambient light sample I’ve digged out from XNA 3.5 and updated to monogame.

HLSL:

float4x4 matWorldViewProj;

struct OUT
{	
	float4 Pos: POSITION;
};

OUT VertexShaderFunction( float4 Pos: POSITION )
{	
	OUT Out = (OUT) 0;
	Out.Pos = mul(Pos, matWorldViewProj);
	
	return Out;
}

float4 PixelShaderFunction() : COLOR
{
	// Set ambient intensity to 80%
	float Ai = 0.8f;

	// Set the color of our ambient light
	float4 Ac = float4(0.85, 0.75, 0.2, .0);
	
	// return our color to the COLOR register
	return Ai * Ac;
}

technique AmbientLight
{	
	pass P0
	{		
		VertexShader = compile vs_2_0 VertexShaderFunction();		
		PixelShader = compile ps_2_0 PixelShaderFunction();
	}
}

And below another one I’ve dug up from the forum:

#if OPENGL
#define SV_POSITION POSITION
#define VS_SHADERMODEL vs_3_0
#define PS_SHADERMODEL ps_3_0
#else
#define VS_SHADERMODEL vs_4_0_level_9_1
#define PS_SHADERMODEL ps_4_0_level_9_1
#endif

matrix WorldViewProjection;

static const float4 MyRed = float4(1.0f, 0.0f, 0.0f, 1.0f);
static const float4 MyBlue = float4(0.0f, 0.0f, 1.0f, 1.0f);

struct VertexShaderInput
{
    float4 Position : POSITION0;
    float4 Color : COLOR0;
};

struct VertexShaderOutput
{
    float4 Position : SV_POSITION;
    float4 Color : COLOR0;
};

VertexShaderOutput MainVS(in VertexShaderInput input)
{
    VertexShaderOutput output = (VertexShaderOutput)0;
    float4 pos = mul(input.Position, WorldViewProjection);
    output.Position = pos;

    float4 col = (pos.x * MyRed) + pos.x * (1.0f - MyBlue);
    output.Color = col;

    return output;
}

float4 MainPS(VertexShaderOutput input) : COLOR
{
    //return MyRed; // works
    return input.Color; // use vertex shader
}

technique BasicColorDrawing
{
    pass P0
    {
        VertexShader = compile VS_SHADERMODEL MainVS();
        PixelShader = compile PS_SHADERMODEL MainPS();
    }
};

This is the draw method I’m using which works completely fine with basic effect, however I don’t know what I’m doing wrong when I try to use the cusotm one:

Game.GraphicsDevice.Clear(Color.CornflowerBlue);

vertexBuffer = new DynamicVertexBuffer(Game.GraphicsDevice, typeof(VertexPositionColor), ((VertexPositionColor[])CurveData[0]).Length, BufferUsage.WriteOnly);
indexBuffer = new DynamicIndexBuffer(Game.GraphicsDevice, IndexElementSize.ThirtyTwoBits, ((int[])CurveData[1]).Length, BufferUsage.WriteOnly);

vertexBuffer.SetData((VertexPositionColor[])CurveData[0], 0, ((VertexPositionColor[])CurveData[0]).Length);
indexBuffer.SetData((int[])CurveData[1]);

Game.GraphicsDevice.SetVertexBuffer(vertexBuffer);
Game.GraphicsDevice.Indices = indexBuffer;

RasterizerState rasterizerState = new RasterizerState();
rasterizerState.CullMode = CullMode.None;            
Game.GraphicsDevice.RasterizerState = rasterizerState;

customEffect.CurrentTechnique = customEffect.Techniques["BasicColorDrawing"];            

foreach (var pass in customEffect.CurrentTechnique.Passes)
{
	pass.Apply();
					
	customEffect.Parameters["WorldViewProjection"].SetValue(WorldMatrix*ViewMatrix*ProjectionMatrix);

	Game.GraphicsDevice.DrawIndexedPrimitives(PrimitiveType.TriangleList, 0, 0, indexBuffer.IndexCount / 3);
}

I’d like to have the mesh with a texture shader eventually. Any tips would be appreciated.

Not sure what that first shader snippet does … it just puts out a static color on the mesh?

the second one looks technically fine, I am just not sure if you really intend to use the transformed X coordinate which is in Projection Space and I can’t see what sort off coordinates you send into the shader or what you expect to be the output.

if “return MyRed” works without other changes, you at least know, that the mesh is at the expected position and culling is correct. I would recommend working with UV (Texture Coordinates), so every Vertex has a distinct “anchor” and you don’t need to work around with different spaces - always depending on what you want to achieve

Thanks for the reply. The first shader is the one I got from here:

Also as I said, with basic effect I have the correct result:

Game.GraphicsDevice.Clear(Color.CornflowerBlue);
            
basicEffect.View = ViewMatrix;
basicEffect.Projection = ProjectionMatrix;
            
basicEffect.VertexColorEnabled = true;            

vertexBuffer = new DynamicVertexBuffer(Game.GraphicsDevice, typeof(VertexPositionColor), ((VertexPositionColor[])CurveData[0]).Length, BufferUsage.WriteOnly);
indexBuffer = new DynamicIndexBuffer(Game.GraphicsDevice, IndexElementSize.ThirtyTwoBits, ((int[])CurveData[1]).Length, BufferUsage.WriteOnly);

vertexBuffer.SetData((VertexPositionColor[])CurveData[0], 0, ((VertexPositionColor[])CurveData[0]).Length);            
indexBuffer.SetData((int[])CurveData[1]);

Game.GraphicsDevice.SetVertexBuffer(vertexBuffer);
Game.GraphicsDevice.Indices = indexBuffer;

RasterizerState rasterizerState = new RasterizerState();
rasterizerState.CullMode = CullMode.None;            
rasterizerState.FillMode = FillMode.Solid;
Game.GraphicsDevice.RasterizerState = rasterizerState;

foreach (EffectPass pass in basicEffect.CurrentTechnique.Passes)
{
	pass.Apply();                 
	Game.GraphicsDevice.DrawIndexedPrimitives(PrimitiveType.TriangleList, 0, 0, indexBuffer.IndexCount / 3);
}

Which is just a solid colored mesh. The first element of CurveData is the list of VertexPositionColor with a predefined color. I’m passing in the same View and Projection matrices, so I assumend it should work ok for a shader. For UV I would have to use VertexPositionTexture right? I’m not familiar with it, and I don’t know what should be passed as the texture coordinate in my case. For the moment I would be pretty satisfied to have a working shader, which would just give me the same result as with basic effect. And I would continue from there.

Ah ok, so you basically wanna render a mesh just flat with a single color but with a gradient that goes from red to … non-blue, depending on where on the screen the mesh is getting output I guess. For that the position has to end up in Screen Space (0-1)

So in your VS you actually just calculating Coordinates in Projection Space, which is -1 to +1 and that may not be what you expect basing on your Color calculation. But to get this into Screen Space should be as simple as

float3 screenpos = pos.xyz / pos.w

at least if I remember correctly. Now you should have the screen position in 0-1 range for the color calculations.

Be aware tho, doing this in the Vertex Shader will interpolate that Color Value for the PS over a polygon, so I am not sure if this guarantees a nice gradient - if not, send the screenpos to the PS instead and calculate the color for each Pixel there

Another option would be to use the VPOS semantic in the Pixel Shader, which will get you the actual Pixel Position in Viewport Resolution (afaik) but you’d need to divide that by the actual resolution to get a 0-1 value

be aware, you currently also scale the colors alpha-value, which may not matter if you set blendstate to opaque (I think it is by default)