Any texture after the first one, needs to be passed as a parameter into a format similar to what I suggested. This can be done right after loading the effect. fx.Parameters[“MaskTexture”].SetValue(MaskTexture); - but you’ll need to modify the sampler in the way I mentioned above.
Also I’m pretty sure the pixel shader input must match exactly to the vertex shader output (order too).
Probably everything else about your code will be identical to before and should work the same.
There is no vertex shader, so matching the output isn’t really a thing. That appears to be fine, too, as I was able to use just the pixel shader to color the image (set it to a solid color).
I’ll try the texture. In the original code, I didn’t pass either texture in. Instead, I set two textures in the graphics device:
GraphicalDevice.Textures(0) = inittarget
GraphicalDevice.Textures(1) = mStencil
where the initarget is the background texture. This results in a tan rectangle in most of the tests I ran. Since tan would be the default background color, in this case, I would assume that pretty much means that no shading happened at all. That was the code that worked in XNA4.0, so it may have been a difference between DX9 and DX11.
The change between your sampler MaskSamp and mine had no impact.
One thing I noted was that my mask wasn’t really a mask at all. I was working with the A value, but my PNG was using black for the transparent areas, with an alpha value of 255, so ALL pixels should have been drawn. There was no masking going on at all. There are a couple ways I could have fixed that, but what I ended up doing was just creating a new mask where the transparent parts were actually transparent. The alternative would be to multiply by R, G, or B, rather than A, since those would all be 0 if black was the transparent area.
Swapping to the transparent image had no impact, either. At this point, I’m pretty much baffled. This is the pixel shader I’m currently working with in an attempt to get just the mask to draw on the screen:
float4 PSSix(float2 texCoord: TEXCOORD0) : COLOR0
{
float4 bitMask = tex2D(MaskTexture, texCoord);
return float4(bitMask.a,0,0,bitMask.a); //
}
All that does is results in a red rectangle, which indicates, to me, that for every texCoord, the color returned from the MaskTexture is (?,?,?,1), where I neither know nor care what the RGB portion is.
My understanding of the tex2D method is that it gets the value from the texCoord of the mask texture. In this case, the mask is 128x128, with a bunch of it being transparent (a = 0) while most is opaque (a=1). Therefore, if I am understanding tex2D correctly, for this shader to be producing a solid red rectangle of size 128x128 (the size of the area being drawn), either texCoord would have to always be the same thing (a point somewhere in the middle of the texture), or the texture being sampled is not what I thought it was.
I guess the next step is to determine which of those two scenarios is the right one.
SpriteEffect.fx in monogame contains the default vertex shader using the format I mentioned. Gfx card / pipeline won’t do pixel shading without one because it needs to know what to do with the vertices first. It may work but to be safe, I’d probably try to match the output data to the input data. I always see everyone doing that.
GraphicalDevice.Textures(1) = mStencil
Passing Texture(0) [s0] will work but passing textures 1,2,3,4,5… will not work this way. At least I’ve never seen it happen yet. Ppl always pass these textures using something like: fx.Parameters[“MaskTexture”].SetValue(MaskTexture);
And then using the sampler style I mentioned before.
Also you may want to check your Blendstate and ensure it’s matched to the texture build in Content. ie: If Content has the texture set to Premultiply alpha, then you’d use AlphaBlend otherwise it would be NonPremultiplied. Sometimes one doesn’t match and it can mess things up a bit - in this case I’d say that likely isn’t the problem.
1 Like
I believe you need to define both a texture and a sampler2D in your shader and then use the sampler when you call tex2D.
texture MaskTexture: register(t1);
sampler2D MaskSampler : register(s1)
{
Texture = (MaskTexture);
MipFilter = None;
AddressU = Clamp;
AddressV = Clamp;
};
float4 PSSix(float2 texCoord: TEXCOORD0) : COLOR0
{
float4 bitMask = tex2D(MaskSampler, texCoord);
return float4(bitMask.a,0,0,bitMask.a); //
}@Zarquon: That change had no noticeable impact. I believe it is doing roughly the same thing in a slightly more verbose fashion, but that’s not a bad thing.
@AlienScribble: Using the VertexShader argument got me considerably closer. I’m now seeing the background, but the mask is not masking anything. There are several pieces I need to check on that to see what is going on, but at least I’m now seeing something recognizable, which makes it far easier to understand.
Changing from AlphaBlend to NonPremultiplied had no impact.
For test purposes, this was the PixelShader I was using, which I expected to just show the bitmask:
float4 PSSix(VSOutput input) : COLOR0
{
float4 bitMask = tex2D(MaskSampler, input.uv);
return float4(bitMask.b,0,0,1); //
}
The result was this:
What this is showing is the background (the fish, the slightly wavy appearance behind the fish, and the circle). The grey scale square, the text, and the tan are all drawn afterwards.
If you look at the shader, all I’m doing is sampling the mask, which is either a transparent and white, or black and white image (I have both, and have tried both, with the same result). Instead of using the A value, I left that at 1, and used the B value (in place of the R, which is why the whole thing looks red). Note that the red color is not bright red. That B value is not 1. I have a different example where I do use 1 rather than the B value, and it is a clearly different, brighter red.
So, what I’m seeing is that the entire image has a solid, semi-transparent, layer on top of the correct underlying layer. Using the VertexShader output as an argument got the proper background showing, which is certainly a step in the right direction, but the output is also clearly blending in the same value for all pixels, not a sample taken from the texture, as far as I can see.
I’m clearly not understanding something, as I would expect that pixel shader to simply return a variation on the mask itself (either transparent and red or black and red, depending on the mask used). I’m largely ignoring the input, aside from using it to identify a point in the mask texture, yet I’m seeing a semi-transparent overlay on the background.
Yah, that’s odd.
The shader seems fine so I’d try reducing other possible causes.
I’m guessing you already tried changing this:
GraphicalDevice.Textures(1) = mStencil
to this(?):
fx.Parameters[“MaskTexture”].SetValue(mStencil);
And you said it’s not being processed as a premultiplied texture in the Content builder and you’re using non-premultiplied blendstate… so I’m wondering how stuff around the draw part of your code looks. Maybe there’s some other clues there?
I’m back to this, with some semi-new information. So, just to reiterate, this is the drawing code:
GraphicalDevice.SetRenderTarget(fishTarget)
'This line sets the backcolor to the backcolor of the control, which would mean that anything outside the stencil will appear as the background color
fishTarget.GraphicsDevice.Clear(RUGroupBackColor)
mSprites.Begin(SpriteSortMode.Immediate, BlendState.AlphaBlend)
Dim useEffect = mEffect
Dim snShrt As Single = shrt
useEffect.Parameters("MaskWidth").SetValue(snShrt)
useEffect.Parameters("MaskHeight").SetValue(snShrt)
useEffect.Parameters("BaseTextureWidth").SetValue(snShrt)
useEffect.Parameters("BaseTextureHeight").SetValue(snShrt)
useEffect.Parameters("BaseTexture").SetValue(initTarget)
useEffect.Parameters("MaskTexture").SetValue(mStencil)
useEffect.CurrentTechnique = useEffect.Techniques("Technique1")
useEffect.CurrentTechnique.Passes(0).Apply()
'Merge the initTarget (which has the fish, the RWPB, and the border) into the current.
mSprites.Draw(inittarget, New Rectangle(0, 0, shrt, shrt), Nothing, Color.White, 0.0, Vector2.Zero, SpriteEffects.None, 0)
mSprites.End()
What I’m trying to do as a test is to SOLELY draw the stencil, thereby ignoring the underlying texture. To do this, I’m using this very simple technique:
float4 PSSix(VSOutput input) : COLOR0
{
float4 bitMask = tex2D(MaskSampler, input.uv);
return float4(bitMask.r,bitMask.g,bitMask.b,bitMask.a); //That's a slow way of returning just the pixel from the mask.
}
Essentially, I’m ignoring the input aside from using the coordinates in the call to tex2D. What I
am returning is just the float4 (poorly, as the comment notes).
I also have this in the file (along with 5 other pixel shaders so that I can try different things without recompiling the effect):
texture MaskTexture : register(t1);
sampler2D BaseTexture : register(s0);
sampler2D MaskSampler : register(s1) {
Texture = <MaskTexture>;
addressU = Clamp;
addressV = Clamp;
};
struct VSOutput{
float4 pos : SV_POSITION;
float4 col : COLOR0;
float2 uv : TEXCOORD0;
};
As far as I can see, what this SHOULD be doing is getting the pixel from the MaskSampler at the uv coordinates supplied. I then went to look at the stencil itself as a Texture2d (as opposed to the png that it loads from), and it looks pretty good. Not ideal, as some anti-aliasing could be less than what I want, but there are large swaths of black pixels (0,0,0,0) and white pixels (255,255,255,255).
What is coming out when I use the code shown is whatever is in the background (initTarget).
I kind of think that I understand why, because the final line in the code is this:
mSprites.Draw(initTarget, New Rectangle(0, 0, shrt, shrt), Nothing, Color.White, 0.0, Vector2.Zero, SpriteEffects.None, 0)
That will draw whatever is in initTarget. I was assuming that the effect was altering what was in initTarget via the technique. That doesn’t appear to be the case. If I comment out that line, then I get my standard tan background color, which would suggest that whatever the effect is doing, it is not changing initTarget, or whatever changes it is making are not being displayed.
Basically, that final line seems to be everything that is done in the code block. If it is there, then the render target gets whatever was in initTarget. If the line isn’t there then the render target gets nothing. Commenting out all the code that deals with the effect has the exact same result. It is as if the effect was doing nothing at all.
The only argument against that is that if I use a technique that simply returns a red pixel as the pixel shader, I get a solid red square, which means that the technique IS doing something, it just doesn’t do what I’d expect. In fact, it kind of acts like the MaskTexture is being set to the background image, not to the stencil image.
I realize this is long, but I’ve been studying this over and over and there’s a lot of data. Any suggestions?
Considering it optimizes out what isn’t used. Maybe it also does something weird where it wants to make use of slots in order of appearance depending on usage. As an experiment, I’d try commenting BaseTexture s0 if it’s not used (at the moment), and try changing Mask stuff to s0, t0 and see if it samples as expected. Another experiment to try: return bitmask * 0.96 + tex2D(BaseTexture, input.uv) * 0.04
If the experiment works, it’ll show 96% of the bitmask result and hint(4%) that the basetexture is being sampled. If this happens, then you know that not using the BaseTexture has something to do with it whatever weirdness is going on with where it’s sampling from.
At least it’s a way to figure out what might be happening anyway. ;p
You could take a look at this maybe it will help.
Despite the name of the post this is just a shader and game1
I made this as a example to show how one could do a 2d version of a planet scrolling.
Or a scrolling background behind say a window in 2d.
With transparency and color blending thrown in as well ect.
1 Like
@AlienScribble: Some progress, I believe, but so many tests that I’m not quite sure of the outcome. The key point was that I didn’t quite follow your first suggestion. I commented out BaseTexture, changed mask to s0, but neglected to change t1 to t0. The result was no change. I then…well, then I tried the second suggestion which resulted in no change. No change just means that the whole rectangle was the background image with a circle on it plus fish, as seen earlier.
I then noticed my oversight and repeated the first suggestion while changing t1 to t0. Using this PS technique:
float4 PSSix(VSOutput input) : COLOR0
{
float4 bitMask = tex2D(MaskSampler, input.uv);
return float4(bitMask.r,bitMask.g,bitMask.b,bitMask.a); //That's a slow way of returning just the pixel from the mask.
}
The result was this:
That’s the mask, and looking right. No background image, but with that technique, which ignored the background image, there shouldn’t have been a background image. So, progress.
I felt that the next thing to try was a brute force approach to the background, which was this:
float4 PSFive(VSOutput input) : COLOR0
{
float4 bitMask = tex2D(MaskSampler, input.uv);
float4 tex = tex2D(BaseTexture, input.uv);
return float4(tex.r,tex.g,tex.b,bitMask.a);
}
There’s no real sampling there, but in the test case, the images are the same size as the target, so it’s a 1:1 mapping. The result was the same as the previous image, except the inside of the circle is flat black.
One other point is that while doing all this, I had changed t0 back to t1 while using this latter technique. That resulted in a solid black square (no masking). I noticed that mistake and turned t1 back to t0, which resulted in the black circle.
From this, I take it that I need to declare the variables like so:
texture MaskTexture : register(t0);
sampler2D BaseTexture : register(s0);
sampler2D MaskSampler : register(s1) {
Texture = <MaskTexture>;
addressU = Clamp;
addressV = Clamp;
};
I can’t say why, though. Clipping happens if MaskTexture is in t0, but not in t1. The PSSix function is just using the sampler, and it looks right, but switching to PSFive, which is very similar to PSSix, except that it uses the base rather than the mask for the RGB values, all I get is black, which suggests that the base is empty.
At this point, I’m using those registers without understanding them, but they seem to hold the key.
@willmotil: Having looked at your example, I’m even more convinced that the key to this problem is the fact that I am throwing this part of my effect file:
texture MaskTexture : register(t0);
sampler2D BaseTexture : register(s0);
sampler2D MaskSampler : register(s1) {
Texture = <MaskTexture>;
addressU = Clamp;
addressV = Clamp;
};
at the problem without understanding what that does. I believe that the pixel shader function is reasonably straightforward, but I’m not setting up the textures in the proper fashion. I see that you did very similar things (you used Wrap rather than Clamp, but I think I understand that part), but not quite the same.
I’m looking for a better understanding of what those few lines of code are doing. I have a couple books on the way that might help, but any explanation/insight would be valuable, at this point. That’s most likely where my issues lie, and I’m throwing code at it blindly, at this point.
Ah, I forgot something. I think someone found out that for some weird reason, the order that you sample in the pixel shader should start with s0’s sampler. I think I’ve never noticed it myself because coincidentally the order has always matched up for me.
Btw - you should be able to do like this (I’ve never found a need for the Texture and ‘T’ register stuff):
sampler2D BaseSampler : register(s0) {
Texture = <Texture>; // this will be set by spritebatch
}
sampler2D MaskSampler : register(s1) {
Texture = <MaskTexture>; // passed in
addressU = Clamp; // resample at the edge of the texture if trying to sample past the edge
addressV = Clamp; // if using Wrap, it'll start sampling from the other side, if mirror it'll go backward
};
The part I’m about to show is actually what I’m talking about.
For whatever reason I think s0 needs to be sampled first and then s1:
So instead of this:
float4 bitMask = tex2D(MaskSampler, input.uv);
float4 tex = tex2D(BaseSampler, input.uv);
//You might need to order them like this instead (I know it doesn't make sense):
float4 tex = tex2D(BaseSampler, input.uv);
float4 bitMask = tex2D(MaskSampler, input.uv);
I think it might be an s0 thing since I’ve not noticed the order being important with other textures.
Like I said, it’s never happened to me before but I always end up sample s0 first so that may be why.
I have heard of someone noticing this before.
Maybe someone can confirm this?
1 Like
I wrote a very lengthy reply before you wrote this. I was methodically testing a variety of different changes and getting a fairly comprehensive picture of the shape of failure…and then it mostly worked. I knew that I was not using the right Technique, as I had six different Techniques all with slight changes. Once I switched to the right one, it worked completely.
So, what I found was that this was what was essential:
#define PS_SHADERMODEL ps_4_0_level_9_1
texture MaskTexture : register(t0);
sampler2D BaseTexture : register(s0);
sampler2D MaskSampler : register(s1) {
Texture = <MaskTexture>;
addressU = Clamp;
addressV = Clamp;
};
struct VSOutput{
float4 pos : SV_POSITION;
float4 col : COLOR0;
float2 uv : TEXCOORD0;
};
//
float SomeValue;
float MaskLocationX;
float MaskLocationY;
float MaskWidth;
float MaskHeight;
float BaseTextureLocationX; //This is where your texture is to be drawn
float BaseTextureLocationY; //texCoord is different, it is the current pixel
float BaseTextureWidth;
float BaseTextureHeight;
float4 PixelShaderFunction(VSOutput input) : COLOR0
{
//We need to calculate where in terms of percentage to sample from the MaskTexture
float maskPixelX = input.uv.x * BaseTextureWidth + BaseTextureLocationX;
float maskPixelY = input.uv.y * BaseTextureHeight + BaseTextureLocationY;
float2 maskCoord = float2((maskPixelX - MaskLocationX) / MaskWidth, (maskPixelY - MaskLocationY) / MaskHeight);
float4 bitMask = tex2D(MaskSampler, maskCoord);
float4 tex = tex2D(BaseTexture, input.uv);
//It is a good idea to avoid conditional statements in a pixel shader if you can use math instead.
return tex * (bitMask.a); //
}
I’m still baffled by the line putting the texture into t0. If that line is commented out, then no masking happens at all. In fact, it’s as if the effect was doing nothing whatsoever. However, the code runs, and one line of that code is telling:
useEffect.Parameters("MaskTexture").SetValue(mStencil)
If you comment out a variable that is used, then there will be no such parameter, and attempting to set a parameter that doesn’t exist will throw an exception, as it should. So, the fact that this runs when t0 is commented out is significant because it means that MaskTexture is still defined, which it is, as part of the MaskSampler. Also, note that MaskTexture is not used in the Technique, only MaskSampler (which contains MaskTexture) is used in the Technique.
So, putting the stencil into t0 is not necessary for the parameter to exist, but it IS necessary to get the effect to work. Without that step, it does nothing at all. That would suggest that it is MaskSampler that isn’t necessary, but ONLY MaskSampler is referenced in the shader, so the effect wouldn’t even compile without MaskSampler. Therefore, t0 is essential for some reason, though it isn’t clear why it should be.
Comparing back to my very first post, which contained my old XNA 4.0 code, I note that BaseTexture was never declared in that shader. That shouldn’t compile, and indeed it does not. It DID work under XNA 4.0, though, so apparently there was some kind of implicit declaration going on back then.
It also looks like a bad shader results in no shader at all. If it compiles, but doesn’t do anything good, then you get no effect, which in this case means that there is no masking at all. If this is true, then it suggests that commenting out t0 causes some kind of internal problem, possibly with setting the texture in MaskSampler. I can still pass in the stencil as a parameter, and the parameter is still there, but since it doesn’t go through t0…what? It isn’t real?
I don’t believe this is the only question, either. Commenting out all the effect code results in just drawing the background, which makes sense, because a call to .Draw with the background is the final line in the Sprite composition (shown in the original post). If the effect is commented out, then all that is happening is drawing an existing image onto a specific render target, and that’s a pretty simple process.
However, I was also able to get nothing to draw at all. At the moment, I can’t recreate those conditions.
The key points I have learned from this thread are:
-
Even though I don’t have a vertex sampler, I still need to pass in the results of the vertex sampler, which indicates that MG is using the default sampler when no other is supplied.
-
I have to put the stencil into t0, even though it is then passed into a sampler that is in s1. I have no idea why this is, but it appears to be essential.
-
If the effect compiles, that doesn’t mean that it will work. This may indicate that internal exceptions due to invalid or incorrect parameters are quietly being swallowed.
Interesting. Perhaps setting t0 is fixing whatever strange thing happens if you don’t sample the base texture first? So I’m guessing maybe base texture is switched into t1 since t0 is already taken?
I never actually use the t0,t1 stuff but I think I always sample from base texture first in pixel shader (s0).
What happens if you comment out the line: texture MaskTexture : register(t0);
And then in the pixel shader:
Place this line as the very first line before any other sampling:
float4 tex = tex2D(BaseTexture, input.uv);
1 Like
If you look back to what I think is post #16, you will see PSFive, which is almost what you are describing, except that I take from the MaskSampler before the BaseTexture.
If I use that PSFive sampler with t0 included I get the correct output, except that the area that should be transparent is white.
If I use PSFive as shown with t0 commented out I get no masking at all. The background image is shown without any stencil.
If I then swap the lines in PSFive such that I get tex prior to bitMask, I’m back to getting the correct output with the white areas where it should be transparent.
float4 PSFive(VSOutput input) : COLOR0
{
float4 tex = tex2D(BaseTexture, input.uv);
float4 bitMask = tex2D(MaskSampler, input.uv);
return tex * (bitMask.a);
}
If I then fix the return line:
return tex * (bitMask.a);
I get the correct output with t0 no longer in use.
The final test is to uncomment t0. This resulted in the correct output, as well, so with tex and bitMask in the proper order (tex coming first), I get the correct output regardless of whether t0 is used or not.
Therefore, you are clearly on to something. You are correct, the order of setting tex vs bitMask makes all the difference, and t0 only makes a difference if bitMask is set before tex. So, what does that suggest? I can clearly get rid of t0 entirely, which is nice, as it simplifies the code, but what exactly was t0 doing when tex and bitMask were being set in the wrong order? Why does the order matter?
It suggests to me that variables in HLSL are a much looser thing than in languages I am used to. The order of setting two independent variables shouldn’t matter, but that test shows that it clearly does. That seems to suggest that the variables BaseTexture and MaskSampler are entwined in some way. Also, if the variables are set in opposite order (s1 is sampled before s0), then adding t0 ‘fixes’ the problem, but if the variables are set in correct order (s0 is sampled before s1), then t0 has no impact whatsoever.
I’d say I learned something, but I sure can’t say what it is that I learned.
1 Like
Yah, it’s strange. I think I was just ordering them correctly by luck. It might be just for the first texture but later textures seem to work in any order.
Oh I just noticed something. It might not be important but,
since spriteBatch is drawing inittarget - internally it sets s0 / t0 texture (BaseTexture) to inittarget already.
This means you can probably comment out this line:
//mEffect.Parameters("BaseTexture").SetValue(inittarget)
I’m not sure it would make any difference though.
You are correct, commenting out the line has no impact, so the line was not worthwhile.
One thing still has me slightly baffled, but I’m not sure how far I want to study it. I have two render targets. initTarget is built up of a series of calls to Draw. This sets the background image, draws on fish if necessary, and draws two concentric circles. Those are the items that remain once the masking has been completed.
As the next step, I do the masking, which is what this thread has really been about. However, I notice that the final line of the masking is the call to Draw where the initTarget is drawn on the current RenderTarget (fishTarget). This step puts the two pieces together, the composite background image in initTarget along with the stencil.
It seems to me that I could add the stencil first, and do all the drawing in one pass. Basically, the way I have it now there is the one call to .Draw to put the initTarget in place, but initTarget is just built up via a series of .Draw statements, so I would think that I could move the stencil code prior to all these .Draw statements and it would work the same.
This is not quite the case. It sort of works, but some of the drawing ends up being kind of odd.
Here’s what it looks like using the two render targets, which is the way I want it to look:
And here’s what happens if I use just a single render target:
It’s not horrible, but not ideal, either. The circle is drawn as a series of points that make up a circle. Therefore, the circles are really a string of beads. When I use the two RenderTarget approach, the string of beads blend together to make a nice circle. When I use the single RenderTarget approach, something about the stencil is causing the string of beads to resolve into individual points.
I’m curious as to why that is. There would be the slightest performance advantage to using a single RenderTarget, but the performance is excellent already, so I’m inclined to go with the better looking approach. I’d just like to understand why adding the pixel shader to that single RenderTarget and doing the composition and stencil in a single step is causing a change in the way the circles are rendered?
EDIT: Actually, upon studying those images a bit more closely in a paint program, I notice that there are other differences, as well. If one looks closely at the fish tails in the two images, you’ll notice that the bottom of the lower tail, and the top of the upper tail have both been chopped off. Upon studying the zoomed image, quite a bit of the lower tail has been removed, while very little of the upper tail has been. Interestingly, those are the ONLY differences: The circles, the lower part of the bottom tail, and just the tip of the upper part of the top tail.
I think I can understand the string of beads result for the circles a whole lot better than I can understand how parts of the fish tails could have been clipped…except that…if I draw a circle onto the image, it would perfectly chop off those parts of the tails if the circle was drawn at the right height. In other words, it looks as if there was masking done on the fish image with a smaller mask than the one used on the whole image, but perhaps of the same, circular, shape. The argument against that is that the snouts of the fish should also have been impacted if the mask were truly circular, and they have not been. To include all of the jaws, the clipping has to have happened with an ellipse that was slightly elongated.
If you are trying to make a circle per draw on the shader you can actually do that algorithmically as a added on function to your pixel shader i made a method for it somewere ill dig it out in a bit.
currentPixel is the current interpolated position sent in from the vertex shader to the pixel shader
testPosition is the center point about which the circle is to be drawn thats gotta be within the actual drawn screen quads for the draw call.
So these are shader functions you put them before the vertex or pixel shaders of your own so they are compiled first and available.
//--- directly draw lines on the shader.
// boils it down basically to a boolean variable 1.0f is true 0.0f is false.
float4 DrawLine(float2 curpixel, float4 currentColor, float2 a, float2 b, float lineThickness, float4 linecolor)
{
float2 p = curpixel;
float t = lineThickness;
float2 c = b - a;
float2 n = normalize(float2(-c.y, +c.x)); // normalized cross ab
float2 i = -(n * dot(n, p - a) * 2.0f); // inflected perp normal
float dist2line = 1.0f - saturate((i.x * i.x + i.y * i.y) / (t * t)); // distance of point to line.
float isinbounds = saturate(sign(dot(a - p, a - b) * dot(b - p, b - a) + 0.001f)); // + 0.0001f determine point is within segment.
float strength = dist2line * isinbounds;
return lerp(currentColor, linecolor, strength);
}
//--- directly draw circles on the shader.
// boils it down basically to a boolean variable 1.0f is true 0.0f is false.
float4 DrawCircleAtRadius(float2 curpixel, float4 currentColor, float2 testpos, float radius, float lineSize, float4 linecolor)
{
float strength = (1.0f - saturate(abs(distance(curpixel, testpos) - (radius - lineSize / 2)) / (lineSize / 2)));
return lerp(currentColor, linecolor, strength);
}
//--- directly draw points on the shader.
// boils it down basically to a boolean variable 1.0f is true 0.0f is false.
float4 DrawPoint(float2 curpixel, float4 currentColor, float2 testpos, float range, float4 linecolor)
{
float2 dif = curpixel - testpos;
float strength = 1.0f - saturate((dif.x * dif.x + dif.y * dif.y) / (range * range));
return lerp(currentColor, linecolor, strength * strength);
}
in the pixel shader
float4 PsShaderDebugDraw(float4 position : SV_Position, float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
{
// ....
// display circular radius about midpoint m.
currentColor = DrawCircleAtRadius(p, currentColor, m, length(c - b) * 0.5f, 3.0f, lineColorYellow);
I made that for this test project the full shader is in there with a game1 that uses it.
1 Like
That’s an interesting approach, I’ll have to think about it.
You would probably have to take a look at the actual pixel shader that is using it to see how i passed positions and stuff from the vs to the ps to be used and how its called and used in the pixel shader
Its basically checking and modifying colors if they fall into the circle radius range from the position specified as the center to draw it from.
It could be fully based on texel positions instead of pixel positions i sort of slopped it together there. but with what your doing it maybe will work better the way it is.
weird dunno why i had to post that as a hyperlink well there you go edited the previous post.