Weird numerical issue with Android shader

Hey all!

I’m re-visiting my moon lander game on Android, as I came up with an idea for the terrain which performs really nicely on my basic tablet. But, I do have a weird problem with my terrain shader.

The terrain itself is a square quadtree, centred on (0, 0, 0), stretching into the XZ plane with a side length of 5000. In game, I prefer to see a circle of terrain rather than a square slab, so I do a simple radial clip in my shader:

float distanceFromCentre = length(float2(input.PositionWorld.x, input.PositionWorld.z)); clip(distanceFromCentre > 2500 ? -1 : 1);

…in Windows, this produces the intended effect (I’ve also added a fade to black around the rim):

…however when I compile and run the same shader for Android, far too much of the terrain is clipped:

I know it’s not a problem with the clip function because when I choose a simpler condition, it works. For example, when I do this:

clip(input.PositionWorld.x > 0 && input.PositionWorld.z > 0 ? -1 : 1);

…I get a quadrant missing in the correct place, while the rest shows up:

So it must be something to do with the line where distanceFromCentre is calculated. I tried just doing a sqrt(x*x + z*z) instead of using length() but I get the same incorrect result! Can anyone throw me a bone here?

Thanks :slight_smile:

…oh and while I’m here, how do I get anti-aliasing working in Android? I have set PreferMultiSampling = true which works for Windows, but not on my tablet.


1 Like

as you used a constant number for the distance check … are you sure, that your game isn’t scaled somehow because of the smaller screen? That bit of visible terrain looks scaled down to me (or the ship scaled up, whatever)

I can’t see how it would be - it’s exactly the same code. Compiling it side by side for Android and Windows yields these different results for the same start conditions. Also everything else in the world like the position of the ship, the hills and valleys, etc are specified in the same coordinates and everything looks normal (as seen in pic 3, when I change the clip code in the shader). Also the bit of terrain in pic number 2 isn’t scaled down - it’s just clipped like crazy.

Is the desktop version also running gl ?
How and were is this calculation taking place on the vertex shader?
is the value being pass from it to the pixel shader?
is it multiplyed by the projection matrix first?
ect ect…

there is not much code here to do anything but guess…

that your game isn’t scaled somehow because of the smaller screen?

He means like maybe the viewport or backbuffer or something is smaller and so you’re formula.

float distanceFromCentre = length(float2(input.PositionWorld.x, input.PositionWorld.z));
clip(distanceFromCentre > 2500 ? -1 : 1);

Would then no longer be invariant on the different platforms if some sort of scaling took place.

If you were to somehow have both the ship and terrain scaled down by saythe m44 W value in your projection matrix because say the graphics device viewport or backbuffer was different then the desktop version. Then the Constant comparison here value > 2500 on one system would no longer evaluate the same as on the other system under what appears to you to be the same or similar visual conditions.

To put it in a simple math analogy
1 / 10 = 0.1 and so does 10 / 100 = 0.1
but xx + yy of 1 and 10 is not the same as 10 and 100

so that while the proportions maybe the same.
the scale is not so the distance is not constant either…

Just a suggestion as i have zero familiarity with androids quirks,take a look at your projection matrix m14 m24 m34 and especially m44 values.
On the shader you may need to divide the by the w value or something.

Personally i copy the position in the vertex shader and pass a copy thru to the pixel shader before it gets altered don’t really trust dividing by w i have had strange results doing that with texture cubes which i still can’t explain what was happening in the gpu.

Not sure if you have some additional filtering going on - but if only the clipping you should have a very sharp edge on the terrain and it looks quite smoothed out to me in the first image), which can’t be prdouced with clipping alone …

the second image is as sharp on the edges as one would expect with clipping and so is the third image

about your multisampling issue, did you try using HiDef Graphics Profile? Not sure if AA is available otherwise.

Hey - thanks for the reply. The desktop version is using DirectX, I think. It’s just a standard MG Windows project.

I’ll add a bit more about how the code works:

The terrain vertex buffers are already in world space, so I never transform them before they’re sent to the shader. Another way of putting this, is that for the terrain, model space and world space are the same thing.

The PositionWorld value sent into the pixel shader is literally just the data taken straight from the vertex shader input: PSinput.PositionWorld =;

So, I expected (maybe incorrectly?) that when I access PositionWorld in the pixel shader, I should just get the pixel position in world space. For example, let’s say I’m working with the far north-east vertex. In my vertex buffer, this is stored as (2500, 2500) (along with a y component, but we can ignore that). This vertex gets sent into the vertex shader, where it is picked up as VSinput.Position. I send it straight through to the pixel shader as above, where I expect PositionWorld to still be (2500, 2500) since it’s never multiplied by any matrices; simply passed straight through from the vertex buffer.

At what point would this scaling be happening? Is it when I send the vertices into the shader?

reiti - the first shot does have a fade effect around the edge, yes. I clip into a circle and then fade the last few hundred metres before the edge to black.

out of curiosity, what happen’s when you do:

clip(input.PositionWorld.x > 2500 || input.PositionWorld.z > 2500 ? -1 : 1);

just to see if where that number actually is. That’s normally my approach to eliminate step by step

I tried that too - it works fine :joy: The result is exactly the same as if there’s no clipping. I moved it in a bit closer too (> 2300) and that works as expected! Also I put a fade effect on the edge which fades to black between a PositionWorld of 2300 and 2500, and that works!

…plus the texturing is done using PositionWorld as well. I don’t have texture coordinates on the vertices - I just use PositionWorld.xz multiplied by a scale factor.

All I can think is that GLSL floats don’t like large/small values for some reason? A precision or truncation thing?

I think you should try running it on the OpenGL Desktop platform to test the GL shader there. Many times the shader behave differently betweeng DX/GL. Usually it’s issues with floating / int conversions or truncating vectors.

try to be more explicit with floats, like:
e.g. clip(distanceFromCentre > 2500f ? -1f : 1f);

i regulary get compiling errors with “1f” … have to write it as “1.0f” or “1.0” - never investigated why, I just accepted that fact.

Now that someone else said it, I 100% agree,

I think you should try running it on the OpenGL Desktop platform to test the GL shader there

I have already hinted at a band aid fix but giving it as advice would be going down the wrong path. You should make a Gl project if you band aid fix the dx project you wont make a gl project and later you will have a problem that is just as bad and just as hard or worse to debug for that reason. Or something that is far more subtle and spin your wheels into a corner before you catch it.

line up the projects to both be gl first then come back. You should have a set of possibly similar problems to the android project or different problems altogether which is expected.

Good idea - I just compiled a GL project, and, drum roll…

It works :sweat_smile: There are also some subtle texturing problems on the Android one which aren’t present here, no doubt related to the same thing.

I’m stumped!

well next up do the length processing manually :slight_smile:

float dist = sqrt(dot(vector, vector));
float dist = sqrt(xx + zz)

you can also ommit the sqrt and compare with 2500*2500 instead - gives you speed improvement

(correct me, if my math is wrong)

Humm on GL desktop that is unexpected.

K multiple suggestions here,

// ensure the texture filters on the shader are set to point clamp to test, i don’t see how this is the cause.

Texture2D TextureA; // primary texture.
sampler TextureSamplerA = sampler_state
    magfilter = Point;
    minfilter = Point;
    mipfilter = Point;
    AddressU = clamp;
    AddressV = clamp;
    texture = <TextureA>;

I think that value for the distance is coming back different that would make sense so make sure it cant…

Add a additional variable to your vertice shader output struct.

struct VsOutput
    float4 Position     : SV_Position;
    float4 Position3D    : TEXCOORD0;  // <<< for bypassing any gpu vertex to pixel shader alterations.

In the vetex shader set it from the input position and mulitply by the world but not the view or projection.

output.Position3D = mul(input.Position, World);

pass that into the pixel shader and compare it to the origin of visibility radius which is probably…
the ship’s world position itself.

float3  dif = -;
clip( length(dif) - 2500.0f)
{  //...

If that doesn’t work id say post up the whole shader.

reiti - yup, already tried both of those right at the beginning! Same result, which is very confusing! Interestingly if I omit the square root and compare to 2500*2500, it’s the same as if there was no clipping at all. So the problem’s reversed - from too much clip to not enough! Works fine on Windows of course.

willmotil - thanks for the suggestions, will have a look … tomorrow :grin: I’m already passing the Position3D to the pixel shader as a TexCoord though.

There’s actually a similar version of the whole shader (actually, an old version of the whole windows project) on GitHub here:

I’ve since removed that 3x3 filter for the shadow map and changed all texture filtering to linear and wrap/clamp to keep it simple for mobile, but the part causing the problem is still exactly the same.

PS: I also just compared the contents of the compiled XNB effect files for Android and DesktopGL, and they are exactly the same. Actually I tried putting the ‘broken’ shader in place of the working DesktopGL one, and it still works fine on desktop!

Interestingly if I omit the square root and compare to 2500*2500, it’s the same as if there was no clipping at all.

That is really odd. Because it should yield the same result … I think there is a overflow happening.

I once had a case on some android devices where suddenly rendering did not happen outside of coordinates ~32000. Turns out some hardware was just not able to work with such numbers (I guess there were fixed point math pipelines or something)

If we suppose that 2500x2500 makes an overflow (because that’s 6.25 mil) on that phone hardware (whatever it is), it would do the same overflow in the length function.

Are you able to work with smaller numbers just for a test?

Is you’re scene preset ?
By that i mean are you actually moving the ship around in the scene also Is the location different for were the ship is in the world in one project from the other ?
Im imagining this is the case.

More importantly is this the actual distance formula you are using atm?
This seems extremely odd to use this as a calculation unless im missing something and i don’t see how.
This is a world distance calculation to the system coordinates origin not the actual ship or a view frustrum related position.

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
     output.PositionWorld = input.Position;


float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
      // these are still just coordinates relative to the system origin not the ship or a view frustrum or world origin in motion.
      float distanceFromCentre = length(float2(input.PositionWorld.x, input.PositionWorld.z)); 
      clip(distanceFromCentre > 2500 ? -1 : 1);

the above should look something like the below.

// global variable is missing and nothing is calculated for a point on the surface so.
float3 LightPosition;

    float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
    float distanceFromCentre = length( - LightPosition);
    float range = 2500.0f;
    clip(  ( distanceFromCentre * -1.0f  ) + range);

You don’t need an if else either the clip is basically a if else it’s just working off the sign.
Anything below zero or negative is clipped just flip the distance and add the range.

    // limited lighting by distance from light origin.
    float3 dif = WorldLightPosition - input.Position3D;
    float range = 20.0f;
    clip( length(dif) * -1.0f + range);

if i had done it using that formula you posted the results would be fixed to the center of the terrain or the world origin of 0,0,0 not the light.which is itself rotating around and above the system center.

Hey willmotil - there might be a misunderstanding with what exactly the clip is meant to do. It’s only being done on the terrain (there’s a totally separate effect for the ship), and yes, it’s meant to be in world space and totally independent of where the ship’s located.

The terrain is a 5000x5000 (metre) square, and it’s centered on the world space origin of 0, 0, 0. The vertices in the terrain are placed in world space too, so there’s no need to transform them with a matrix to place them correctly; just send them straight through to the shader. That’s what input.Position is.

The clip is meant to take the square terrain and make it appear circular, just for aesthetics. That’s why I have a radius of 2500 - any pixel more than 2500m from the world origin in the x-z is clipped to create the circle you see in the working version.

…also, I looked again at the compiled XNB file, and at the top there is this:

#ifdef GL_ES precision mediump float; precision mediump int;

Reading about this, mediump supposedly can’t go higher than 2^14 (16384). Perhaps this is it…
If so, how can I enforce highp for floats? Is it likely that a mobile won’t support highp?

i see that makes more sense.

Reading about this, mediump supposedly can’t go higher than 2^14 (16384). Perhaps this is it…
If so, how can I enforce highp for floats? Is it likely that a mobile won’t support highp?

Ya one way to find out quick since your map has a set limit just divide the x y by some decent sized amount like 70 then square it against 2500/70.

Dunno if this is really a bug or just a limitation.

Aha, finally!

I divided the distances by the radius of 2500 and then clipped anything greater than 1:

float radialDistance = length(float2(input.PositionWorld.x, input.PositionWorld.z)/2500); clip(radialDistance > 1 ? -1 : 1);

Weird as I thought I tried that earlier on. I think the problem was appearing twice though - once when I clip, and once when I apply the fade effect. Now they both work fine.

…so I guess I’ll go through my shaders and look for anywhere a number will exceed 16,384! What a strange thing to have to do on modern hardware, but it seems like this ‘lowp, mediump, highp’ thing is just part of GLSL and I don’t want to make something that will fail on some devices just on that basis.

Thanks everyone for helping me to suss this out :innocent: This community is super useful, even for graphics stuff that’s not specific to MG.

1 Like