ViewPort Unproject (...) problem.

I just tried my old code from xna, that would turn mouse positions into world coordinates.
It basically used Viewport.Unproject() Im not sure i’ve ever tried to use it in monogame.
Well it doesn’t seem to work as expected any sort of rotation to the view doesn’t translate into world coordinates.

I looked at the code and it appears to be using the xna 2.0 code which was faulty if i remember right.
Either way… I decided to take a look at it and too me it seems that the whole thing is overly complicated.
So i wrote my own version which seems to give usable world positions though the z is slightly off.

Here’s how i was using the original if someone see’s anything wrong.

        // The unproject used inside this is improper 
        public Ray GetScreenVector2AsRayInto3dWorld(Vector2 screenPosition, Matrix projectionMatrix, Matrix viewMatrix, Matrix world, float near, float far, GraphicsDevice device)
        {
            Vector3 farScreenPoint = new Vector3(screenPosition.X, screenPosition.Y, far); // the projection matrice's far plane value.
            Vector3 nearScreenPoint = new Vector3(screenPosition.X, screenPosition.Y, near); // must be more then zero.
            Vector3 nearWorldPoint = Unproject(nearScreenPoint, projectionMatrix, viewMatrix, world, device);
            Vector3 farWorldPoint = Unproject(farScreenPoint, projectionMatrix, viewMatrix, world, device);
            Vector3 worldRaysNormal = Vector3.Normalize(farWorldPoint - nearWorldPoint);
            return new Ray(nearWorldPoint, worldRaysNormal);
        }

        // This is worthless the coordinates aren't rotated in world space by the view matrix.
        // so the whole thing becomes junk.
        public Vector3 Unproject(Vector3 source, Matrix projection, Matrix view, Matrix world, GraphicsDevice gd)
        {
            Matrix wvp = Matrix.Multiply(Matrix.Multiply(world, view), projection);
            Matrix inv = Matrix.Invert(wvp);
            Vector3 clipSpace = source;
            clipSpace.X = (((source.X - gd.Viewport.X) / ((float)gd.Viewport.Width)) * 2f) - 1f;
            clipSpace.Y = -((((source.Y - gd.Viewport.Y) / ((float)gd.Viewport.Height)) * 2f) - 1f);
            clipSpace.Z = (source.Z - gd.Viewport.MinDepth) / (gd.Viewport.MaxDepth - gd.Viewport.MinDepth);
            Vector3 invsrc = Vector3.Transform(clipSpace, inv);
            // k but were is my translation from view space.
            float a = (((clipSpace.X * inv.M14) + (clipSpace.Y * inv.M24)) + (clipSpace.Z * inv.M34)) + inv.M44;
            if (!WithinEpsilon(a, 1f))
            {
                invsrc.X = invsrc.X / a;
                invsrc.Y = invsrc.Y / a;
                invsrc.Z = invsrc.Z / a;
            }
            return invsrc;
        }

Here is the version i made with comments.
Which makes me wonder why they are doing all the extra stuff.

        // uses my unproject.
        public Ray MineGetScreenVector2AsRayInto3dWorld(Vector2 screenPosition, Matrix projectionMatrix, Matrix viewMatrix, Matrix world, float near, float far, GraphicsDevice device)
        {
            Vector3 farScreenPoint = new Vector3(screenPosition.X, screenPosition.Y, far); // the projection matrice's far plane value.
            Vector3 nearScreenPoint = new Vector3(screenPosition.X, screenPosition.Y, near); // must be more then zero.
            Vector3 nearWorldPoint = UnprojectMine(nearScreenPoint, projectionMatrix, viewMatrix, world, device);
            Vector3 farWorldPoint = UnprojectMine(farScreenPoint, projectionMatrix, viewMatrix, world, device);
            Vector3 worldRaysNormal = Vector3.Normalize(farWorldPoint - nearWorldPoint);
            return new Ray(nearWorldPoint, worldRaysNormal);
        }

        // We don't even need the projection or view matrix for world coordinates. 
        // Just the camera world matrix. Basically a CreateWorld(...) using the cameras values.
        public Vector3 UnprojectMine(Vector3 screenPosition, Matrix projection, Matrix view, Matrix world, GraphicsDevice gd)
        {
            // Gonna do this the easy hard way piece by piece.
            var vp = gd.Viewport;
            Vector3 s = screenPosition;
            // Well jump out of screen space to clip space.
            var cx = (((s.X - vp.X) / vp.Width) * 2f) - 1f;
            var cy = -((((s.Y - vp.Y) / vp.Height) * 2f) - 1f);
            // this looks slightly off to me im not sure this is completely proper to think of it like this.
            // i mean the user sees the near plane as basically zero depth even if its not so i dunno.
            // then again maybe im not to sure yet.
            var cz = (s.Z - vp.MinDepth) / (vp.MaxDepth - vp.MinDepth);
            //
            // Ok at this point we have cliped space mouse coordinates ... ? right .... ?  hummm....
            //
            // However... we haven't reversed the w ...
            // In truth we haven't ever applied the w either...
            // We never had to go thru view projection space.
            // Which means, the coordinate system we are actually within is ? ...
            // ModelSpace... ? 
            // lets see if that is true.
            // If c xyz is actually in model space then our screens offset currently is actually the camera space but not the view space.
            // Such that the camera world multiplied by these values will yeild ... 
            // world space coordinates that are defined by the camera as the origin.
            //
            Vector4 v = new Vector4(cx, cy, cz, 1.0f);
            var vw = Vector4.Transform(v, world);
            // Are we there yet ? Let's see.
            // if we are then, We can pass this to create world. Make a final wvp matrix. Multiply a forward pointing quad to get a proper visual mouse point at a specified z depth.
            return new Vector3(vw.X, vw.Y, vw.Z);
        }

Ironically i wrote this straight out and it seems to be nearly working right off the bat the z depth is off it seems though.
The worst part off the monogame one is that weird stuff happens when the system coordinates changes as the camera crosses into positive z system coordinates. Unless im doing something wrong, if anyone knows how to use the monogame version properly let me know, cause either im misunderstanding something or its messed up.

Are you maybe passing the camera’s world matrix into the world parameter of Viewport.Unproject? That would explain why view rotations have no effect. If you want world coordinates you need to pass Matrix.Identity here. The world parameter is used to get object space coordinates, it’s supposed to be the world matrix of some object.

You can’t really get correct results without using the projection matrix. The same screen coordinates can correspond to different world coordinates, if the camera’s field of view is different, so it needs to be taken into account.

hope this help:

    Matrix _world;
    Matrix _objTransform;
    void MTW1()
    {
        Vector3 prje = GraphicsDevice.Viewport.Project(Vector3.Zero, _projection, camera.View, _world * _objTransform);
        Vector3 uprje = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, prje.Z), _projection, camera.View, _world);
        _objTransform = Matrix.CreateTranslation(uprje);
    }
    void MTW2()
    {
        Vector3 prje = GraphicsDevice.Viewport.Project(camera.Position + Vector3.Normalize(camera.Forward) * 5, _projection, camera.View, Matrix.Identity);
        Vector3 uprje = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, prje.Z), _projection, camera.View, _world);
        _objTransform = Matrix.CreateTranslation(uprje);
    }
    void MTW3()
    {
        Vector3 uprje1 = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, 0.1f), _projection, camera.View, _world);
        Vector3 uprje2 = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, 1.0f), _projection, camera.View, _world);
        Vector3 uprje = Vector3.Normalize(uprje2 - uprje1) * 5 + uprje1;
        _objTransform = Matrix.CreateTranslation(uprje);
    }

    //to object world
    Ray GetRay()
    {
        Vector3 uprje1 = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, 0.1f), _projection, camera.View, _world * _objTransform);
        Vector3 uprje2 = GraphicsDevice.Viewport.Unproject(new Vector3(MousePosition, 1.0f), _projection, camera.View, _world * _objTransform);
        return new Ray(uprje1, Vector3.Normalize(uprje2 - uprje1));
    }


     effect.World = _world * _objTransform;//_mod

Ah i see i just tested mine with the Fov changable from the keyboard and it did indeed move the near point in and out though minorly. Is it possible to directly use the m11 m22 m33 from the projection matrix to adjust for that ?

… The original version… The far point though…
Another problem im seeing with the original Unproject besides the rotation isn’t accounted for, you can see below. You can see whats going on with the far point as it crosses the system 0 z axis boundry coordinates. Take a look at the matrix z depth m43 in the image as it goes negative and positive. Worse is when the result is not within epsilon you basically get a nan or something cause all i see is a straight line up and down.

this is actually the original with a minor correction without that it does even worse stuff.

Im taking a look at this now again im not sure i fully understand what pumkin did there yet but ill give it a go.

And what about my first remark? Is the world matrix you pass to Unproject Identity? If it’s not Identity, you will not get correct world coordinates.
In your custom version of Unproject the world parameter is the camera world matrix, as you state in your comments. That’s not what the world parameter in the original Unproject is for.

That was one of the first things i tried.

Im just drawing the ray as a quad when it comes out of the function.
Then moving around in the world in this case just forward backwards.

instanceObjMouseRay.Scale(.01f, .01f, 150f);
instanceObjMouseRay.Position = worldRay.Position;
instanceObjMouseRay.Forward = worldRay.Direction; 
instanceObjMouseRay.CreateTheFinalWorldViewProjection(viewSpaceTransformer);
DrawForwardQuad(GraphicsDevice, effect, viewSpaceTransformer.GetWorldViewProjection(instanceObjMouseRay), textureOrient);

http://i936.photobucket.com/albums/ad207/xlightwavex/miscellaneous/UnprojectOddity02_zpspiw2hiet.gif


This is what the function gives with matrix identity for the world.
Remember this function is supposed to return world coordinates from screen coordinates.

The rotation is not accounted for either with that.

The problem with this function im seeing is Not with any point created on the near plane, that is working as you can see in the image and is basically correct.

The problem with this function is with points that are placed on the far plane.

Rotation of the view matrix isn’t accounted for.
The meaning of what matrix is meant by “world” in this functions parameter context is ambiguous at best.
Their is a weird flip as the view matrix m43 value goes from negative to positive and values within epsilon can occur as well, that really mess the unproject up.


world

    //Matrix wvp = Matrix.Multiply(Matrix.Multiply(world, view), projection);
    // same results
        Matrix wvp = Matrix.Multiply(view, projection);

public Vector3 Unproject(Vector3 source, Matrix projection, Matrix view, Matrix world, GraphicsDevice gd)
{
Matrix wvp = Matrix.Multiply(Matrix.Multiply(world, view), projection);
Matrix inv = Matrix.Invert(wvp);
Vector3 clipSpace = source;
clipSpace.X = (((source.X - gd.Viewport.X) / ((float)gd.Viewport.Width)) * 2f) - 1f;
clipSpace.Y = -((((source.Y - gd.Viewport.Y) / ((float)gd.Viewport.Height)) * 2f) - 1f);
clipSpace.Z = (source.Z - gd.Viewport.MinDepth) / (gd.Viewport.MaxDepth - gd.Viewport.MinDepth);
Vector3 invsrc = Vector3.Transform(clipSpace, inv);
float a = (((clipSpace.X * inv.M14) + (clipSpace.Y * inv.M24)) + (clipSpace.Z * inv.M34)) + inv.M44;
if (!WithinEpsilon(a, 1f))
{
invsrc.X = invsrc.X / a;
invsrc.Y = invsrc.Y / a;
invsrc.Z = invsrc.Z / a;
}
return invsrc;
}

If world isnt supposed to be the camera’s world via create look at what is it supposed to be ?
I don’t get it. What would be the point of sending in the world matrix at all, is this supposed to be the mouse positions as a world matrix ?
I dont think a inverse transform will yeild the applied translation on a view matrix in order to rotate a point. As once you build the view matrix the create look at function changes the translation into view space coordinates they aren’t the same as what went in.

The world matrix of some object. Unproject returns object-space coordinates, unless you use Identity.

Is that ViewProjection debug output in the video supposed to be a rotating camera? With only 2 changing values that can’t be right.

Yes im only moving forward and back to demonstrate specificly the additional odd results.

Ignore the y axis rotation string in the image thats just for the object in the distance that is spining. So is oscillation move and cycle.

http://i936.photobucket.com/albums/ad207/xlightwavex/miscellaneous/Bugs/UnprojectOddity03_zpsrwaww1hs.gif

would you mind upload the project?

Sure let me clean it up a bit and make a more specific test case so its clear that its directly the function itself. Ill upload it soon as i do. I have crap all over this one from testing different methods and adjustments. The class i was using to test had more specific tests and more of them but it would of been to confusing to look at.

Edit…

Game1 isn’t the test i forgot to delete that.
Not barebones but if you look at the draw method in Game_TestUnproject its pretty basic.

I didn’t directly use seperate matrices because i didn’t want to take forever and i wanted all the ui stuff in there so you can move it around for yourself. If you are skeptical of those classes you can look at the matrice output or switch them out.

it’s not like that…

unproject screenpoint you have nearworldpoint and farworldpoint and viewpoint(cameraposition) are on the same line
so for example, draw a line from near to farwordpoint and what you get on screen is only a dot(can’t even see it) at screenpoint

in your case it a line

Yes sorry my function was hosed too but.

The function in that example IS Monogames. I yanked it straight out of mg’s Viewport class.

If i simply move the camera backwards and m43 my z depth goes positive, or i spin the camera around. I start to see the arrow X, that is coming purely from the function all im doing really is drawing what comes out of it, as i move the camera in the world.

Just tap E then hold D let the camera spin around and you will see the arrow pop out.
I move the camera in the world the viewMatrix is just built from the camera.
That little line >> | << wont last long if you spin the camera around.

ack…
yes…you can see it if you change the view… but from the code you calculate it every drawcall, that calculate for newview(if you change campos, camup or camforward) hence you can only see the line…

it seem like you put arrow from nearworldpoint and point the head to (0,0,0) so when you cross z 0 cause the clip…you can check by turn camera to look at (0,0,0)

note:correct for ray

Vector3 worldRaysNormal = Vector3.Normalize(farWorldPoint - nearWorldPoint);
return new Ray(nearWorldPoint, worldRaysNormal);

Edit:
quick check

1 Like

You are using the near and far clipping planes for source.Z in Unproject. I think you need to use 0 and 1 instead. Otherwise this line doesn’t make sense

clipSpace.Z = (source.Z - gd.Viewport.MinDepth) / (gd.Viewport.MaxDepth - gd.Viewport.MinDepth); // >> Oo << 

MinDepth and MaxDepth of the viewport are 0 and 1.

1 Like

Oh shit …

Otherwise this line doesn’t make sense
clipSpace.Z = (source.Z - gd.Viewport.MinDepth) / (gd.Viewport.MaxDepth - gd.Viewport.

yep.

            worldRay = GetScreenVector2AsRayInto3dWorld(mouseState.Position.ToVector2(), viewSpaceCameraTransformer.Projection, viewSpaceCameraTransformer.View, wObjCamera.World, nearPlane, 1f, GraphicsDevice);
            wObjMouseRay.Scale(.01f, .01f, 150f);
            wObjMouseRay.Position = worldRay.Position;
            wObjMouseRay.Forward = worldRay.Direction;
            Helpers.DrawForwardQuad(GraphicsDevice, effect, viewSpaceCameraTransformer.GetWorldViewProjection(wObjMouseRay), textureOrient);

Just a sliver of a line barely visible behind the mouse.
Just gotta make sure its actually right.

            //______________________________________________________________________________________________________`

            worldRay = GetScreenVector2AsRayInto3dWorld(mouseState.Position.ToVector2(), viewSpaceCameraTransformer.Projection, viewSpaceCameraTransformer.View, wObjCamera.World, nearPlane, 1f, GraphicsDevice);
            wObjMouseRay.Scale(.01f, .01f, 150f);
            wObjMouseRay.Position = worldRay.Position;
            wObjMouseRay.Forward = worldRay.Direction;
            Helpers.DrawForwardQuad(GraphicsDevice, effect, viewSpaceCameraTransformer.GetWorldViewProjection(wObjMouseRay), textureOrient);

            // change the color or texture or something when the dot from ray to the ray point to object position are -1 or 1 or within some super small amount.
            //________________________________________________________________________________________________________

            wObjRandomthing.Position = new Vector3(0f, 0f, -10f);
            wObjRandomthing.Forward = Vector3.Forward;

            var n1 = Vector3.Normalize(wObjRandomthing.Position - worldRay.Position);
            var n2 = worldRay.Direction;
            var result = Vector3.Dot(n1, n2);
            Texture2D tmptexture;
            if (result > .999f)
                tmptexture = textureOrient;
            else
                tmptexture = texture;

            Helpers.DrawQuad(GraphicsDevice, effect, viewSpaceCameraTransformer.GetWorldViewProjection(wObjRandomthing), tmptexture);

I want to thank both of you guys i really thought this method was broken.

Though it kinda is easy to screw it up and it sure is confusing. I actually think i had it at 1 at some point then changed it.

public Vector3 Unproject(Vector3 source, Matrix projection, Matrix view, Matrix world, GraphicsDevice gd)

Z It’s definately Not the far plane lol its A >>>>>>>> 0 to a 1 …no more <<<<<<<<<.