Not able to get correct World mouse coordinates with Unproject

1

Hi everyone,

This is my first attempt at writing a game with monogame. I’m using spritebatch to draw a bunch of sprites on a XY plane and I’m passing a basic effect with my camera matrices where:

world = Matrix.CreateScale(1, -1, 1);
view = Matrix.CreateLookAt(new Vector3(0, 0, 1.5f), new Vector3(0, 0, 0), Vector3.UnitY);
projection = Matrix.CreatePerspectiveFieldOfView(MathHelper.ToRadians(90), (float)_windowWidth / (float)_windowHeight, 0.1f, 100f);

I’m using Unproject to get the World space coordinates of my cursor to add and drag the drawn sprites on the plane, and this doesn’t give me the coordinates that I want. I’m using a workaround which I’m sure is not a good solution, by multiplying the unprojected mouseposition by the camera position Z coordinate * 20. Otherwise I can barely move the points and they are added near Vector.Zero, which is understandable since the values are very small without the multiplication.

Vector3 previousClientToWorldMousePosition = Game.GraphicsDevice.Viewport.Unproject(new Vector3(previousMouseState.X,
                previousMouseState.Y, 0f),
                ProjectionMatrix,
                ViewMatrix,
                WorldMatrix);

Vector3 curentClientToWorldMousePosition = Game.GraphicsDevice.Viewport.Unproject(new Vector3(currentMouseState.X,
            currentMouseState.Y, 0f),
            ProjectionMatrix,
            ViewMatrix,
            WorldMatrix);

deltaXmouseWorldPosition = previousClientToWorldMousePosition.X - curentClientToWorldMousePosition.X;
deltaYmouseWorldPosition = previousClientToWorldMousePosition.Y - curentClientToWorldMousePosition.Y;

//This is the workaround
viewMatrixZoomRelatedTranslation = -ViewMatrix.Translation.Z * 20; 
                                    
if (currentMouseState.LeftButton == ButtonState.Pressed && intersectPointIndex != null)
	{
		var pointIndex = intersectPointIndex.Value;                
        path.MovePoint(pointIndex, new Vector3(path[pointIndex].X - deltaXmouseWorldPosition * viewMatrixZoomRelatedTranslation, path[pointIndex].Y - deltaYmouseWorldPosition * viewMatrixZoomRelatedTranslation, path[pointIndex].Z));
	}            

if (currentMouseState.RightButton == ButtonState.Pressed && previousMouseState.RightButton == ButtonState.Released && intersectPointIndex == null)
	{
		path.AddSegment(new Vector3(curentClientToWorldMousePosition.X * viewMatrixZoomRelatedTranslation, curentClientToWorldMousePosition.Y * viewMatrixZoomRelatedTranslation, 0f));
	}

I’m assuming I should add some kind of scale or transformation Matrix for the matrices passed to the Unproject function, but I have no idea how it should be done. Any help will be appreciated.

Also the reason I’m using basiceffect lookat camera it to later apply an orbit camera and rotate around drawn sprites if anyone is curious :slight_smile:

2

The first parameter to Viewport.Unproject is a Vector3 for a reason. X and Y alone are not enough to unproject to an exact point. You also need the depth.

You set Z to 0, which will put your unprojected point on the camera’s near clipping plane. 1 would have put it on the far clipping plane.

So at this point, without knowing the right depth yet, the result is really a line. It starts at the camera position, goes through this near point, through the far point, and through every point in between.
All these points are overlapping from the camera’s point of view, only separated by depth, that’s why they are all valid solutions.

Which point on this line should you pick now? You need to bring in more information to lock down the depth. Fortunately in your case that’s easy to provide.
You are picking sprites. The sprites are all on the XY plane at Z=0. You just need to find the intersection point btw your pick-line and the XY plane.

Vector3 unproject = viewport.Unproject(...); // any z btw 0 and 1 is fine here
Vector3 dir = Vector3.Normalize(unproject - camPos);
float distToPlane = -camPos.Z / dir.Z;
Vector3 intersection = camPos + dir * distToPlane;
1 Like
3

Thanks for the reply. Ok, so first Im unprojecting the mouse position with the source vector Z set to a value between 0 and 1. Then calculating the Vector3 with depth like you described:

Vector3 direction = Vector3.Normalize(currentClientToWorldMousePosition - ViewMatrix.Translation);
float distanceToPlane = ViewMatrix.Translation.Z * direction.Z;
Vector3 mouseXyPlaneIntersection = ViewMatrix.Translation + direction * distanceToPlane;

But this still doesn’t give me the correct coordinates of the mouse cursor on the XY plane.
Also correct me if I’m wrong, if my camera position is set as Vector3(0,0,1.5f) and I’m looking at Vector.Zero I’m getting the same distance to the XY plane regardless of my mouse cursor position.

image
image865×490 11.1 KB

On the screenshot cursor is on the red dot, the coordinates used for drawing it in spritebatch were (-1, 0, 0). The path position is set to Vector.Zero and it’s correctly set in the center of the screen. As you can see (hopefully) on the screenshot. The world coordinates calculated with unproject are incorrect for both. Y is fine, because it’s 0 in this case, but X is not.
Let me know what am I missing or maybe I haven’t understood your explanation.

4

There are two problems (at least):
First you turn the division from here

into a multiplication

And second you can’t use ViewMatrix.Translation to get the camera’s position. The view matrix is the inverse of the camera matrix. It has to be the same vector you pass to Matrix.CreateLookAt: (0, 0, 1.5f)

The distance you are calculating here is the distance between the camera and the intersection point. Maybe I shouldn’t have called it distToPlane, but distToIntersection.

1 Like
5

Ok, now it works! I don’t know why I haven’t noticed the multiplication… Thanks a lot for you help and explanation :slight_smile:

6

You’re welcome. Are your actually planning to change the topdown view at some point? If the only reason for using a world, view and projection matrix is to get some scaling, then you could have just used a scaling matrix instead.

7

Yes, I want to create and orbit camera around the center of the drawn sprites with an option to zoom in and out. My idea was to use Matrix.CreateLookAt and update the camera position. Something like:

Vector3 cameraPosition = cameraDirection * desiredOrbitalDistance;

But I’m just getting started with this.

8

Ok, so I managed to create a rotating camera. I’m updating the View matrix using the new camera position and looking at Vector.Zero. Now and I’m not exactly sure how to calculate the distToIntersection to get the depth for the unprojected mouse coordinates when I’m facing the XY plane with a different angle. I’d greatly appreciate any help.

9

It should already work for different camera angles? Do you get an intersection point that’s not at z=0?

10

You’re right, it works nicely for different camera angles. I used the inverted ViewMatrix.Translation vector instead of the updated camera position value. For the initial view it worked fine of course, but after changing the camera position it didn’t make much sense. Thanks a lot for your help :slight_smile:

11

Wait, I think I missed a minus, it should be:

float distToPlane = -camPos.Z / dir.Z;

Now I’m wondering why it works for you. Did you do anything to compensate for that? Isn’t distToPlane always negative?

1 Like
12

And now thanks a lot for that! I was able to move the points without the negative distance, as the movement relied only on the delta X and Y of the mouse cursor world position. However adding new points with different camera angles was messed up. Now everything works as it should!

13

I edited the code in my original post, in case somebody else copies it.

1 Like
14

Hi, reviving an old thread. How can I calculate the depth of my unprojected point if it’s no longer on an XY plane, but has been moved along the Z axis?

15

You can change this line into a more generic version that takes the Z position of the plane into account:

float distToPlane = (plane.Z - camPos.Z) / dir.Z;