Ray picking fails when world matrix is not identity

Android
1

I have a simple game where you can select objects. Selection is implemented with ray picking. World, view and projection matrices are same for objects drawing and screen touch point unprojection (obviously).

The case is: On start, world matrix is Matrix.Identity, ray picking works as expected. When I rotate the scene (with Matrix.CreateFromYawPitchRoll) or zoom, picking fails.

What can explain this behavior?

2

you need to better describe how your program works for us to understand

3

Ok.

Each object displayed is stored in class:

class GameObject {
    Matrix pos;
    Model model;
 }

When I click, following is invoked:

float rayIntersects(GameObject go, Ray r) {
    foreach (var mesh in go.model.Meshes) {
    var bs = mesh.BoundingSphere.Transform(go.pos * worldMatrix);
    var t = r.Intersects(bs);
    return t?.Value ?? float.NaN); }
}

When worldMatrix is identity, it’s ok, when it changes slightly (approx. 3-5 degrees), it still works, but I should point off the model to select it; in other cases it fails.

Also, I implemented algebraic solution from here, results are same.

4

Well the first thing that springs to mind is why are you using the world matrix?

If the position of the mesh (go.pos) is in world coordinates then it is redundant and in fact incorrect as the world matrix is a 4x4 matrix which includes translation.

If what you are trying to do is apply the mesh parts matrix then you need to …

   Matrix[] transforms = new Matrix[model.Bones.Count];
   Model.CopyAbsoluteBoneTransformsTo(transforms);
   foreach (ModelMesh mesh in model.Meshes)
   {
           Matrix World = transforms[mesh.ParentBone.Index];
   }
5

I use world matrix to store global transforms. All objects have fixed positions relative to center object; to rotate and zoom scene, I apply world matrix (since I need arcball-like rotations and it’s easy to implement it via Matrix.CreateFromYawPitchRoll).

The thing is, it worked as expected. Then I implemented unprojection (because of problems with game restart), and it suddenly crashed. Reverting to older code didn’t fix problem.

6

I am a little confused by that. Are you using the system they used in Elite where the camera is fixed at {0,0,0} and the entire universe moves around you?

If so then the code you are using still has the problem that each MeshPart in the Model has it’s own world transform which you are not applying.

These transforms are stored in the Model.Bones array as shown above.

7

Yes, universe is rotating around the camera.

I get BoundingSphere from mesh. Doesn’t it already positioned to take account of mesh transforms?

8

Nope.

Try multiplying the position of the bounding sphere by the world matrix in the bones array

9

Ok, what I’ve did so far:

  1. Slightly changed the code of rayIntersects:
float rayIntersects(GameObject go, Ray r){
    var trans = new Matrix[go.Model.Bones.Count];
    go.Model.CopyBoneTransformsTo(trans);
    // also tried go.Model.CopyAbsoluteBoneTransformsTo(trans);
    foreach (var mesh in go.Model.Meshes){
        var trs = trans[mesh.ParentBone.Index];
        var bs = mesh.BoundingSphere.Transform(trs);
        // following is same
    }
}
  1. Tried different combinations of go.pos (A), mxWorld (B) and trans[mesh.ParentBone.Index] (for both CopyAbsolute… and CopyTransforms…) © for trs in code above:
  • [A | B | A * B] * C - bs.Center is far away from real position
  • C * A - same object is selected independent on where I click
  • C * A * B - same object is selected independent of scene rotation and click position (occasionaly, other object is selected - the one I point to).
10

I solved the problem.

For better understanding, fragment of Unproject from gist I used:

Vector3 Unproject(Vector3 source){
    Matrix transform = matrixWorld * matrixView * matrixProjection;
}

And in rayIntersects :

float dist = ray.Intersects( mesh.BoundingSphere.Transform(gamePosition * matrixWorld) );

So, calculated position was result of trasnform of displayed position by world matrix, that introduced an error.

Solution: in rayIntersects I transform bounding sphere only by gamePosition.