Hi 
I would like to integrate 2D sprite in my 3D environnement.
The idea is not to “mix”, but really integrate, like if the sprite is a model in the scene.
I managed to display a sprite + a 3D scene, but the sprite is a kind of background and huge compared to my 3D model. The game is a Dungeon Master like (in 3D) but I want the monsters in 2D. So the sprites need to by “in” the dungeon, not behind or on top.
Here is, kindly, the idea:
Thanks for your help!
Create a 3d quad with texture on it as the Sprite and use either Matrix.CreateBillboard or Matrix.CreateConstrainedBillboard to make it always face camera.
For effects and things that always face the camera (even when looking down on them) use the version without constraints.
For monsters, objects, and other things that are “standing” on the ground use the constrain version to lock their rotation to only be around y axis.
Thanks for your help!
Any sample /link for the quad with a texture? I’m not sure how to do that.
@Rockford One simple way to do it is to create a flat quad in blender / 3ds max or the modeling software of your choice and import it as a model. If you want you can use the plane model from GeonBit (I’m not 100% it will work with the billboard matrix, you might need to rotate it):
Or if you need more control you can create the vertices yourself and use draw primitives. Drawing code:
GraphicsDevice.DrawUserIndexedPrimitives
(
PrimitiveType.TriangleList,
Vertices, 0, 4,
Indexes, 0, 2);
And to build the vertices you can take a look at this function:
But make sure the vertex type you use fit your effects.
IMO since this is new to you, you should follow these steps:
Hope this helps
That’s great! I’ll give a try soon and let you know.
That is a quick and dirty way of doing it, but, depending on your ultimate purpose, that may be a bad idea, especially if you’re doing it for many sprites or are going for precision. Creating a billboard matrix is processor intensive (it uses a couple square roots) and results in a mere approximation to cancel out the effects of the view matrix. A much more efficient approach would be to find the centre point in the view port (via the world and view matrices), and then explode the corners of a quad out from there, preferably via a vertex shader. That way, you won’t run into any rounding error, etc, and it will always appear perfectly aligned and without any distortion or jaggies. Personally, I never resort to CreateBillboard.
Ya i agree.
There are some caveats say your billboards are trees that sit on terrain and in your game you fly a plane that you can roll.
In that case the quads world.Up has to maintain some / or exact orientation equivalency to the terrains up.
(really in that case your going to probably want more then one quad for a tree).
The camera forward or back (depending on how you wind the quad) gets used to align the quads forward to face the camera.
You can find the left or right from the cross of the up forward, which you then normalize.
Your quad should probably be build so its u,v direction match the screen so.
0-----2–> x+
|…/.|
|…/…|
1…3
y+
z depth 0
So basically thats all fairly cheap a couple gets and a cross product + the translate being reset maybe, to populate the world matrix. You can probably instance them too billboards are good candidates for instancing.
If you were referring to using a vertex shader when you described the airplane roll caveat, a quick and efficient way to account for this would be to pass the objective “up” vector (0, 1, 0) through the view matrix, which will result in a unit vector pointing skyward, and you can then cross it with “backward” (0, 0, 1) to obtain the horizontal direction relative to the view. With these vectors, in conjunction with the centre point positional data as described above, you can easily produce a quad in 2D space that is, by its intrinsic nature, parallel to the screen. This can all be done in the vertex shader, and therefore have negligible impact on performance (unlike CreateBillboard).
Ya sorry i meant build the world matrix yourself. Just basically assign the up from the terrain to a initial trees world matrix like on construction, it really never has to change.
Like so.
For all the drawn tree objects when you construct or initialize them
trees.Up = terrain.Up;
trees.Translate = new Vector3( someWorldPosition );
Each time your view matrix changes update all the trees.
trees.Forward = view.Forward;
trees.Right = Vector3.Cross(trees.Forward, trees.Up);
Since the up and view forward is normalized already the cross products result will also be normalized.
So you don’t need to even normalize or use any square roots or to even store anything but the tree objects world matrix. (really you don’t even need that if none of your trees need to slant over) When you mulitply that by the view * projection matrix on the cpu or vertex shader to get a WorldViewProjection matrix, it should line … right up. Basically O(n) time.
You might need to account for gimble lock when the terrain up is equal to the view forward or its inverse view.Backwards. (if you were to dive straight down) but that’s not a big deal, its more of a handled event then a part of the normal operations. You would have to handle that anyways.
This way works for moving objects as well as non moving static objects like trees. For actual static objects like trees though, you can improve this even more, build a indexed primitive array put all the trees in the proper world positions, instance them, do the crossing on the shader, pass the terrain up and the view or camera forward build the matrix there, and make it like O(1) time a single call.
Oh ok; we’re talking about two different solutions (mine was similar to what you described at the end). Yeah, that sounds good. I like that a lot better than what CreateBillboard does. The only (minor) qualm I have with that approach is that you might get a tiny amount if rounding error because the world matrix is cancelling out the view orientation, which might possibly cause some alignment imperfections, and you wouldn’t want to do this operation on, say, hundreds of individual particles. But for scenes like the OP demoed, that should be just fine.
(An example of said alignment issues that I’ve encountered would be Z values differing just enough to throw off the depth stencil on multiple passes, resulting in lighting/texture banding.)
I’m back at coding this week. Thanks for all this advices.
I think I’ll start using Matrix.CreateBillboard because I’m beginner in 3D, and your alternative ways are quite obscure for me without a sample.
I already know of to create a 3D scene and display models (like a cube or so) and moving the camera or the models, rotating, etc.
I’ll now learn to create a quad on the fly, texture it with a 2D texture, like a sprite (with transparence), then apply the Billboard to make it face cam. I’ll then try to reread your discussion to try something more optimized.
Almost! But cannot use the billboard…
I managed to create the quad, to understand TriangleList and TriangleStrip, to display it, to texture it, to make it deal with alpha!
I see the sprite, I can move the camera, rotate the camera…
But if I apply a billboard, it disappears.
My quad :
`
quadVertices = new VertexPositionTexture[4];
quadVertices[0].Position = new Vector3(-1, -1, 0); // Lower left
quadVertices[1].Position = new Vector3(-1, 1, 0); // Upper left
quadVertices[2].Position = new Vector3(1, -1, 0); // Lower right
quadVertices[3].Position = new Vector3(1, 1, 0); // Upper right
quadVertices[0].TextureCoordinate = new Vector2(0, 1);
quadVertices[1].TextureCoordinate = new Vector2(0, 0);
quadVertices[2].TextureCoordinate = new Vector2(1, 1);
quadVertices[3].TextureCoordinate = new Vector2(1, 0);
`
How I display the quad:
mySpriteEffect.View = pView;
mySpriteEffect.Projection = pProjection;
mySpriteEffect.World = pWorld;
//mySpriteEffect.World = Matrix.CreateBillboard(Vector3.Zero, cameraPosition, pWorld.Up, pWorld.Forward);
mySpriteEffect.Texture = pTexture;
// No need for this, but it will work with advanced effect with multiple passes
foreach (var pass in mySpriteEffect.CurrentTechnique.Passes)
{
pass.Apply();
graphics.GraphicsDevice.DrawUserPrimitives(
PrimitiveType.TriangleStrip,
pQuad,
0,
2);
}
My camera is at 0,0,10.
My world is : Matrix.Identity
My view is:
view = Matrix.CreateLookAt(
cameraPosition,
targetPosition,
Vector3.Up
);
view = view * Matrix.CreateRotationY(camRotation);
Target position is 0,0,0.
My projection is:
projection = Matrix.CreatePerspectiveFieldOfView(
MathHelper.ToRadians(45f),
16f/9f,
1f,
100f
);
Any idea?
If I compare Matrix.Identity with the result of CreateBillboard, all seems inverted…
You may want to set the CullMode to None while testing the create billboard function.
Your texture coordinates are also inverted unless your doing that on purpose.
quadVertices[0].TextureCoordinate = new Vector2(0, 0);
quadVertices[1].TextureCoordinate = new Vector2(0, 1);
quadVertices[2].TextureCoordinate = new Vector2(1, 0);
quadVertices[3].TextureCoordinate = new Vector2(1, 1);
One thing to note right away is you never do this.
The view is meant to be created by CreateLookAt from a world object and passed to the shader.
If you try to rotate it or translate it you basically will destroy its functionallity and get garbage output from everything you draw.
You can rotate your camera’s target forward normal if you want and then send that to create look at.
So this means you need to be aware of how CreateLookAt works if you want to directly pass your cameras objects matrix’s Forward you got from CreateWorld to CreateLookAt directly.
The parameter of CreateLookAt(…, Target, …) is a little obscure.
What this parameter expects is a actual target position to point the camera at within your world, but it doesn’t offset that position from the position parameter you also pass to the function. This means if you want to pass some vector that represents the idea of a direction not a position You must account for that distinction yourself. Say you are sending it a rotation to look at, then it can get a little tricky.
For example.
var m = Matrix.CreateRotationY(someKeyboardControlledAngle)
var forwardNormal = m.Forward;
The above value is actually a normalized directional forward vector now.
Its a vector that’s elements are composed of values between -1 to 1
you cant use that as a target position in create look at directly,
it will always look at the same place.
Also the target passed to create look at gets normalized.
So in this particular case of using a normal direction to look at…
you want to denormalize it * 10.
so this is how you pass it in.
view = Matrix.CreateLookAt(
cameraPosition,
cameraPosition + (forwardNormal * 10f),
Vector3.Up
);
Ironically CreateWorld takes the idea of a direction not a position for it’s forward parameter.
Keep the idea of a view and projection matrix seperate as if these are stand alone items.
The projection matrix is used on all your world objects so is the view. the view however is created from just one world object you have already completely set up.
When that world object (the one you picked to be the camera), position or rotation changes,
you should rebuild the view matrix afterwards from its values.
To make that clear the view is a special transformation that must be applied to all world objects but it is not a world object itself nor is it’s built translation / (position), the same as the world object it is created from. As such you need to just remember that the view is created from a world object that you designate to be the camera in your world.
Thanks a lot!
I have implemented an FPS camera style according to your advice, and I calculate the direction of the camera to deal with the movements:
Example:
if (Keyboard.GetState().IsKeyDown(Keys.Z)) // forward
{
Matrix forwardMovement = Matrix.CreateRotationY(camDirection);
Vector3 v = new Vector3(0, 0, -speed);
cameraPosition.Z += v.Z;
cameraPosition.X += v.X;
}
And my view is now:
Matrix rotationMatrix = Matrix.CreateRotationY(camDirection);
Vector3 forwardNormal = rotationMatrix.Forward;
view = Matrix.CreateLookAt(
cameraPosition,
cameraPosition + (forwardNormal * 10f),
Vector3.Up
);
I have fixed vertices and texture coordinates:
quadVertices[0].Position = new Vector3(-1, -1, 0); // Lower left
quadVertices[1].Position = new Vector3(-1, 1, 0); // Upper left
quadVertices[2].Position = new Vector3(1, -1, 0); // Lower right
quadVertices[3].Position = new Vector3(1, 1, 0); // Upper right
quadVertices[0].TextureCoordinate = new Vector2(0, 1);
quadVertices[1].TextureCoordinate = new Vector2(0, 0);
quadVertices[2].TextureCoordinate = new Vector2(1, 1);
quadVertices[3].TextureCoordinate = new Vector2(1, 0);
Then I removed the culling:
RasterizerState rasterizerState = new RasterizerState();
rasterizerState.CullMode = CullMode.None;
graphics.GraphicsDevice.RasterizerState = rasterizerState;
And my quad appears (with the billboard applied as world) !
And I can note that it is reversed (My picture is mirrored). It explains why it was not visible with the culling. Why?
If the word is Matrix.Identity, the quad is on the right side, but with the Billboard matrix, wrong side…
Here is my code for the billboard:
mySpriteEffect.World = Matrix.CreateBillboard(Vector3.Zero, cameraPosition, Vector3.Up, null);
I tried to tink the 4th parameter (cameraForwardVector) but it seems ignored.
This is the last step I believe. I’m wondering what I’m doing wrong.
Edit: I tried to use CreateBillboard on a 3D model, and same result, I then see the back of the model…
CreateBillboard interally negates the forward vector. Which means the entire world matrix for the billboarded world object is spun around towards the camera. Also this method really expects all the fields to be inputed, except in simple cases. It also manually normalizes the forward you pass it. So it takes a direction.
One way to handle this is to just use your inverted texture coordinates or put a bool on your create quad that lets you flip the uv to whichever way you want it.
// raw usage.
var billboardPos = new Vector3(0, 0, -5f);
var cameraPos = Vector3.Zero;
var up = Vector3.Up;
var forward = billboardPos - cameraPos;
// this forward one passed ends up being negated so it will be -forward or backward.
// this means the quad will be fliped around towards you.
// that can affect the culling direction of the quad.
Matrix objsBillboardMatrix = Matrix.CreateBillboard(billboardPos, cameraPos, up, forward);
// To demonstrate in a gif.
// How i would use it practically to ensure that in all rotations it will work.
// If that is indeed desired here i task the billboards up to be the cameras up.
var billboardPos = wObjRandomthing.Position + new Vector3(1f,0f,0f);
var cameraPos = wObjCamera.Position;
var up = wObjCamera.World.Up;
var forward = billboardPos - cameraPos;
Matrix objsBillboardMatrix = Matrix.CreateBillboard(billboardPos, cameraPos, up, forward);
// set this objects world matrix to be a billboard.
wObjRandomthing.world = objsBillboardMatrix;
Helpers.DrawQuad(GraphicsDevice, effect, viewSpaceCameraTransformer.GetWorldViewProjection(wObjRandomthing), textureOrient);
In the below image you can see that while both arrows in the world are undergoing severe rotations and position changes in all axis in relation to the camera, the one arrow texture stays billboarded to the camera.
It’s not billboarded to some terrain’s up, ( and camera to object positions, forward) (which in your case you would probably prefer to do) you would use the terrains up, to make it keep that upwards orientation, in the case of the camera rolling.
Thanks!
I’m still confused, I still do not understand why it’s not “facing” the camera :(.
I don’t understand when you write “Which means the entire world matrix for the billboarded world object is spun around towards the camera”. Does it mean it’s normal that the result show the back of the quad? Does not sound like the behavior expected by the function. Is my quad wrong? But I got the same result with a 3D model…
I tried your code for calculating the forward and the result is the same (forward = 0,0,-10), I see the back of my quad. If I pass the inverted value of forward (forward = 0,0,10)… no change. It seems the forward parameter do not change the orientation.
The only way I found is:
mySpriteEffect.World = Matrix.CreateBillboard(Vector3.Zero, cameraPosition, Vector3.Up, forward);
mySpriteEffect.World *= Matrix.CreateRotationY(MathHelper.Pi);
EDIT : Ignore this last workaround it’s working only if the quad is at 0,0,0 of course…
But I feel like I workaround something I do wrong on the first place.
Ok sorry i was thinking you could invert the forward to fix that well.
Does it mean it’s normal that the result show the back of the quad?
Normally id say no in fact im not sure why the function is negating the forward direction honestly i would do this myself with create world. Either way it’s sort of trivial for a billboard since you want the quads rotation locked to the cameras backwards then again the CreateLookAt swaps the forward as well.
I wouldn’t worry too much about the texture being off though just flip the u or v or both.
Just keep a seperate quad for this specifically.
I build a quad just for billboarding and then adjust the uv’s (u,v) as needed.
If you want to flip a texure horizontally or vertically…
just flip the whole columns x or y from 0 to 1 or 1 to 0.
for example. To flip this around so its upside down.
change this (u,v)
quadVertices[0].TextureCoordinate = new Vector2(0, 1);
quadVertices[1].TextureCoordinate = new Vector2(0, 0);
quadVertices[2].TextureCoordinate = new Vector2(1, 1);
quadVertices[3].TextureCoordinate = new Vector2(1, 0);
to this
quadVertices[0].TextureCoordinate = new Vector2(0, 0);
quadVertices[1].TextureCoordinate = new Vector2(0, 1);
quadVertices[2].TextureCoordinate = new Vector2(1, 0);
quadVertices[3].TextureCoordinate = new Vector2(1, 1);
if you had instead wanted to flip it so its left was its right.
change this (u,v)
quadVertices[0].TextureCoordinate = new Vector2(0, 1);
quadVertices[1].TextureCoordinate = new Vector2(0, 0);
quadVertices[2].TextureCoordinate = new Vector2(1, 1);
quadVertices[3].TextureCoordinate = new Vector2(1, 0);
to this.
quadVertices[0].TextureCoordinate = new Vector2(1, 1);
quadVertices[1].TextureCoordinate = new Vector2(1, 0);
quadVertices[2].TextureCoordinate = new Vector2(0, 1);
quadVertices[3].TextureCoordinate = new Vector2(0, 0);
The Matrix.CreateBillboard is like CreateWorld in that it gives you a world matrix but like CreateLookAt.
When the camera rotates or moves around in your world, you need to update these billboards.
It just takes a bit of practice.
Thank you
Inverting the texture means I need to keep the culling off. And I guess it’s not performance friendly right?
Perhaps I should do the math by myself, calculate the right angle and rotate the quad accordingly.
Inverting the texture means I need to keep the culling off. And I guess it’s not performance friendly right?
You have plenty of ways to deal with culling.
You can switch the culling direction order on the graphics device before you draw.
(not performance freindly to switch state, but there is nothing to cull its a billboard)
You can change the actual vertice order passed i.e. the order of the vertices you make the quad from.
(zero penaltys)
You can use a vertex position normal texture structure and assign the normal direction yourself per vertice to point in either direction forward or backwards. (only penalty is its 12 bytes bigger)
The way culling works is like so you have a triangle like this.
A
|_\B
C
If you make the triangle vertice position A B C for vertice [0] [1] [2]
That is a clockwise triangle it has a clockwise winding order.
If you send in the vertices in the opposite order C B A for vertice [0] [[1] [2]
Then you get a counter clockwise triangle.
The culling you set on the graphics device has the options for both clockwise and counterclockwise as well as none.
The way these windings work is that they relate to the direction of a surface normal extending out from a triangle.
A cross product creates a normal from the found from 3 vertices, the order that you send them in to be crossed creates either a forward or backward surface normal to the perpendicular of the plane of the that the triangle lies upon, (the plane that all three vertices form in space).
Perhaps I should do the math by myself, calculate the right angle and rotate the quad accordingly.
Once you fully understand how the matrix parameters are interacting with the quad.
You will start to more clearly see how to set a matrix yourself without using createbillboard at all.
Ouaw… OK. Lot of concepts to digest when you deal with 3D, I need to improve myself. I’m impressed by your knowledge.
My need is to display creatures in a dungeon, and be sure that the quad is facing the camera when the player encounter one of them.
I try to have a code very simple.
I tested Matrix.CreateLookAt considering the quad is the camera, and the camera is the target, but cannot get something from that, the rotation was inverted…
Here is my try:
I just rotate the quad according to the cam rotation, then I move the quad to its position.
mySpriteEffect.World = Matrix.Identity;
mySpriteEffect.World *= Matrix.CreateFromAxisAngle(Vector3.UnitY, camDirection);
mySpriteEffect.World *= Matrix.CreateTranslation(quadPosition);
I did a test and it seems to work. What do you think?