To use a depth buffer with SpriteBatch together with rendering of the Tiled maps you need to respect SpriteBatch's orthographic projection which depth is from 0 (near plane) to 1 (far plane):
If you so wish, you could alternatively use your own shader program (known as Effect in MonoGame/XNA) to override the orthographic projection to use whatever you want for near and far planes. In either case, as long as the orthographic projections are the same, you can use same range of depth values for both.
Note that to use the depth buffer so that draw commands no longer use the painter’s algorithm, you have to change the pipeline state which is done with SpriteBatch.Begin parameter:
Which should match GraphicsDevice.DepthStencilState when rendering the 3D model of the Tiled map. Not using a depth buffer is appropriate in cases where you want to use the painter’s algorithm which is more intuitive and simpler for most people. To use the painter’s algorithm with the Tiled maps you would simply begin/end SpriteBatch as many times as needed for Tiled layers.
“To use the painter’s algorithm with the Tiled maps you would simply begin/end SpriteBatch as many times as needed for Tiled layers.”
I think this pretty much answers my question.
My only confusion is… I’ve got trees on my tilemap that the player sprite should be able to walk behind (be rendered behind) and stand in front of (rendered on top). How do I know which order to render layers in?
It’s up to you. RPG Maker 2003/2005/XP have historically solved this problem by using a different layer for “ground” (below player, no alpha-blend), “mask” (below player, alpha-blend), and “fringe” (above player, alpha-blend). If it makes sense for your game you could have a ground layer, sky layer, etc.
EDIT: Sorry, it’s not clear how that answers your question exactly. The tiles that need to be rendered on top of a player like roof tiles would be on the sky layer while the tiles that should be rendered below the player would be on the ground layer.
Whether the tree should be rendered behind or in front of the player, depends on the player’s position. The trees are currently just a bunch of tiles on different layers (in case of overlap). How do I know when to render the player?
If all the trees were sprites, I could render them at the right depth.
Would rendering as sprite objects be the better option, or is this unnecessary?
I’m trying to consider performance.
Whether the tree should be rendered behind or in front of the player, depends on the player’s position.
With painter’s algorithm it depends on which draw call is submitted first. SpriteBatch submits draw calls on SpriteBatch.End (and sometimes sooner if you switch textures). Think of it like this: you have a sandwich of sprites between Tiled layers.
With depth buffer enabled (disabled by default), you can have previous triangles rendered stay on top based on depth values but with some limitations.