Background Reference Material

ShaggyTheHiker · October 14, 2020, 5:08pm

I’m finding that there is an ever expanding category of “things I don’t know”. I was going to add that this is about graphics, but I guess I could just leave that statement where it is, because it’s true for everything.

However, since this is MonoGame, the question I have is this: I’m being tripped up by my lack of understanding on how some of the principles used in MonoGame graphics works, so I’m looking for a good reference to explain what is going on. I’m not interested in things like 3D graphics, projections, and the like, because I learned long ago that just understanding the math isn’t going to let me write 3D objects. What I’m looking for is some documentation that explains in some detail, what is happening at various steps.

Many of the questions on here, come from the fact that people know to call something like SpriteBatch.Draw, but don’t know what is happening. Many of the answers are of the form, “try it this way”, which is quite appropriate for a community message board. After all, nobody is expecting to get a book in response, but does such a book exist? That’s what I’m mostly looking for: Not the solution to any immediate problem, but a resource for how the different elements are working. Perhaps it’s all just trial and error, and nobody has written such a book, but if there is something like a Petzold for DirectX graphics, that would be nice.

Beckett_Firestone · October 14, 2020, 6:11pm

I agree with you. I think it would be helpful if there was more behind-the scenes documentation.

willmotil · October 14, 2020, 6:44pm

Well specific questions in general are easier to answer then broad ones.

Take for example when asking about what happens when you draw a sprite with spritebatch.

Consider these 3 classes.

github.com

MonoGame/MonoGame/blob/develop/MonoGame.Framework/Graphics/SpriteBatch.cs#L428


      
          			}
          		}
          
                  /// <summary>
                  /// Submit a sprite for drawing in the current batch.
                  /// </summary>
                  /// <param name="texture">A texture.</param>
                  /// <param name="position">The drawing location on screen.</param>
                  /// <param name="sourceRectangle">An optional region on the texture which will be rendered. If null - draws full texture.</param>
                  /// <param name="color">A color mask.</param>
          		public void Draw (Texture2D texture, Vector2 position, Rectangle? sourceRectangle, Color color)
          		{
          			CheckValid(texture);
                      
          			var item = _batcher.CreateBatchItem();
          			item.Texture = texture;
                      
                      // set SortKey based on SpriteSortMode.
                      item.SortKey = _sortMode == SpriteSortMode.Texture ? texture.SortingKey : 0;
          
                      Vector2 size;

You will notice item.Set{ … }

github.com

MonoGame/MonoGame/blob/develop/MonoGame.Framework/Graphics/SpriteBatchItem.cs#L26


      
          		public VertexPositionColorTexture vertexBL;
          		public VertexPositionColorTexture vertexBR;
          		public SpriteBatchItem ()
          		{
          			vertexTL = new VertexPositionColorTexture();
                      vertexTR = new VertexPositionColorTexture();
                      vertexBL = new VertexPositionColorTexture();
                      vertexBR = new VertexPositionColorTexture();            
          		}
          		
          		public void Set ( float x, float y, float dx, float dy, float w, float h, float sin, float cos, Color color, Vector2 texCoordTL, Vector2 texCoordBR, float depth )
          		{
                      // TODO, Should we be just assigning the Depth Value to Z?
                      // According to http://blogs.msdn.com/b/shawnhar/archive/2011/01/12/spritebatch-billboards-in-a-3d-world.aspx
                      // We do.
          			vertexTL.Position.X = x+dx*cos-dy*sin;
                      vertexTL.Position.Y = y+dx*sin+dy*cos;
                      vertexTL.Position.Z = depth;
                      vertexTL.Color = color;
                      vertexTL.TextureCoordinate.X = texCoordTL.X;
                      vertexTL.TextureCoordinate.Y = texCoordTL.Y;

Looking back to that Original draw method you will notice FlushIfNeeded which relates to a batch of items.

github.com

MonoGame/MonoGame/blob/develop/MonoGame.Framework/Graphics/SpriteBatcher.cs#L242


      
                      _batchItemCount = 0;

          		}

          

                  /// <summary>

                  /// Sends the triangle list to the graphics device. Here is where the actual drawing starts.

                  /// </summary>

                  /// <param name="start">Start index of vertices to draw. Not used except to compute the count of vertices to draw.</param>

                  /// <param name="end">End index of vertices to draw. Not used except to compute the count of vertices to draw.</param>

                  /// <param name="effect">The custom effect to apply to the geometry</param>

                  /// <param name="texture">The texture to draw.</param>

                  private void FlushVertexArray(int start, int end, Effect effect, Texture texture)

                  {

                      if (start == end)

                          return;

          

                      var vertexCount = end - start;

          

                      // If the effect is not null, then apply each pass and render the geometry

                      if (effect != null)

                      {

                          var passes = effect.CurrentTechnique.Passes;

You can look around further into that class to see that it is dealing with batches of items and what SpriteBatch items are batched up ? … are they sprites ? … No they are quads, which are pairs of triangles.
So a pair of triangles makes a square and we call that a sprite we also call em a quad so same thing 3 names.

You will see a diagram in this class in ascii art and should now immediately recognize the meaning here now.

github.com

MonoGame/MonoGame/blob/develop/MonoGame.Framework/Graphics/SpriteBatcher.cs#L101


      
                  EnsureArrayCapacity(Math.Min(newSize, MaxBatchSize));

              }

              var item = _batchItemList[_batchItemCount++];

              return item;

          }

          

          /// <summary>

          /// Resize and recreate the missing indices for the index and vertex position color buffers.

          /// </summary>

          /// <param name="numBatchItems"></param>

          private unsafe void EnsureArrayCapacity(int numBatchItems)

          {

              int neededCapacity = 6 * numBatchItems;

              if (_index != null && neededCapacity <= _index.Length)

              {

                  // Short circuit out of here because we have enough capacity.

                  return;

              }

              short[] newIndex = new short[6 * numBatchItems];

              int start = 0;

              if (_index != null)

The above function has pointer math all over it, that is to speed it up as much as possible as it is at the very center of spritebatch.

Im sure you have seen images of faces and other things made up of tons of triangles and now you can see that it all is just triangles.

Sometimes its easier to see a custom implementation all in one class file but spritebatch draws quads pairs of triangles and just wraps it up in nice methods that make things easy like to do rotations on the vertices and automatically sets up the matrices and has a build in shader stored in byte code it uses and such.

It’s essentially a even more fancy version of the below simple quad draw example.

But keep in mind ultimately your gpu is just this grid like you saw in school in math class. It is ranging from -1 to +1 x and y with some special sauce that round off to discrete points that relate to the screen resolution and store pixels and data for them. Of course it can do all kinds of things that amp this grid up like rasterize triangles with edge points that match up to points on a image.

For example if you look on this grid and take the value -.5 and think of your screen from left to right with a screen resolution of 1000 width trying to fit in this little space.

Then that puts it -.5 about a quarter ways from the left of your screen or 250 pixels.
So if you take -1 to 1 that’s a width of just 2 so do this… add 1 so the range is 0 to 2 then divide by 2 you get a range now from 0 to 1 then multiply it by 1000 the screen width and if you do that to the -.5
(-.5 + 1 = .5 / 2 = .25 * 1000) … you get that 250 pixel x position.
That is what the orthographic projection matrix is doing for spritebatch and so when you look at the batcher you see those little numbers and you are inputing big numbers relating to the screen.
You position quads with a world matrix and move everything all of the world with a view matrix and walaaa your little grid is anything you want it to be. Including having a layer depth a z coordinate.
Of course all of that is dumbed down to were you just use spritebatch.Draw and its familiar methods related to the screen resolution which we of course know when we change it can be anything.

So sprite batch makes its super easy to draw squares out of triangles and pretty them up in a really simple way so you don’t have to do a lot of extra stuff yourself like in the example post above.

ShaggyTheHiker · October 14, 2020, 9:31pm

That’s a pretty nice description. I know just enough of computer graphics to recognize that everything is a triangle, and see how this fits in. I suppose one of the better advantages to something like MonoGame is the open source nature, so the code behind many of the objects is available to examine.

Still, that’s one piece of several. For example, with some profiling in a different thread, I recognized that there is something odd happening in my test project that is not happening in my main project, because the performance in the test project is so bad that it the main project wouldn’t even be functioning if it was equally as poor. To figure out what is going on, I need to dig into more of what is happening along the way. The one thing I was able to rule out, was pretty much everything to do with the SpriteBatch. That leaves plenty more, such as the GraphicsDevice and RenderTargets, that I would want to understand better. Specifically, I’m seeing behavior that suggests that memory (probably video memory) is being cached under certain circumstances…and that seems really improbable.

Nobody should be asked to write a comprehensive discussion on this topic in posts, or even necessarily in a blog, because if they did, they’d have written a book, yet gotten nothing for it.

MrValentine · October 15, 2020, 7:48am

There are so many books, my bookshelf is almost brimming with MG and XNA related content…

ShaggyTheHiker · October 15, 2020, 2:23pm

Great, which one do you like best for low level discussion of the subject? I’m never going to be much of an artist, but I’m pretty good with code of all sorts, and hardware. I started into programming by learning ASM and CPU architecture (though that was a LONG time ago, and the world has changed), so I’d rather have a low level grounding that a high level discussion of transformations (of which I have a fair, and rather useless, background).

MrValentine · October 16, 2020, 2:16am

Are you asking me? or?

If me, then, when you say low level, do you mean Source or MG?

If Source, go to the github page and download the sourcecode… won’t get lower than that unless you compile C#? I don’t really understand your question…

For MonoGame coding as in how to throw features on the wall and see what sticks, I just started reading Learning C# by Programming Games 2nd Edition…

Which is the most recent book on MonoGame on the market…

However, almost any XNA book will do you the trick too.

Do let me know if that answered your question?

willmotil · October 16, 2020, 2:34am

Ah my point was typically when you dig in you just read things or look at code till you get to a part that doesn’t make sense to you.

Then you ask a question you may have many many questions for many things you can’t understand and then you tend to ask a question like so …

im not sure how to phrase this question but how does spritebatch work.

That’s a pretty general and vauge question its really hard to answer without writting a big article right.

So my point is one thing at a time, the more thought you put into how specific your question is and clear it is… then the more clear and descriptive the answers become.
The answer you get can only be as good as the question you ask.

To say there is no limit on the number of clear questions you can ask and the probability is that the clearer the question you ask. Then the quicker, clearer, better is the answer that you’ll get.

Start by browsing thru some tutorials or simple code examples there are plenty on the forums or even referencing the monogame documentation links at the top of the page which also have tutorials and information, or the source code itself then find clear questions to specific answers you need.

MrValentine · October 16, 2020, 4:14am

I really hate the reply system on this discourse thing…

ShaggyTheHiker · October 16, 2020, 7:30pm

@MrValentine: Yes, I was asking you for a recommendation on a book you found particularly good.

At this point, I have a working program (with the exception of one particular control that I asked a specific question about in a different thread), and am pretty content with the switch from XNA 4.0 to MG 3.8, but it brought to my attention the simple fact that there is a whole lot that I no longer understand about how things work at the chip or memory management level. Because I have a working program, and am content with that, I’m not looking for a tutorial, I’m looking for a greater understanding of what is happening. If you recognized the Petzold reference in my original post, you’d realize that I’m looking to get to the roots of things. Petzold wrote the one of the best books on Windows programming…for Win95, or so. It was a massive doorstop of a program, but thorough.

When I was first looking at graphics programming, there weren’t GPUs, and the 256 color palette was still a thing, so palette cycling was still a technique. Things have changed a LOT since that time. I can get by with a superficial understanding, but would like a better foundation. So, if you have a reference that you found particularly good or thorough, I’d be very much interested in hearing about it.

@willmotil: I agree with you, and really appreciated the detailed response you gave. I have a couple threads on here this week (which is as long as I’ve been on this forum) about specific topics. What I’m looking for here is that broad foundation. Digging into the source is certainly something I will be doing, but I’d like to get an even deeper foundation.

Personally, I feel that whenever we encounter a problem that we have not encountered before (not just in programming), we form a mental model of how the system works. Our brains will try to filter any new input such that it fits within the model, even to the point of disregarding a few points that don’t quite fit. Only once reality clearly violates this mental model in a significant way, do we shift the mental model in a different direction. Over time, as we learn more, the model becomes better and better, though it may never be totally correct.

Where I’m currently at with MG / XNA is just enough of a mental model that my program works acceptably, but also enough to know that my model is seriously deficient. Filling in those gaps to build a better foundational understanding of what is happening is my goal. Therefore, there is no specific question in this thread aside from a recommendation for good reading material. Specific questions are in specific threads.

MrValentine · October 17, 2020, 12:49am

I don’t think there are any books for what you are asking for, have you looked at the API pages?

Look up any XNA books on amazon, there are kindle editions too. Click the previews to see the TOCs…

ShaggyTheHiker · October 18, 2020, 5:25pm

Searching Amazon for Monogame is made decidedly more interesting by the various meanings of the word.

MrValentine · October 18, 2020, 9:06pm

Ahahahahahaah

For a recent book, Learning C# by Programming games Second Edition.
Windows 8 and Windows Phone 8 Games Development…
Anything with XNA in the title…
C# Game Programming for serious game creation

Most of them have Kindle variants…

ShaggyTheHiker · October 19, 2020, 2:19pm

I have one on order. I considered getting more, but I really can’t read more than one thing at a time, so I felt I could wait. Thanks for the recommendations, though. That’ll give me the next book.