Tiled Testability Tedium

Hello! I’ve been using MonoGame on and off for a few years now, and started to work with the Monogame.Extended.Tiled package. I had some issues (Apparently unless you install the prerelease versions of everything, the pipeline tool doesn’t work!) but mostly figured them out.

But then I realized that to implement my collision and movement system, I was going to have to build an invisible 3D representation of my Tiled maps. Conveniently, we have this nice set of objects that load this stuff! Alright, time to write some tests! I’ll just construct a TiledMap and… oh.

Oh.

You can construct a TiledMap, entirely manually, but then who knows if it will match one loaded through the content system. No, that won’t do at all. Okay, so I guess we’ll just start up a ContentManager and… experience failure again. Because even stubbed out as deeply as you can go, eventually the ContentManager tries to load a GraphicsDevice and GraphicsAdapter, and those are impossible to substitute with a mock.

Okay, lets try something more oulandish: We’ll try replacing it all with Pose! And get a BadImageFormatException!

This is seriously doing my head in. I just want an accurately loaded map as input for a series of tests of code that will process TiledMap instances. Does anyone have any hints for me?

Yeah, unfortunately writing unit tests with MonoGame has never been the easiest thing to do.

This is mostly due to the design decisions made in the original Microsoft XNA which MonoGame is based upon. It pretty much makes any kind of mocking impossible because there’s lots of global static state that couples everything to the Game class.

That said, I’ve been slowly refactoring MonoGame.Extended to make it more testable. The Content Pipeline stuff is by far the biggest problem area. The plan is to provide a way to completely bypass the Content.Load method if you want. This will work for many things (XML, JSON, game data, etc) but you’d still need to do something about textures and sounds, etc.

Manually deserializing the XML will get you some of the way there.

using (var reader = new StreamReader(mapFilePath))
{
    var mapSerializer = new XmlSerializer(typeof(TiledMapContent));
    var map = (TiledMapContent)mapSerializer.Deserialize(reader);
}

If you want to look into it, take a look at how the content pipeline importers do it.

This… helps somewhat. I’m still left with a minor issue, namely that my code under test will use a TiledMap. I can try and map these I guess.

I’ve decided that I’ll just make a separate small game project, and just make a testing framework to run inside it. It isn’t a great idea, but at this point I think it’s the best I can do, if I want to be able to have quick-running tests of code that interacts with actual types.

For my unit testing, I just created wrappers for everything. Extracted an interface for the things I needed, and then passed those through. Since I have interfaces for the MonoGame stuff, I can mock it. For the content manager stuff, this did require that I go through a factory to handle creation. I should really find out if there’s a way to publicly expose your repository in TFS… but for now, here’s some code.

The same could probably be done with the MonoGame.Extended stuff and you can do it inside your own project. I know this isn’t your core problem, I just wanted to offer this because it doesn’t take terribly long to do this kinda thing and it really removes the roadblock for writing unit tests for your code :slight_smile:

    /// <summary>
    /// Represents functionality for managing content.
    /// </summary>
    public interface IContentManager
    {
        /// <summary>
        /// Load an asset from the content manager for the specified type.
        /// </summary>
        /// <typeparam name="T">The asset type to load as.</typeparam>
        /// <param name="assetName">The name of the asset to load.</param>
        /// <returns>The asset, loaded as the specified type.</returns>
        T Load<T>(string assetName);
    }
    /// <summary>
    /// An implementation of IContentManager that wraps a MonoGame ContentManager object.
    /// </summary>
    public class MonoGameContentManager : IContentManager
    {
        private ContentManager _wrappedObject = null;

        /// <summary>
        /// Create a new instance.
        /// </summary>
        /// <param name="objectToWrap">The MonoGame content manager to wrap.</param>
        public MonoGameContentManager(ContentManager objectToWrap)
        {
            _wrappedObject = objectToWrap ?? throw new ArgumentNullException();
        }

        /// <summary>
        /// Load an asset from the content manager for the specified type.
        /// </summary>
        /// <typeparam name="T">The asset type to load as.</typeparam>
        /// <param name="assetName">The name of the asset to load.</param>
        /// <returns>The asset, loaded as the specified type.</returns>
        public T Load<T>(string assetName)
        {
            return _wrappedObject.Load<T>(assetName);
        }
    }
    /// <summary>
    /// Represents an object that creates ITexture2D objects.
    /// </summary>
    public interface ITexture2DFactory
    {
        /// <summary>
        /// Create a new ITexture2D object.
        /// </summary>
        /// <param name="device">An object representing a graphics device.</param>
        /// <param name="width">The width of the texture.</param>
        /// <param name="height">The height of the texture.</param>
        /// <returns></returns>
        ITexture2D Create(IGrfxDevice device, int width, int height);

        /// <summary>
        /// Create a new ITexture2D object using a content manager.
        /// </summary>
        /// <param name="contentManager">The content manager that can load the texture.</param>
        /// <param name="assetName">The asset name the texture is stored as.</param>
        /// <returns>A newly created ITexture2D object</returns>
        ITexture2D Create(IContentManager contentManager, string assetName);
    }
    public class MonoGameTexture2DFactory : ITexture2DFactory
    {
        public ITexture2D Create(IGrfxDevice device, int width, int height)
        {
            Texture2D newTexture = new Texture2D(((MonoGameGraphicsDevice)device).WrappedObject, width, height);
            return new MonoGameTexture2D(newTexture);
        }

        public ITexture2D Create(IContentManager contentManager, string assetName)
        {
            Texture2D newTexture = contentManager.Load<Texture2D>(assetName);
            return new MonoGameTexture2D(newTexture);
        }
    }
    /// <summary>
    /// Represents an object that stores information about a texture.
    /// </summary>
    public interface ITexture2D : IGraphicsWrapper
    {
        /// <summary>
        /// Set the pixel data for this texture.
        /// </summary>
        /// <typeparam name="T">The type of data representing the type for each pixel.</typeparam>
        /// <param name="data">An array of data representing the pixels</param>
        void SetData<T>(T[] data) where T : struct;

        /// <summary>
        /// Set the pixel data for all pixels in this texture to the specified value.
        /// </summary>
        /// <typeparam name="T">The type of data representing the type for each pixel.</typeparam>
        /// <param name="data">A value that all pixels will be set to.</param>
        void SetData<T>(T data) where T : struct;

        /// <summary>
        /// The width, in pixels, of this texture.
        /// </summary>
        int Width { get; }

        /// <summary>
        /// The height, in pixels, of this texture.
        /// </summary>
        int Height { get; }
    }
    public class MonoGameTexture2D : WrapperBase<Texture2D>, ITexture2D
    {
        public MonoGameTexture2D(Texture2D objectToWrap)
            : base(objectToWrap)
        {
        }

        public int Height => this.WrappedObject.Height;
        public int Width => this.WrappedObject.Width;

        public void SetData<T>(T[] data) where T : struct
        {
            this.WrappedObject.SetData<T>(data);
        }

        public void SetData<T>(T data) where T : struct
        {
            T[] dataArray = new T[this.Width * this.Height];
            for (int i = 0; i < dataArray.Length; i++)
                dataArray[i] = data;

            this.WrappedObject.SetData<T>(dataArray);
        }
    }
1 Like

Right, and I think that in general, that is a good approach to things. However, that only helps if I modify the source for all of Monogame. I have a class I want to write to interact with instances of TiledMap, which I only have access to through the DLL, and the only way one of these is constructed is using types which are also in compiled code already.

If everything used interfaces and was more flexible about things that aren’t needed not existing, I wouldn’t have any issues at all. It is extremely important that the TiledMap be constructed by the same ContentManager as my game, because I don’t want to miss some nuance in an example I have contrived.

I am thankful you put this much effort into replying, though!

I may actually post the testing framework I’m going to try to build here, so that someone else in this situation can benefit from it. I could probably even package it. Running into minor issues getting it running still, though.

You don’t though. You just create wrappers and factories for the things out of that DLL that you want to use. In the example above, there’s a lot more functionality on Texture2D than I use, but since I only really use SetData, With, and Height, that’s all I expose.

The factories create the actual MonoGame types, so everything that’s needed for the constructors is supplied, it’s just abstracted away from the place where my actual game uses it. So in this case, TiledMap will be constructed by the same ContentManager as your game, you just kind of hide it away a little.

The implementation of IContentManager holds a reference to the real ContentManager object that you give it to wrap. When TiledMap needs to construct, it can be given the reference to the real ContentManager object inside the IContentManger implementation. You can then pass that TiledMap instance to an implementation of, say, ITiledMap if you wish, which in turn wraps the MonoGame.Extended TileMap functionality. Then anything in your own game can interact with ITileMap instead of MonoGame.Extended’s TiledMap.

I think you misunderstand. I 100% get the value to wrapping something, so I don’t need direct dependencies. However, here’s what I’m actually trying to do: I want to write code that interacts with a TiledMap instance. It is the input data of the code I’m writing, and there are complex ways that it could be constructed, and I specifically don’t want to mock it. If I was writing mathematical functions, I would not try to mock the integers I take as input, because then all I’d be testing is my mock. And while I might have some luck designing an object that acts similarly to an integer, I would not be able to design an object that acts like a TiledMap, especially when I specifically do not know how it acts in every scenario.

Therefore, to write a method that processes a TiledMap instance, I cannot approximate or wrap, or else my code may respond only to things that I assumed worked a certain way, but do not in a real instance.

The only things, in this scenario, which it would make sense to mock, are the parts to do with the GraphicsDevice, which has no reason to be involved in things like loading content files that don’t need to be displayed. These cannot be mocked, because code I do not control uses them to construct my TiledMap instances, for some reason.

Again, I have done a lot of wrapping for the sake of isolation in the past, but that does not apply to my current situation. I may even do a great deal of wrapping functionality on this project in the future. But for right now, I have to interact with real, concrete types which cannot be instantiated in a unit test.

Ah, I did misunderstand. My apologies :slight_smile:

Actually, I gave this some more thought.

I’m wondering if the misunderstanding might be that you think you need to test TiledMap at all. You don’t, it’s not your code. When you take in a 3rd party library like that, all you can do is assume it works (maybe functional test a little :D). What you do need to test (assuming you’re goal is to write testable code) is your interactions with that library. Did your code create the object correctly, did it make the right calls with the expected parameters? That kind of thing.

So in this case, an abstraction and a factory is exactly what you need. Isolating TiledMap from your code allows you to test that your code (and your code only) does the expected thing. This is a different scenario entirely than what you were saying about integers. I don’t need an abstraction for integers because they’re a simple type and I can compare them directly. As long as my unit test gets the expected value, nothing else really matters. As you say, TiledMap is more complex than that and likely doesn’t have an equivalence operator either.

Again, wrapping a TiledMap and allowing it’s creation via a factory in no way allows you test TiledMap. It does let you test all of your code that would normally interact with the TiledMap.

(Sorry, if I’m still not understanding you correctly, perhaps it might not be a bad idea to post some simplified code samples of how you’re using TiledMap and what parts you want to test?)

Basically, I’m making an isometric game with free 3D movement and 2D art for the maps. This essentially requires that I make a 3D world based on the map I read in.

The structure of the data in this object, of the tileset, and the maps themselves, is what I need to represent correctly. Again, I refuse to assume anything about this object structure. How properties work, along with other things, has even changed over time. There are absolutely incorrect assumptions that could be made, and I will not deal with that.

It isn’t functionality, it’s data. I’m not mocking it.

Anyway, I’ve made some progress when it comes to making a test framework (console based) - I originally wanted to just make a game instance in a unit test, but that wasn’t working, even when i moved the tests to my main project and just imported the MSTest2 packages. It seems that they run in a context that isn’t capable of handling the SDL dependency. Doing it as a console app fixed that issue pretty nicely. I’m able to run the game for one frame, then pass the content into my MEF2 container to provide it to any test class which requests it. I’ve only tested it on .NET Core 2.1 and .NET 4.8, and it definitely only handles that SDL dependency in the latter, but it does work!

Once I get it polished up to levels I approve of, I’ll probably release a Nuget package for anyone else in similar situations.

Tiled doesn’t come with any in-built collision data out-of-the-box, so I assume you’re placing something within the Tiled map editor which can be processed at runtime to generate the mesh data needed for your collision system? I’m also assuming this is done at runtime as you’re using an existing library to process and import your map.

I don’t have any understanding of the Monogame.Extended implementation detail, so @craftworkgames might be able to shed some light on that, but a standard flow through a Content.Pipeline plugin would be the following:

  • ContentImporter<Tmx>
  • ContentProcessor<Tmx, ProcessedTmx>
  • ContentWriter<ProcessedTmx>

Each of these can (and should) be unit tested completely separately.

I have the following coverage for my TiledMapProcessor class:

Tiled processor tests

The most complex test in these is the ShouldDecodeAndDecompressTileData method, but even that is fairly small:

[Theory]
[InlineData("csv", "", "\n1,2,4,\n3,1,2,\n2,4,1\n")]
[InlineData("base64", "", "AQAAAAIAAAAEAAAAAwAAAAEAAAACAAAAAgAAAAQAAAABAAAA")]
[InlineData("base64", "zlib", "eJxjZGBgYAJiFiBmBmJGKB8mBuIDAAGwABU=")]
[InlineData("base64", "gzip", "H4sIAAAAAAAAE2NkYGBgAmIWIGYGYkYoHyYG4gMALN/AqyQAAAA=")]
public void ShouldDecodeAndDecompressTileData(string encoding, string compression, string value)
{
    var layer = new TileLayer
    {
        Width = 3,
        Height = 3,
        Data = new Data
        {
            Value = value,
            Encoding = encoding,
            Compression = compression
        }
    };

    var result = new TiledMapProcessor().DecodeTileLayerData(layer);
    
    Assert.Equal(9, result.Count);
    Assert.Equal(1, result[0].Gid);
    Assert.Equal(2, result[1].Gid);
    Assert.Equal(4, result[2].Gid);
    Assert.Equal(3, result[3].Gid);
    Assert.Equal(1, result[4].Gid);
    Assert.Equal(2, result[5].Gid);
    Assert.Equal(2, result[6].Gid);
    Assert.Equal(4, result[7].Gid);
    Assert.Equal(1, result[8].Gid);
}

I don’t need to load an entire Tiled map from disk to test if my content processor can successfully decode and decompress the various different configurations that Tiled offers - I can isolate the appropriate data, represent it in code, and use that to unit test the DecodeTileLayerData method.

Likewise, my other tests each have different stripped-down versions of TiledMap and TileLayer objects with the minimum amount of properties required to test that single method:

[Fact]
public void ShouldNotProcessInvalidEncoding()
{
    Assert.Throws<NotImplementedException>(() =>
    {
        var layer = new TileLayer
        {
            Data = new Data
            {
                Encoding = "invalidEncodingType",
            }
        };
        new TiledMapProcessor().DecodeTileLayerData(layer);
    });
}

This method of representing data as code can be used in your collision system. Distil down only what is required from a map to generate the collision meshes.

For me personally, I represent collision in Tiled by painting red tiles on a special Collision layer. As such, my collision generator only requires I pass in a single TileLayer instance. Here is a unit test for that:

[Fact]
public void ShouldCreateSeparateColliders()
{
    var tileLayer = new TileLayer
    {
        Width = 3,
        Height = 3,
        Tiles = new TileLayer.Tile[3, 3]
    };

    var collisionTile = new TileLayer.Tile { Gid = 1 };
    
    /*
     * x 0 x
     * 0 x 0
     * x 0 x
     */
    tileLayer.Tiles[0, 0] = collisionTile;
    tileLayer.Tiles[0, 1] = null;
    tileLayer.Tiles[0, 2] = collisionTile;
    tileLayer.Tiles[1, 0] = null;
    tileLayer.Tiles[1, 1] = collisionTile;
    tileLayer.Tiles[1, 2] = null;
    tileLayer.Tiles[2, 0] = collisionTile;
    tileLayer.Tiles[2, 1] = null;
    tileLayer.Tiles[2, 2] = collisionTile;

    var physicsLayer = 1;
    var colliders = CreateColliders(tileLayer, physicsLayer).ToList();
    
    Assert.Equal(5, colliders.Count);

    foreach (var collider in colliders)
    {
        Assert.Equal(1, collider.PhysicsLayer);
        Assert.Equal(32, collider.Bounds.Width);
        Assert.Equal(32, collider.Bounds.Height);
        ...
    }
}

Then I have other tests to cover other scenarios, such as ensuring that if multiple tiles are next to each other, the system properly merges these in to a single collision box.

Finally, my implementation of the collision system has no knowledge of what a TiledMap even is - in fact the physics and rendering are separated in to completely different assemblies that cannot access each other.

My pathfinding is unit tested by passing in fabricated Collider objects which are substantially easier to create than Tiled objects.

There is still value in ensuring your systems fully work with actual Tmx files and content writers and the such, but those are integration tests, not unit tests. I still haven’t found a good way to do end-to-end integration tests in MonoGame due to the problems with instantiating a Game class instance in CI.