Testing with MonoGame on Linux: threading error

Hello,

I am currently trying to add some visual tests to my game. I’ve managed to write working tests when running independently, but I get an exception when I run all the tests together.
Here’s a minimal example that exhibits the problem :

[TestFixture]
[Apartment(ApartmentState.STA)]
public class SimplestGame
{
    TimeSpan timeLimit = new TimeSpan(0, 0, 0, 0, 100);

    [Test]
    public void RunSimplestGame()
    {
        Console.WriteLine("Starting...");
        var game = new DummyGame(timeLimit);
        Console.WriteLine("Running...");
        game.Run();
        Console.WriteLine("Exiting...");
        game.Dispose();
    }
}

[TestFixture]
[Apartment(ApartmentState.STA)]
public class SimplestGame2
{
    TimeSpan timeLimit = new TimeSpan(0, 0, 0, 0, 100);

    [Test]
    public void RunSimplestGame()
    {
        Console.WriteLine("Starting...");
        var game = new DummyGame(timeLimit);
        Console.WriteLine("Running...");
        game.Run();
        Console.WriteLine("Exiting...");
        game.Dispose();
    }
}

public class DummyGame : Game
{
    private readonly GraphicsDeviceManager graphics;
    private readonly TimeSpan limit;
    private SpriteBatch spriteBatch;

    public DummyGame(TimeSpan limit)
    {
        graphics = new GraphicsDeviceManager(this);
        this.limit = limit;
    }

    protected override void Initialize()
    {
        base.Initialize();
    }

    protected override void LoadContent()
    {
        Content.RootDirectory = "Content";
        spriteBatch = new SpriteBatch(GraphicsDevice);
    }

    protected override void UnloadContent()
    {
    }

    protected override void Update(GameTime gameTime)
    {
        if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed
            || Keyboard.GetState().IsKeyDown(Keys.Escape)
            || Elapsed(gameTime))
        {
            Exit();
        }
        base.Update(gameTime);
    }

    protected override void Draw(GameTime gameTime)
    {
        GraphicsDevice.Clear(Color.CornflowerBlue);
        base.Draw(gameTime);
    }

    private bool Elapsed(GameTime gameTime)
    {
        if (gameTime.TotalGameTime > limit)
            return true;

        return false;
    }
}

Both tests (they are identical) run fine if I run them alone, but the second one crashes if I run them together in nunit, with the following error :

One or more errors occurred.
Stack trace:
at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <9bbab8f8a2a246e98480e70b0839fd67>:0
at System.Threading.Tasks.Task1[TResult].GetResultCore (System.Boolean waitCompletionNotification) [0x0002b] in <9bbab8f8a2a246e98480e70b0839fd67>:0 at System.Threading.Tasks.Task1[TResult].get_Result () [0x0000f] in <9bbab8f8a2a246e98480e70b0839fd67>:0
at MonoDevelop.UnitTesting.NUnit.NUnitAssemblyTestSuite.RunUnitTest (MonoDevelop.UnitTesting.UnitTest test, System.String suiteName, System.String pathName, System.String testName, MonoDevelop.UnitTesting.TestContext testContext) [0x00245] in <5a50c7f8fd15492b8f57b877d888bb04>:0
at MonoDevelop.UnitTesting.NUnit.NUnitTestSuite.OnRun (MonoDevelop.UnitTesting.TestContext testContext) [0x00001] in <5a50c7f8fd15492b8f57b877d888bb04>:0
at MonoDevelop.UnitTesting.UnitTest.Run (MonoDevelop.UnitTesting.TestContext testContext) [0x00020] in <14600d39865a4deabab34102d35bdc57>:0

Running the tests in the console gives some more details:

Errors, Failures and Warnings

  1. Error : /home/…/bin/Debug/MyGame.Tests.dll
    System.Runtime.Remoting.RemotingException : Tcp transport error.
    ----> System.Runtime.Remoting.RemotingException : Connection closed
    –RemotingException

Server stack trace:
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.ReceiveMessageStatus (System.IO.Stream networkStream, System.Byte[] buffer) [0x00017] in <981d2e773a85450680fa518175a3273b>:0
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg, System.Runtime.Remoting.Channels.ITransportHeaders requestHeaders, System.IO.Stream requestStream, System.Runtime.Remoting.Channels.ITransportHeaders& responseHeaders, System.IO.Stream& responseStream) [0x0005e] in <981d2e773a85450680fa518175a3273b>:0
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg) [0x00066] in <981d2e773a85450680fa518175a3273b>:0

Exception rethrown at [0]:
at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_remoting_wrapper(intptr,intptr)
at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent.Run(NUnit.Engine.ITestEventListener,NUnit.Engine.TestFilter)
at NUnit.Engine.Runners.ProcessRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00025] in :0
–RemotingException
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.StreamRead (System.IO.Stream networkStream, System.Byte[] buffer, System.Int32 count) [0x00011] in <981d2e773a85450680fa518175a3273b>:0
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.ReceiveMessageStatus (System.IO.Stream networkStream, System.Byte[] buffer) [0x00000] in <981d2e773a85450680fa518175a3273b>:0

If I comment out the game.dispose() instruction in the RunSimplestGame method, I can run them together without error, so I suspect that something is shared between the two tests and disposed before the second one is finished.
Am I missing something? Any idea of what needs to be done to ensure that both tests are effectively independent?

Thanks!

What version of MonoGame does the bug occur on:
MonoGame 3.7.0.1644
What operating system are you using:
Ubuntu 18.04
What MonoGame platform are you using:
DesktopGL

Does someone know how to run multiple visual monogame tests in nunit similar to the minimal example in the previous post?

Any help would be greatly appreciated!

Thx