Parallel.ForEach and garbage collection

High guys,

I just added an entity component system to my game and as usual I tried to spread the cpu loading of the component updates across the available cores using Parallel.ForEach.

This screwed up the game horribly.

Ever 4-5 seconds I had a spike where the cpu just tanked. I tracked it down to garbage collection.

Watching the monitor you can see the base memory usage going up and up and the garbage collector firing a complete refresh every 4-5 seconds.

I looked at my code and couldn’t find any major sources of garbage, so as a test I removed all usage of Parallel.ForEach.

The problem instantly went away.

I put it back in and tested in debug x86 and Release:x86

The pattern is exactly the same, though more pronounced in debug than release.

I am going to have to rewrite my ECS to use software threading, but I’m pretty pissed off. Parallel.ForEach is such a powerful methodology and I have used it a lot in C# outside monogame without ever having these issues.

Might be a good idea to look at what’s going on.

check this to see if it’s your problem: https://stackoverflow.com/questions/31747992/garbage-collection-and-parallel-foreach-issue-after-vs2015-upgrade

Without more information about what monogame functions you are calling inside the parallel.for will be very hard to track down if it’s a MG problem, which I doubt unless you’re using functions which require UI thread access.

Also, game programming is very different to “generic programming”. You can use Parallel.For in any database app and never realize the same problem, but being videogames a realtime application, they instantly show up.

The way you initialize the objects within Parallel.ForEach can lead to this problem too.
Do you use the single “func” prototype or do you aggregate the results in a second action ? Do you also use loops in the func ?

ForEach<TSource, TLocal>(IEnumerable<TSource>, Func<TLocal>, Func<TSource, ParallelLoopState, TLocal, TLocal>, Action<TLocal>)

Firstly, this has nothing to do with MonoGame. Parallel.ForEach is part of the .NET Framework. The helper classes for multithreading in .NET can help to get things going quickly in a multithreaded manner, but there are also several ways you can make things worse by not realising what is going on under the hood.

Show us some of the code you use and we may be able to help with your problem.

You might want to try the following patch. Actually I think you are the person with the perfect case scenario to do so and provide the MG team with useful feedback.


http://teamcity.monogame.net/viewLog.html?buildId=53654&buildTypeId=MonoGame_PackagingWindows&tab=artifacts

Also, ContentManager is not thread safe. If all threads use the same instance you have to make a custom one by overriding a couple of methods and use thread safe datastructures (from System.Collection.Concurrent) and some locks here and there. Otherwise it will fail every now and then with unexpected results.

That hot fix does look like it will be related to what I am seeing.

//Parallel.ForEach(ActiveGameObjects.Keys, (s) => foreach (int s in ActiveGameObjects.Keys) { ActiveGameObjects[s].Update(stage,dt); }//);

Inside the various game object updates there really wasn’t any Monogame specific code, just general C#

However inside the game object update I have another parallel loop which doesn’t cause a problem.

        public void Update(int stage, float dt)
        {
            Parallel.ForEach(Components.Keys, (s) =>
            {
                if (Components[s].UpdateStage == stage)
                {
                    Components[s].Update(dt);
                }
            });
        }

It is entirely possible the first parallel code block changes the thread timing enough to generate a thread lock in the garbage collector.

Sorry for the delay replying, I had to jump on a plane to fix some bugs in a game that went gold last week.