I gave up on the Content Manager and just load stuff manually though it might only work on windows.
Code as follows:
FileStream fileStream = new FileStream(loadingName + ".png", FileMode.Open);
Texture2D texture = Texture2D.FromStream(Game1.GraphicsDeviceInstance, fileStream);
then later when you want to unload it you have to call:
A lot of my levels have similar assets to the levels before and after them so being able to only unload and load what was required did a lot for my loading times.