1) Open the Monogame Pipeline Tool!
2) Under Project, select Content!
3) Go to Edit -> Add -> Existing Item!
If you added all your content allready in the folder Content in your Monogame project these should now be visible, as this is the official way to add ressources!
This is not necessary, just browse to your directory, but for the sake of quickness, add them beforehand and you are set!
4) Now, click the Build icon, its the gear icon on the right!
You should have no errors whatsoever and your png files should be part of the list of things that have been built, if not, rebuild, or clean and then build again!
I dont know if this is still needed, but for the sake of completion and for others that stumble upon this, here you go!