MonoGame.Forms - Create your Editor Environment!

@Jens_Eckervogt (and anyone else) - I have successfully made a Winforms control that allows for a single dll to be used interchangeably with MG dx or gl versions. Requires only a single API to be used with it and handles like standard Winforms controls (Drop a control, make it unique by modifying an externally set event). The only issue with it is it loses a bit of performance due to how I get the stuff rendered so best used for a very low amount of controls (The more controls needed the slower it all gets per present).

(Note about yesterday): I was handling like contexts which required alot of management and I realized that way way more work than necessary so I literally fixed all my issues at the same time and gained a bit of performance back by simplifying what i was doing.

@SandboxBlizz - Unless you end up redoing your dx handler the way I did (Which would downgrade alot of performance yours gets) ud need to compiler condition against DX and GL separately and have the GL end use my method of handling stuff if you want working GL support. Your version definitely includes some 1-ups over myne so Im not attempting to shadow ur work out or anything yours would obviously be more optimal as is for complex editors -> Especially since you include all the special background loop stuff for simulating the Update(Game Logic).

[UPDATE]
Github: https://github.com/SpiceyWolf/MonoGame.Framework.Winforms
Nuget: Install-Package MonoGame.Framework.Winforms -Version 1.0.0

2 Likes

Hey @SpiceyWolf

I took a small look at your repo and already did some small performance tests.

I noticed that you made use of GraphicsDevice.GetBackBufferData() which was originally implemented to have a quick way of taking a screenshot or snapshot of the current BackBuffer (if I remembering correctly) and should be a bit slow if called continuously.

However for me and some quick first test it doesn’t turned out that badly. I managed to have 9 GL controls (from you) and 9 DX controls (from MonoGame.Forms) and they worked all without lags and below 10% CPU usage (Intel Core i5-7600K). Further it’s possible to adjust the intervall, which raises the performance.

Before you posted your solution of MonoGame GL controls, I played around with the abbility of SDL_Surface returning the pixels of the screen as a pointer and I marshaled it to a byte array. So I had kinda the same approach and i’m thinking it is a way we can and should go (though i’m not an expert at this and still thinking of having a robust SwapChainRenderTarget.GL would be the best thing).

That’s why I plan to catch up on your idea with the BackBufferData and creating a SwapChain (alike) with it. Then it could possibly a simple thing to implement it to the MonoGame.Forms library without much hassle as a consistent integration.

I would then let the community and MonoGame.Forms users do their own integration and performance tests and see how it turnes out on different machines and editor needs.

As I said earlier in this thread: I’m always open for your ideas and suggestions as well as your sample projects showing feature ideas and so on to make the MonoGame.Forms library more awesome. So I will take my time and start experimenting with it further.

I start working on it on monday and will reply back with my experience.

Till then thank you very much for your effort and have a nice day and weekend everyone! :slight_smile:

Im glad you could get some ideas from it… As far as it goes I couldnt see any other way of doing it the SwapChain method though since I believe even SDL directly limits your window generation and either way MonoGame limits you to one graphics device on a GL build (At least it seems to since I couldnt find the generation code in source and in my test projects it crapped out trying to make as econd window) and so the only stable way i could get more than 1 control to load and function properly was simply ripping the graphic contents out manually and plastering them where i wanted xD If it could be implimented to create a GL based SwapChian in MonoGame directly i feel that would eliminate all the problems of just doing it the classic way and gaining back the inevitably lost performance… BUT as for now, this will at least get people working in GL and I dont see any code in my own source that shouldnt function properly in Linux and Mac (Ive used the bitmap handling code in Linux before and never had a problem)

What exactly doesn’t work with the canonical shared contexts for each handle?

Abusing a single GL context for multiple windows instead of shared contexts mangles the driver’s multiple GPU support.

Monogame doesnt expose anything like the SwapChain for GL (Most likely for non direct compatibility) and it doesnt allow you to make more than 1 graphics device for GL based projects which prevents doing more than 1 control in winforms… This of course assuming you did something like TK and had a platform detection and custom method per platform to embed 1 context yourself - though since MonoGame ALSO seems to disallow graphics device creation outside of a game window at the same time it meens you are trying to embed the window itself rather than just a context handle. For the meens of making GL based controls work unless an official access can be grabbed for proper support directly within MonoGame ull have to use my method or some pinvoke hacks… Either way if you are choosing to use MonoGame within a winforms control in the first place you shouldnt be the type of person concerned with “multiple GPU support” or high end handlers outside of maybe a couple controls designed for making content editors for a game.

Not only that if winforms + high end should be a huge concern -> MonoGame.Forms will do you just fine youll have to settle for being limited to windows/DX support only for high performance.

So, I finally had time to take a deeper look at the OpenGL support and I want to share my experience and findings with this post -> related to the MonoGame.Forms feature set.

The difference from the GitHub example from @SpiceyWolf and the MonoGame.Forms library is, that MonoGame.Forms doesn’t use the Game class as well as it does not make use of the GraphicsDeviceManager from the MonoGame.Framework, because originally I had the vision of communicating with the MonoGame.Framwork from out of custom UserControl based classes, powered by Service classes to define own update routines and extra functionallity making the integration in a WindowsForms environment more smooth and easier to use by the end user, because we don’t need an extra game project to hook up.

The caveat is that we need to initialize and create MonoGame specific stuff manually by hand. So let my start explaining from the very beginning:

Creating the GraphicsDevice

When you simply try to create a GraphicsDevice in a DesktopGL project you will instantly receive errors. Right from the start it’s quite obvious that we need to modify the source of the MonoGame.Framework to make that work.

To create and initialize the GraphicsDevice successfully, it’s necessary to construct the SDLGamePlatform, which for itself creates the SDLGameWindow by using the default BackBuffer Width and Height, which is okay. The only thing we are adding here is the SDL_WindowFlag SDL_WINDOW_OPENGL to make sure to internally create an OpenGL Context. Additionally and right after this, we initializing platform specific stuff by extracting the function PlatformInitialize from the GraphicsDeviceManager.SDL class.

Now we can successfully create a GraphicsDevice and pointing a reference to it inside the SDLGameWindow for later: ((SdlGameWindow)SdlGameWindow.Instance).GetGraphicsDevice = GraphicsDevice;

Creating the (pseudo) SwapChainRenderTarget.GL

To render stuff from the BackBuffer to the GraphicsDeviceControl (Windows.Forms.Control), we need to get the BackBuffer Data from the GraphicsDevice and create a Bitmap from that to draw it in the OnPaint event.

We are using this technique from the GitHub repo of @SpiceyWolf.

GraphicsDevice.GetBackBufferData() is using Gl.ReadPixels [ref], which can be slow when calling this in short intervalls. It also increases the RAM usage by around ~30 to ~80 MB.

You will also notice the decrease of performance the bigger the custom control is we are rendering to. Having a maximized window on a FullHD resolution with 2 rendering surfaces and the shortest intervall timer (1ms) raised my CPU usage to ~30%. Raising the intervall to 50ms results in CPU usage ~12% and still having a smooth editing experience (tested with the MultipleControls-Scenario of MonoGame.Forms and scrolling the HexMaps).

Having a ControlSize of 728x399 and intervall of 1ms leads to ~60fps and ~10% CPU usage.

Tested with Intel Core i5-7600K (non-overclocked).

I also found a solution to make the GL.ReadPixels calls significantly faster by using Pixel Buffer Objects (PBO), but for that we need a BufferTarget which is currently not implemented in MonoGame which is GL_PIXEL_UNPACK_BUFFER. I tried implementing this by myself but received an invalid enum exception from OpenGL. So someone with more knowledge is needed here. This task should be very simple.

Input Support (Keyboard/GamePad/Mouse)

Because we are creating the SDLGameWindow with the SDL_WINDOW_HIDDEN flag (it’s invisible and never shown to the user), we are not receiving Keyboard, GamePad or Mouse input anymore.

To resolve this we need to bring back the SDLEventQueue by calling SDLRunLoop manually after each draw of the BackBuffer. This atleast brings the GamePad input back.

Keyboard and Mouse input are still not working because the hidden SDLWindow can’t receive SDLEvents for Mouse and Keyboard.
We resolve this by invoking these events manually by using the Sdl.PushEvent function.

Well, the truth is a bit more complicated, because we first need to catch the Keyboard and Mouse events from WindowsForms and converting them to a format which SDL “understands”.

Here is an example of a Keyboard input event:

public Sdl.Event GetKeyEvent(SDLK.Key key, SDLK.ModifierKeys modifierKeys, bool down)
{
    Sdl.Event evt = new Sdl.Event();
    evt.Key.Keysym.Scancode = 0;
    evt.Key.Keysym.Sym = (int)key;
    evt.Key.Keysym.Mod = (int)modifierKeys;
    if (down)
    {
        evt.Key.State = (byte)SDLB.ButtonKeyState.Pressed;
        evt.Type = Sdl.EventType.KeyDown;
    }
    else
    {
        evt.Key.State = (byte)SDLB.ButtonKeyState.NotPressed;
        evt.Type = Sdl.EventType.KeyUp;
    }

    return evt;
}

protected override void OnKeyUp(KeyEventArgs e)
{
    if (!LockKeyboardInput)
    {
        try
        {
            if (!designMode)
            {
                Sdl.Event evt = GetKeyEvent((SDLK.Key)Enum.Parse(typeof(SDLK.Key), e.KeyCode.ToString()), (SDLK.ModifierKeys)e.Modifiers, false);
                Sdl.PushEvent(out evt);
            }
        }
        catch { }
    }
    base.OnKeyUp(e);
}

We do the same for MouseButtons and MouseMotion and start receiving Keyboard and Mouse events in the SDLEventQueue, which makes it also possible to use Keyboard.GetState() and Mouse.GetState() from the MonoGame.Framework (GamePad.GetState() also works of course).

Window Resizing

At some point I added window resizing support to MonoGame.Forms were it’s possible to resize the window, which contains a custom user control and everything got updated accordingly, when a resize event occured (the drawn contents as well as the Camera2D position and custom RenderTargets).

Critical here is the call to Sdl.Window.SetSize() which updates the hidden SDLWindow in the background for us.

RenderTargets

Currently RenderTargets are not functional, because it’s not possible to set or dispose them. The reason for this is the Blocking of the UI Thread. When avoiding those calls we would receive GL Errors. If someone has a clue I will be happy to hear a solution.

Conclusion

Making this changes and placing additions to the SDL backend made OpenGL support in MonoGame.Forms happen without changing in the core how the library works.

Having an intervall timer led to an acceptable performance - even for UpdateWindows in realtime. It’s also possible to make GL.ReadPixels faster in the future by using PBOs.

DrawWindows with AutomaticInvalidation turned off only consuming ~1% CPU usage, which suites them perfect as preview controls for textures and all kind of stuff were a lot of draw calls to update the controls are not necessary.

Now that we support more than 1 platform, I need to restructure the project. I thought of this:

  • MonoGame.Forms.Core
  • MonoGame.Forms.DX
  • MonoGame.Forms.GL
  • MonoGame.Forms.Tests.DX
  • MonoGame.Forms.Tests.GL

So before making this publicity available I still need to take some more time, effort and work. But I think I can make this available in the next couple of weeks. We will see. (this is not a promise ;))

So let me hear what you think and most importantly: think about the above mentioned problems and try to help making the library better!

Thank you all for your attention and suggestions / ideas so far! It’s really appreciated.

Have a nice one!

1 Like

2 things I want to point out -

1: Unless you need some keyboard/mouse functionality im not aware of in SDL that has benefits over the winforms stuff u can heavily simplify the “Translation” code by simply using the normal winforms handlers for keyboard/mouse and just translating mouse coordinates alone to the space in your scene based on camera or scale etc…

2: I cant remember off the top of my head but i HAVE encountered a good few features between MonoGame DX and GL that exist but dont work in GL and hearing how your change in implimentation can work you MIGHT be able to cheat and make it a single dll compatible with both if you have an “Initializer” check to set a background variable like in the base class of the controls themselves or something and it be like IsGL or something -> Then when its initializing try to invoke one of the functions and determine if you are running in GL based on that (probably with the use of TryCatch) then have it use 1 set of stuff based on if GL and another if DX.

(Additional): You might try to find out what Tom Spilman had in mind for the GL stuff before unblocking as you are most likely handling the relevant stuff he wants to uncripple the GL side of MonoGame and then work towards getting a PR to fix his problems and be a hero in the MG community for saving GL for us all xD

Getting the keyboard and mouse events from WindowsForms is already possible just by subscribing to the corresponding events. This is also used internally to check if the mouse cursor is inside a (resized) control, but it’s also possible to subscribe to those event from custom UserControls.

The benefit of converting WindowsForms input to SDL input is, that this makes it possible to use the Keyboard.GetState() and Mouse.GetState() functionallity from the MonoGame.Framework. They are not working for themselfs.

Another benefit is that you have a way faster click detection of the mouse buttons than with the WindowsForms mouse events, which can be very useful in editors as I experienced myself.

Wow you’re magician!!! I don’t know how did you magic that? my god here is whitch in MonoGame’s Community!!!

Nice job Now I have tested. It is really nice trick. I have switch to Gtk Sharp 3.

But Gtk Sharp 3 uses GLArea + MonoGame = MonoGame.Framework.Gtk3 ( I am here working in progress… )

And For lovely Mac Users: MonoGame.Framework.Xammac or MonoGame.Framework.AppKit

For Android MonoGame.Framework.Android - if we use userinterface of Android ( Xamarin.Android.dll )
For iOS MonoGame.Framework.iOS.

Et voilà - MonoGame.Forms running on Linux -> Ubuntu 18.04 LTS (Bionic Beaver)





Performace slightly decreased, coz running in a virtual machine.

I’m currently cleaning up the repo and working on that the documentation (Wiki) also becomes multi platform compatible :wink:

I also want to more automate the compiling process by automatically create the NuGet packages for both platforms and such.

But i’m very optimistic now to release the next major update at some point in the next week.

So lean back and eat popcorn (sugar free) :heart:

2 Likes

Wait you are using Gtk Sharp 3. Wow! What is your source now? I know Gtk Sharp 3 using for Ubuntu

Wait I see WinForms for Ubuntu it looks hurting

Yeah, because: Why doing that extra work? :wink:

It’s just the regular Forms project from Visual Studio compiled using Mono. And in the end the library is called “MonoGame.Forms”, right? :smiley: (no offense, just joking around)

Soooooooooooooooo I am happy because it works but it looks not finish.

I have tested with Gtk Sharp 3.

MonoGame “embeds” in Gtk Sharp 3 ^^.

Yeah I know but Timer has not for Gtk Sharp 3 Just I will fix later…

RenderArea is class with GLArea of Gtk Sharp. I am really happy because Linux and Mac wait longer to MonoGame embedding in Gtk Sharp. Now apple and pengiun are excited because MonoGame meets to Gtk Sharp 3

Downlaod ( work in progress )

// EDIT:

I tried DrawingArea.

I think better Bin or Widget I will try to make Advanced RenderArea.

What do you use monogame version?
Because GraphicsDevice.GetBackBufferData(pixelData);

‘GraphicsDevice’ does not contain a definition for ‘GetBackBufferData’ and no extension method ‘GetBackBufferData’ accepting a first argument of type ‘GraphicsDevice’ could be found (are you missing a using directive or an assembly reference?) (MonoGame.Framework.Gtk3)

Yes I am using MonoGame Framework 3.7

PS: It is impossible Google Search hides me. :frowning:

You use just MonoGame Framework 3.6 latest version ( from Github ) Please use WindowGL or LinuxGL or MacGL. Thanks!

But monogame 3.6 does not have this functionality

GraphicsDevice.GetBackBufferData(pixelData);

Yes because new version of MonoGame 3.7 please ask Cra0zy or other have upload to MonoGame’s source ( github or other source of development ( I think it is inofficial )

and I have throw error:

System.ArgumentException: Parameter is not valid.
   at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
   at Microsoft.Xna.Framework.Gtk3.Gtk3Backend.MasterWindow.EndRender() in C:\Users\SSB\Downloads\MonoGame.Framework.Winforms-master\MonoGame.Framework.Winforms-master\MonoGame.Framework.Gtk3\Gtk3Backend.cs:line 127
   at Microsoft.Xna.Framework.Gtk3.RenderWidget.OnDrawn(Context cr) in C:\Users\SSB\Downloads\MonoGame.Framework.Winforms-master\MonoGame.Framework.Winforms-master\MonoGame.Framework.Gtk3\RenderWidget.cs:line 90
   at Gtk.Widget.Drawn_cb(IntPtr inst, IntPtr cr)
   at GLib.ExceptionManager.RaiseUnhandledException(Exception e, Boolean is_terminal)
   at Gtk.Widget.Drawn_cb(IntPtr inst, IntPtr cr)
   at Gtk.Application.gtk_main()
   at Gtk.Application.gtk_main()
   at Game1.Program.Main() in C:\Users\SSB\Downloads\MonoGame.Framework.Winforms-master\MonoGame.Framework.Winforms-master\ExampleGame\Source\Program.cs:line 28

GtkBackend like UniversalBackend

RenderWidget

using Gtk;
using Gdk;
using Cairo;
using GLib;

namespace Microsoft.Xna.Framework.Gtk3
{
    public class RenderWidget : DrawingArea
    {
        private Color _clearColor;
        private Graphics.Viewport _view;
        public event EventHandler Render;

        public RenderWidget()
        {
            _clearColor = new Color();
            _view = new Graphics.Viewport(0, 0, Allocation.Width, Allocation.Height);
        }

        public Color BackColor
        {
            get
            {
                return _clearColor;
            }
            set
            {
                _clearColor = new Color(value.R, value.G, value.B);                   
            }
        }

        protected override bool OnConfigureEvent(EventConfigure evnt)
        {
            _clearColor = BackColor;
            _view = new Graphics.Viewport(0, 0, Allocation.Width, Allocation.Height);

            Render += new EventHandler(onRender);
            SizeAllocated += (sender, e) => { _view = new Graphics.Viewport(0, 0, Allocation.Width, Allocation.Height); };
            return base.OnConfigureEvent(evnt);
        }

        protected virtual void onRender(object o, EventArgs args)
        {
        }

        protected override void OnDestroyed()
        {
            Dispose();
            base.OnDestroyed();
        }

        protected override bool OnDrawn(Context cr)
        {
            PointD p1, p2, p3, p4;
            p1 = new PointD(0, 0);
            p2 = new PointD(Allocation.Width, 0);
            p3 = new PointD(Allocation.Width, Allocation.Height);
            p4 = new PointD(0, Allocation.Height);

            cr.MoveTo(p1);
            cr.LineTo(p2);
            cr.LineTo(p3);
            cr.LineTo(p4);
            cr.LineTo(p1);
            cr.ClosePath();

#pragma warning disable CS0612 // Type or member is obsolete
#pragma warning disable CS0618 // Type or member is obsolete
            cr.Color = new Cairo.Color(_clearColor.R, _clearColor.G, _clearColor.B);
            cr.FillPreserve();
            Gdk.CairoHelper.SetSourceWindow(cr, GdkWindow, 0, 0);
#pragma warning restore CS0618 // Type or member is obsolete
#pragma warning restore CS0612 // Type or member is obsolete
            cr.Paint();

            Gtk3Backend.BeginDraw(WidthRequest, HeightRequest);
            Gtk3Backend.GraphicsDevice.Clear(_clearColor);
            Gtk3Backend.GraphicsDevice.Viewport = _view;

        //    Render -= null;

            Gtk3Backend.EndDraw();

#pragma warning disable CS0618 // Type or member is obsolete
            cr.SetSourceSurface(GdkWindow.CreateSimilarSurface(Cairo.Content.Color, 0, 0), 0, 0);
#pragma warning restore CS0618 // Type or member is obsolete
            // e.Graphics.DrawImage(UniversalBackend.Present(), 0, 0, Width, Height); ???
            return base.OnDrawn(cr);
        }
    }
}

Test App made by @SpiceyWolf

I have replace to Gtk Sharp 3:

using Gtk;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Gtk3;

namespace Game1
{
    /**
     *  MonoGame embeds into Gtk Sharp 3 Wow - But I will fix soon
     */
    class MainWindow : Window
    {
        private HBox _mainBox;
        private RenderWidget _area1;
        private RenderWidget _area2;

        // MonoGame
        private SpriteBatch sb;
        private Texture2D tex1;
        private Texture2D tex2;

        public MainWindow():base(WindowType.Toplevel)
        {
            Title = "MonoGame GLAreas embed in Gtk Sharp 3";
            SetDefaultSize(800, 400);
            DeleteEvent += new DeleteEventHandler(deleteEventHandler);
            // Build user interface
            _mainBox = new HBox();

            _area1 = new RenderWidget();
            _area1.BackColor = new Gdk.Color(1, 0, 0);
            _area1.Render += new RenderHandler(render_01);
            _mainBox.PackStart(_area1, true, true, 0);

            _area2 = new RenderWidget();
            _area2.BackColor = new Gdk.Color(1, 1, 0);
            _area2.Render += new RenderHandler(render_02);
            _mainBox.PackEnd(_area2, true, true, 0);

            Add(_mainBox);
            ShowAll();

            // MonoGame's feature
            sb = new SpriteBatch(Gtk3Backend.GraphicsDevice);
            tex1 = Gtk3Backend.Content.Load<Texture2D>("Content/1");
            tex2 = Gtk3Backend.Content.Load<Texture2D>("Content/2");
        }

        private void deleteEventHandler(object o, DeleteEventArgs args)
        {
            // for Gtk Window
            Application.Quit();
        }

        private void render_02(object o, RenderArgs args)
        {
            sb.Begin();
            sb.Draw(tex2, Vector2.Zero, Color.White);
            sb.End();
        }

        private void render_01(object o, RenderArgs args)
        {
            sb.Begin();
            sb.Draw(tex1, Vector2.Zero, Color.White);
            sb.End();
        }
    }
} 

I am surprised because it crashed now. How did you stop to Gtk Sharp 3. Oh my god how are apple and pengiun now? They are unhappy again. If Gtk Sharp 3 and MonoGame crash initialy.

// EDIT MonoGame Framework 3.7 from build download of TeamCity is down of MonoGame :frowning: http://teamcity.monogame.net/viewLog.html?buildTypeId=MonoGame

// EDIT 2:
@SpiceyWolf
You forget force exit

UniversalBackend.ForceExit()

I have added new method if you forget to exit if you close WinForm I think you need add:

Enviromment.Exit(-1);

Than it works fine.

You shouldnt need platform specific compiles for mac and linux… whatever pinvokes u have they wont cause a problem unless u attempt to call them at runtime (And if u dont on the incorrect platform they are unused) and .Net has a couple of functions for determining if you are on Linux Mac, Xbox, or a bunch of variants of windows so you can use them to split up your platform specific calls to just have 1 big DesktopGL MonoGame.Forms

No no, I just used mono to compile the original MonoGame.Forms project.

In the project itself I already use preprocessor directives to determine if DirectX or OpenGL is in use. There are no P/Invokes.

Having one big DesktopGL project wouldn’t make sense in the moment, because the WindowsDX project is still a lot faster. Reason for this is the SwapChainRenderTarget, which is still only available in a WindowsDX project.

The “SwapChainRenderTarget.GL” in MonoGame.Forms still needs big performance improvements, but i’m optimistically to archive this performance boost with Pixel Buffer Objects in the future.

It could make sense to open an issue on the MonoGame repo where we ask for the specific BufferTarget implementation. But first I want to release the basic structure.

Yea if u can manage to make a fairly accurate SwapChain recreation in GL im sure theyd allow that API to embed into the official DesktopGL and u could then finally have 1API work for both versions VERY efficiently :smiley: