Switching to windowed mode DX crash

Okay, so this will be most likely a quite hard issue to solve. Game switches without issues during runtime into full screen but when switched back to window mode:

An unhandled exception of type 'SharpDX.SharpDXException' occurred in SharpDX.dll

Additional information: HRESULT: [0x887A0001], Module: [SharpDX.DXGI], ApiCode: [DXGI_ERROR_INVALID_CALL/InvalidCall], Message: Unknown

My theory is this is caused by content lost, as graphic device should restart during this procedure. But top of the stack looks like this:

at SharpDX.Result.CheckError()
at SharpDX.DXGI.SwapChain.ResizeBuffers(Int32
bufferCount, Int32 width, Int32 height, Format newFormat, SwapChainFlags
at Microsoft.Xna.Framework.Graphics.GraphicsDevice.CreateSizeDependentResources(Boolean useFullscreenParameter)
at Microsoft.Xna.Framework.GraphicsDeviceManager.ApplyChanges()
at CoOpSpRpG.CONFIG.applyResolution(Boolean resizing) in g:\GitHub\WTF\WTFNeutral\WTFNeutral\Static\CONFIG.cs:line 279
at CoOpSpRpG.CONFIG.toggleFullScreen() in g:\GitHub\WTF\WTFNeutral\WTFNeutral\Static\CONFIG.cs:line 246

We are not even sure if it can reach point where content lost can be handled as it seems it crashes before it. I know we can use borderless windows mode and for now we will do so but we would like to have proper fullscreen with ability to switch to windowed in future. Any leads will be helpful.