Video player on dx hosed ?

Have you installed any codecs lately? could be clashing…

What coding are you using for the audio? avoid 24bit audio, it used to cause so many issues in playback for me in the past.

Hmm, what formats are supposed to be supported anyway?

Found this but no video format associated:

Well codec’s are a grey area to me most of these clips that used to work are sitting in a older project i put them all in a new one as well but no joy.

System.InvalidOperationException: 'cannot start video’

Im honestly scratching my head at what is going on im open to any suggestions.

I had a ffmpeg routine that would put all my vids into a mp4 format that monogame’s video player would play. But that no longer works, worse i can’t seem to generate even a single reliable mp4 or wmv format now for monogame that will run.

The (only) one of my old video’s that i can get to play… has this format.

Works.

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.56.100
  Duration: 00:05:06.27, start: 0.000000, bitrate: 478 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 345 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)

I tried to generate another one with a script to get it as close to that as possible but even this close doesn’t work. This video is a clip off a thor movie for testing that used to work.

Doesn’t work.

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.17.100
  Duration: 00:01:52.18, start: 0.000000, bitrate: 302 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 4:3 DAR 16:9], 170 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc
(default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)

altered it again still doesn’t work.

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.17.100
  Duration: 00:01:52.20, start: 0.000000, bitrate: 309 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 4:3 DAR 16:9], 172 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (defaul
t)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)

the most ridiculous part is it all plays in vlc with no problems.

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.17.100
  Duration: 00:01:18.41, start: 0.000000, bitrate: 1030 kb/s
    Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 680x480 [SAR 1722:1003 DAR 287:118], 922 kb/s, 29.97 fps, 29.97
 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 101 kb/s (default)

I think what i need to know is what kind of encoding can monogame’s video player reliably decode.

Looking at the code, it is really nasty.

In VideoContent.cs I can see this code

 var result = ExternalTool.Run("ffprobe",
                string.Format("-i \"{0}\" -show_format -select_streams v -show_streams -print_format ini", Filename), out stdout, out stderr);

It then parses the returned string to get things like width, height and r_framerate

I would try that command from the cmd prompt and see what it returns for your videos.

The error comes from this code

             PlatformPlay();

            _state = MediaState.Playing;

            // XNA doesn't return until the video is playing
            const int retries = 5;
            const int sleepTimeFactor = 50;

            for (int i = 0; i < retries; i++)
            {
                if (State == MediaState.Playing )
                {
                    break;
                }
                var sleepTime = i*sleepTimeFactor;
                Debug.WriteLine("State != MediaState.Playing ({0}) sleeping for {1} ms", i + 1, sleepTime);
#if WINDOWS_UAP
                Task.Delay(sleepTime).Wait();
#else
                Thread.Sleep(sleepTime); //Sleep for longer and longer times
#endif
            }
            if (State != MediaState.Playing )
            {
                //We timed out - attempt to stop to fix any bad state
                Stop();
                throw new InvalidOperationException("cannot start video"); 
            }

Which basically waits for a period of time while it hopes the underlying platform loads the video and starts playing it.

The underlying video decoder is from using SharpDX.MediaFoundation;

Might be worthwhile checking if that has changed recently.

I can’t see any obvious bugs in the code, I am worried about the threading though. No idea what thread affinity MediaFoundation has got.

I also think the timeout could be too fast.

(1 + 2 + 3 + 4) * 50 is only 500 mS

Spent an hour putting together a post, and just deleted it, could you show me some code to load a video? I have the xnb and mp4 in Content, but my video code from XNA4 does not seem to be fully working or maybe some things have changed?

I have:

protected override void LoadContent()
{
    // Create a new SpriteBatch, which can be used to draw textures.
    spriteBatch = new SpriteBatch(GraphicsDevice);

    // TODO: use this.Content to load your game content here

    video = Content.Load<Video>("video/test");
    player = new VideoPlayer();
    player.IsLooped = true;
    player.Play(video);
}

Keeps telling me IsLooped is not implemented…

And this:

// TODO: Add your drawing code here
spriteBatch.Begin();
spriteBatch.Draw(player.GetTexture(), GraphicsDevice.Viewport.Bounds, Color.White);
spriteBatch.End();

Basically I need some help in setting up video playback, I recall people shooting it to an RT, is that a must now?

Would really like to get to the bottom of your issue and now this too as I think it would help everyone.

I am not deep into MG yet to be looking at this as such my experience is minimal at present.

Oh and it forced me to add this of course:

using Microsoft.Xna.Framework.Media;

EDIT

Almost forgot to add, I do see a white frame but no video with IsLooped coded out.

Ya its this is a little messy cause i was hacking it up with consoles thinking i messed it up on my end till i found the old video that worked.

using System;
using System.Collections.Generic;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;

namespace Microsoft.Xna.Framework
{
    public static class VideoController
    {
        public static Video video;
        public static VideoPlayer player;
        public static Texture2D videoFrameTexture;
        public static Texture2D dotTexture;
        public static bool isPlayerInitialized = false;
        public static float videoFps = 1f / 24f;
        public static float timeToNextFrame = 1f / 24f;

        public static void LoadVideoPlayer(GraphicsDevice device, Microsoft.Xna.Framework.Content.ContentManager content, string FileName)
        {
            if (isPlayerInitialized == false)
            {
                VideoController.video = content.Load<Video>(FileName);
                dotTexture = TextureDotCreate(Color.White, device);
                videoFrameTexture = dotTexture;
                videoFps = 1f / video.FramesPerSecond + .01f;
                timeToNextFrame = videoFps;
                //player = new VideoPlayer();
                //isPlayerInitialized = true;
                Console.WriteLine(" video.FileName: " + video.FileName.ToString());
                Console.WriteLine(" video.Width: " + video.Width.ToString());
                Console.WriteLine(" video.Height: " + video.Height.ToString());
                Console.WriteLine(" video.FramesPerSecond: " + video.FramesPerSecond.ToString());
                Console.WriteLine(" videoFps: " + videoFps.ToString());
            }
        }

        public static bool IsPlaying()
        {
            if (player.State == MediaState.Playing)
                return true;
            else
                return false;
        }
        public static string GetInfo()
        {
            Console.WriteLine(" video.FileName: " + video.FileName.ToString());
            Console.WriteLine(" video.FramesPerSecond: " + video.FramesPerSecond.ToString());
            Console.WriteLine(" video.Width: " + video.Width.ToString());
            Console.WriteLine(" video.Height: " + video.Height.ToString());
            string msg = "";
                msg += ">player.Video.FileName: " + player.Video.FileName.ToString() +
                " player.PlayPosition: " + player.PlayPosition.ToString() +
                " player.Video.FramesPerSecond: " + player.Video.FramesPerSecond.ToString() +
                " player.Video.Width: " + player.Video.Width.ToString() +
                " player.Video.Height: " + player.Video.Height.ToString()
                ;    
            return msg;
        }

        public static void PlayOrResumeVideo()
        {
            if (isPlayerInitialized == false)
            {
                timeToNextFrame = videoFps;
                videoFrameTexture = dotTexture;
                player = new VideoPlayer();
                player.Play(video);
                isPlayerInitialized = true;
                timeToNextFrame = videoFps*10f;
                Console.WriteLine(" video.FileName: " + video.FileName.ToString());
                Console.WriteLine(" video.FramesPerSecond: " + video.FramesPerSecond.ToString());
                Console.WriteLine(" video.Width: " + video.Width.ToString());
                Console.WriteLine(" video.Height: " + video.Height.ToString());
            }
            else
            {
                if (player.State == MediaState.Paused)
                {
                    Console.WriteLine(">PlayVideo< status PausedVideo, State:: " + player.State.ToString());
                    // could be null or its stopped
                    player.Resume();
                    // sometimes throws "cannot start video"
                    // An unhandled exception of type 'SharpDX.SharpDXException' occurred in SharpDX.dll
                    // Additional information: HRESULT: [0x80004003], Module: [General], ApiCode: [E_POINTER/Invalid pointer], Message: Invalid pointer
                }
                if (player.State == MediaState.Stopped)
                {
                    Console.WriteLine(">PlayVideo< status StoppedVideo, State:: " + player.State.ToString());
                    // could be null or its stopped
                    player.Play(video); // sometimes throws "cannot start video"
                }
                Console.WriteLine();
            }
        }
        public static void PauseVideo()
        {
            if (isPlayerInitialized)
            {
                if (player.State == MediaState.Playing)
                {
                    Console.WriteLine(">PauseVideo< status PlayingVideo, State: " + player.State.ToString());
                    player.Pause();
                    Console.WriteLine("-player.Pause();  result: " + player.State.ToString());
                    // State returns that it is playing
                }
                Console.WriteLine();
            }
        }
        public static void StopVideo()
        {
            if (isPlayerInitialized)
            {
                if (player.State == MediaState.Playing)
                {
                    Console.WriteLine(">PauseVideo< status PlayingVideo, State: " + player.State.ToString());
                    player.Stop();
                    Console.WriteLine("-player.Stop();  result: " + player.State.ToString());
                }
                if (player.State == MediaState.Paused)
                {
                    Console.WriteLine(">PauseVideo< status PaudedVideo, State: " + player.State.ToString());
                    player.Stop();
                    Console.WriteLine("-player.Stop();  result: " + player.State.ToString());
                }
                Console.WriteLine();
            }
        }
        public static int PlayPosition()
        {
            if (isPlayerInitialized)
            {
                return player.PlayPosition.Seconds;
            }
            else
            {
                return 0;
            }
        }
        public static bool CheckTimeDecrement(GameTime gameTime)
        {
            float elapsed = (float)gameTime.ElapsedGameTime.TotalSeconds;
            timeToNextFrame -= elapsed;
            if (timeToNextFrame < 0f)
            {
                timeToNextFrame = videoFps;
                return true;
            }
            else
            {
                return false;
            }
        }
        public static Texture2D GetVideoFrame(GameTime gameTime)
        {
            if (isPlayerInitialized)
            {
                if (player.State == MediaState.Playing && CheckTimeDecrement(gameTime))
                {
                    Texture2D tmp;
                    try
                    {
                        tmp = player.GetTexture();
                    }
                    catch (InvalidOperationException e)
                    {
                        Console.WriteLine(e.ToString());
                        tmp = dotTexture;
                    }

                    // An unhandled exception of type 'SharpDX.SharpDXException' occurred in SharpDX.dll
                    // Additional information: HRESULT: [0x8007000E], Module: [General], ApiCode: [E_OUTOFMEMORY/Out of memory], Message: Not enough storage is available to complete this operation.
                    if (tmp != null)
                    {
                        videoFrameTexture = tmp;
                    }
                }
            }
            if(videoFrameTexture == null)
            {
                videoFrameTexture = dotTexture;
            }
            return videoFrameTexture;
        }
        public static void Draw(SpriteBatch spritebatch, Rectangle drawRectangle, GameTime gameTime)
        {
            if (isPlayerInitialized)
            {
                spritebatch.Draw(VideoController.GetVideoFrame(gameTime), drawRectangle, new Rectangle(0, 0, videoFrameTexture.Width, videoFrameTexture.Height), Color.White, 0f, Vector2.Zero, SpriteEffects.None, 0f);
            }
        }
        private static Texture2D TextureDotCreate(Color c, GraphicsDevice device)
        {
            Color[] data = new Color[1];
            data[0] = c;
            Texture2D result = new Texture2D(device, 1, 1);
            result.SetData(data);
            return result;
        }
    }
}

then game1

load,

VideoController.LoadVideoPlayer(GraphicsDevice, Content, "WarcraftClip");

update,

if (Keyboard.GetState().IsKeyDown(Keys.P))
{
      VideoController.PlayOrResumeVideo();
      Console.WriteLine( VideoController.GetInfo() );
}

draw,

VideoController.Draw(spriteBatch, destinationRectangle, gameTime);

1 Like

So, solved or? also, is there nothing more basic than that? :grinning:

No not solved at all all my vids just throw the error except for one.
The problem is the content pipeline / video player or something all the h264 vids i load and the wmv’s basically wont play.

This class isn’t the problem. This is just my class to make playing videos simpler and to add a little extra error protection in case of null textures that aren’t actually app crashing if handled.

is there nothing more basic than that?

Though copy paste that into a class in your project.
Then add 3 lines seems pretty simple to me.

That’s odd, hope someone with more experience can chime in here.

It looks like good code though, just wanted to see if getting those errors I got were the same for you.

Hmm, I suspect I could cheat on UWP though…

I mean ill post up my ffmpeg class if you want to dl the ffpeg binary and try to find a script that matches what monogame wants. I mean the one video i have does play at this point that’s the real mystery. Why when nothing else will i suppose if i can just figure out why then it wont matter i can just format all my vids to fall in line.

Not so keen on playing with ffmpeg right now.

Keep us posted on developments though :pray:

I am trying to get around to catching up with MG now… This has me pondering a lot…

Ok so i started at this again today, i managed to generate a h264 that doesn’t die right off the rip, It returns a null texture but the sound plays.

 Metadata:
   major_brand     : mp42
   minor_version   : 512
   compatible_brands: isomiso2avc1mp41
   encoder         : Lavf58.17.100
 Duration: 00:01:18.46, start: 0.000000, bitrate: 781 kb/s
   Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 680x480 [SAR 1:1 DAR 17:12], 636 kb/s, 24 fps, 24 tbr, 12288 tbn, 48 tbc (default)
   Metadata:
     handler_name    : VideoHandler
   Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 137 kb/s (default)

The conversion script i used for this one is as follows.

// sound works not video, but it doesn’t fail to play.
"-vf setsar=1,format=yuv420p -r 24 -c:v libx264 -profile:v main -brand mp42 -color_primaries bt709 -color_trc bt709 -colorspace bt709"

Wish we had a video expert around to fix this player there was a guy talking about including the vlc into mg but i dunno whats up with that now.

Well i got wmv movies to work but not the h264 and i tried like 30 different things.

wmv script is pretty simple.

ffmpeg.FFmpegVideoConvertCustom("-b:v 2M -vcodec msmpeg4 -acodec wmav2 -b:v 192k -r 16");

-i filenameFrom
“-b:v 2M -vcodec msmpeg4 -acodec wmav2 -b:v 192k -r 16”
filenameTo

produceing the following output

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.17.100
  Duration: 00:01:18.64, start: 0.000000, bitrate: 373 kb/s
    Stream #0:0(eng): Video: msmpeg4v3 (MP43 / 0x3334504D), yuv420p, 680x480, 192 kb/s, SAR 1722:1003 DAR 287:118, 16 fps, 16 tbr, 1k tbn, 1k tbc
    Stream #0:1(eng): Audio: wmav2 (a[1][0][0] / 0x0161), 44100 Hz, stereo, fltp, 128 kb/s

1 Like

Switch audio down to 128bit giving 64bits per channel

Only issue is, WMV only works on Windows but I am fine with that :smirk:

Just need mp4 next…

Have you got a copy of MonoGame source code on your machine?

If you have, would you mind doing some tests for me?

  1. Change const int retries = 5; to const int retries = 50;

This will tell us if it is a timing issue with content loading. Possibly caused by threading issues as it obviously used to work.

  1. Display the results of the ffprobe for both wmv and 264

Just in case something in ffprobe has changed which will break the code

  1. Take one of the videos that didn’t work and remove the audio

Which will tell us if it is a video or audio problem

If you can do that I will try and find the time to do some fixes and maybe get looping working

1 Like

I don’t have the source on my machine and it’s been so long since i have built it i dread even trying.

Anyways im sure it is the video not the audio as the audio plays while the video is broken on some of the attempts i made.

The class i posted above handles the null textures and catches the exceptions and continues so you will hear the music while its throwing exceptions to the console and displaying a white rectangle which does in fact happen on a lot of the video’s i converted and tested.

Im suspecting it has something to do with the width height or sar dar values which relate to that but something is probably just broken.

The trick with the wvm is to use the newer windows codec i think a couple videos i had to post reformat them to something else then to wmv to get them to convert to proper but most just worked.

Here is how i change videos over from one to the other and get the information shown.

Okay, can you post a couple of videos somewhere then?

One that works and one that doesn’t.

If you can I will find some time to have a look.

At the moment I am fighting with Metal and could do with a distraction. It looks like ARC is so aggressive it is possible to create an empty texture in one line of code and try to fill it with pixels in the next line of code but ARC has deleted it already… anyway that’s my hell to deal with.

Sure here are 4 files
two of them that used to work in mp4 labeled,
two of them in wmv both converted one worked one that wont.

Two old mp4’s that used to work

  1. still works no changes at all.
  2. the video is broken but the audio plays

Two videos same movie clip one mp4 one wmv one works the other doesn’t.
This wmv wouldn’t work initially after being converted over from mp4. Most the wmv’s do work when converted. Not a big concern im pretty sure its the codec.
These two are a special case were this video still wouldn’t convert properly from mp4 for some reason. I had to re-record it using vlc to mp4 then reformat it which then worked. However the mp4 version here i couldn’t get to work after many many attempts.

Any other mp4’s or rather all others no matter how i tried to convert them now simply do not work at all. These first two mp4’s are old and are now my only working and semi working test mp4’s.

https://drive.google.com/open?id=1FvE0gwLgVcsmhRvj-Cwmd9LPpt6nmS-6

See the post here to convert or try to reconvert to a mp4 if you an find a format that should work you can test it with this i basically gave up on the h264 mp4’s for now its busted.

At the moment I am fighting with Metal and could do with a distraction.
It looks like ARC is so aggressive it is possible to create an empty
texture in one line of code and try to fill it with pixels in the next
line of code but ARC has deleted it already… anyway that’s my hell to
deal with.

Sounds horrible i will pray for your success good luck.

I have a h264 library in c++, I hope to generate c# bindings and add it to the content management system.

But it depends on how much time I get to work on it.

By the way I solved the ARC issue, I just turned it off. :relaxed:

I’m curious if anyone got anywhere with this? I just started working on trying to get some video playing in our title and am running into basically the same issues here.

I’ve able to get mp4 files through the video converter and when I play them using the VideoPlayer, I get audio but always get an exception because GetTexture always returns null. @StainlessTobii I tried changing the retries count as I’m building from source but it made no difference. I was able to debug a little bit into the MonoGame code and the issue seems to be that the SampleGrabber object inside the Video class never has its OnProcessSample callback called. Unfortunately this seems to happen from within SharpDX so I can’t actually step in to see what is happening, but it seems to be the root of the texture always being null.

I was able to mess around a little bit and get the callback to actually happen by not setting the Subtype value on the MediaType in the Video PlatformInitialize

_mediaType.Set(MediaTypeAttributeKeys.Subtype, new Guid("00000016-0000-0010-8000-00AA00389B71"));

This resulted in the callback being hit but resulted in a lot of other errors because the data was in an unexpected format. So it does seem like there is some issue with the format of the video and the format the VideoPlayer wants, but it seems a little weird considering I’m running the mp4 through the content pipeline.

As a side note, I’ve not been able to get any wmv files through the content pipeline so have not been able to test. They all fail with MGCB spitting out an error about an incorrect input format. However this was just for testing purposes as I would prefer to use mp4 anyway.

Apologies for raising the old thread, but after doing a lot of research I’ve found a lot of posts around the internet about having problems but pretty much no solutions. This seems like the closest anyone has been in getting an mp4 to work with MG.

Update - After trying out pretty much every codec known to man, I was able to get an mp4 file loading and playing properly by using the VLC converter and the following options:

  1. Profile - Video for MPEG4 1080p TV / Device
  2. Video Codec - MPEG-4
  3. Audio Codec - MP3

I was not able to get any other format working. The weird part is that the H264 video codec did not work even though the importer in the content pipeline in called the H264Importer.

1 Like