nuget assimp net model loading

Ok so i’ve been working on and off at getting the nuget version of assimp.net to load up a model into MonoGame correctly for the past month or so. I have the basics working as you can see in the gif below though its far from done and its still got tons of problems.

Edit i just fixed this up a bit and made a github repo for it were all the work i do on it will go.

I moved the google drive full project zipped link to the below post further down but that will probably go out of data since i can’t edit posts after some time expire.

Ill accept any input or help on fixing it up or if you want to chip in.

Im however having a problem which is making me scratch my head.
When i look at how other viewers loop animations it seems they know something i don’t.
For example with the dude.fbx when i play the animation in my project and loop it.

I see a poping motion on the hand from the last frames to the first.

But other viewers like the internal vs studio don’t seem to get that effect.

Though it seems to have a stutter itself.

Now i am interpolating rotations to build new frames in total for a set amount of frames determined by a fps value i pass to the loader. So the final frames are just look ups it all looks right in the debuging and i would think its the model but it seems other viewers are handling looping differently or i have a bug.

But i cannot see anything wrong with the interpolation this part has been bugging me for a few days now,

this is the interpolation im doing which looks ok to me though this is probably the biggest method in the class.

        public void SetAnimationFpsCreateFrames(int animationFramesPerSecond, RiggedModel model)
        {
            fps = animationFramesPerSecond;
            TotalFrames = (int)(DurationInSeconds * (double)(animationFramesPerSecond));
            TicksPerFramePerSecond = TicksPerSecond / (double)(animationFramesPerSecond);
            SecondsPerFrame = (1d / (animationFramesPerSecond));
            CalculateNewInterpolatedAnimationFrames(model);
        }

        private void CalculateNewInterpolatedAnimationFrames(RiggedModel model)
        {
            // Loop nodes.
            for (int i = 0; i < animatedNodes.Count; i++)
            {
                // Make sure we have enough frame orientations alloted for the number of frames.
                animatedNodes[i].frameOrientations = new Matrix[TotalFrames];
                animatedNodes[i].frameOrientationTimes = new double[TotalFrames];
                
                // print name of node as we loop
                Console.WriteLine("name " + animatedNodes[i].nodeName);

                // Loop destination frames.
                for (int j = 0; j < TotalFrames; j++)
                {
                    // Find and set the interpolated value from the s r t elements based on time.
                    var frameTime = j * SecondsPerFrame + .0001d; 
                    animatedNodes[i].frameOrientations[j] = Interpolate(frameTime, animatedNodes[i]);
                    animatedNodes[i].frameOrientationTimes[j] = frameTime;
                }
                Console.WriteLine("");
            }
        }

        /// <summary>
        /// Hummm its just a little bit off it seems..
        /// </summary>
        public Matrix Interpolate(double animTime, RiggedAnimationNodes n)
        {
            var nodeAnim = n;
            // 
            Microsoft.Xna.Framework.Quaternion q1 = nodeAnim.qrot[0];
            Vector3 p1 = nodeAnim.position[0];
            Vector3 s1 = nodeAnim.scale[0];
            // 
            double tq1 = nodeAnim.qrotTime[0]; // =0;
            double tp1 = nodeAnim.positionTime[0]; ; // =0;
            double ts1 = nodeAnim.scaleTime[0]; //=0;
            // 
            int qindex1 = 0; int qindex2 = 0; // for output to console only
            int pindex1 = 0; int pindex2 = 0; // for output to console only
            // 
            int i2 = 0;
            if (nodeAnim.qrot.Count > 1)
            {
                i2 = 1;
            }
            Microsoft.Xna.Framework.Quaternion q2 = nodeAnim.qrot[i2];
            double tq2 = nodeAnim.qrotTime[i2];

            i2 = 0;
            if (nodeAnim.position.Count > 1)
            {
                i2 = 1;
            }
            Vector3 p2 = nodeAnim.position[i2];
            double tp2 = nodeAnim.positionTime[i2];

            i2 = 0;
            if (nodeAnim.scale.Count > 1)
            {
                i2 = 1;
            }
            Vector3 s2 = nodeAnim.scale[i2];
            double ts2 = nodeAnim.scaleTime[i2];

            //
            for (int frame = 0; frame < nodeAnim.qrot.Count; frame++)
            {
                var t = nodeAnim.qrotTime[frame];
                if (t <= animTime)
                {
                    //1___
                    q1 = nodeAnim.qrot[frame];
                    tq1 = t;
                    qindex1 = frame; // for output to console only
                    //2___
                    var frame2 = frame;
                    if (nodeAnim.qrot.Count > 1)
                    {
                        frame2 += 1;
                    }
                    if (frame2 >= nodeAnim.qrot.Count)
                    {
                        frame2 = 0;
                        q2 = nodeAnim.qrot[frame2];
                        tq2 = nodeAnim.qrotTime[frame2] + DurationInSeconds;
                        qindex2 = frame2; // for output to console only
                    }
                    else
                    {
                        q2 = nodeAnim.qrot[frame2];
                        tq2 = nodeAnim.qrotTime[frame2];
                        qindex2 = frame2; // for output to console only
                    }
                }
            }
            //
            for (int frame = 0; frame < nodeAnim.position.Count; frame++)
            {
                var t = nodeAnim.positionTime[frame];
                if (t <= animTime)
                {
                    //1___
                    p1 = nodeAnim.position[frame];
                    tp1 = t;
                    pindex1 = frame; // for output to console only
                    //2___
                    var frame2 = frame;
                    if (nodeAnim.position.Count > 1)
                    {
                        frame2 += 1;
                    }
                    if (frame2 >= nodeAnim.position.Count)
                    {
                        frame2 = 0;
                        p2 = nodeAnim.position[frame2];
                        tp2 = nodeAnim.positionTime[frame2] + DurationInSeconds;
                        pindex2 = frame2; // for output to console only
                    }
                    else
                    {
                        p2 = nodeAnim.position[frame2];
                        tp2 = nodeAnim.positionTime[frame2];
                        pindex2 = frame2; // for output to console only
                    }
                }
            }

            Microsoft.Xna.Framework.Quaternion q;
            if (tq1 != tq2)
                q = Microsoft.Xna.Framework.Quaternion.Slerp(q1, q2, (float)GetInterpolationTimeRatio(tq1, tq2, animTime));
            else
                q = q1;

            Vector3 p;
            if (tp1 != tp2)
                p = Vector3.Lerp(p1, p2, (float)GetInterpolationTimeRatio(tp1, tp2, animTime));
            else
                p = p1;

            Console.WriteLine("" + " animTime: " + animTime);
            Console.WriteLine(" q : " + " index1: " + qindex1 + " index2: " + qindex2 + " time1: " + tq1 + "  time2: " + tq2 + "  quaternion: " + q.ToString());
            Console.WriteLine(" p : " + " index1: " + pindex1 + " index2: " + pindex2 + " time1: " + tp1 + "  time2: " + tp2 + "  position: " + p.ToString());

            var r = Matrix.CreateFromQuaternion(q);
            r.Translation = p;

            return r; // * Matrix.CreateTranslation(p); //  * Matrix.CreateScale(s);
        }

        public double GetInterpolationTimeRatio(double s, double e, double val)
        {
            if (e < s)
                s = s + e;
            return (val - s) / (e - s);
        }

Id like to say i do have the basics flushed out and it does basically work for a rigged model.
This has 2 class files … a loader and a model class.
Though the Rigged model class has 4 nested classes i actually had them separate and then nested them on purpose i figured they are so specific to the model they belong to it and as part of it.

Then game1 uses the loader to load a model from the fbx to the riggedmodel.

            // load the model
            RiggedModelLoader modelReader = new RiggedModelLoader(Content, riggedEffect);
            int generatedAnimationFramesPerSecond = 24;
            model = modelReader.LoadAsset("dude.fbx", generatedAnimationFramesPerSecond);

It is somewhat sloppy as this is basically still a sort of rough draft even after all this time
I guess im running out of steam on this and im really looking for some suggestions or direction as well.

I did quite a bit so far.
I think the overall model structural ideology is pretty solid most of it just follows assimps ideology, it has both a tree node linked list and a flat node list of references that point to the tree nodes because for somethings its simpler to iterate a list.
I added the basic stuff so far , and a very basic shader to load it and show it as seen in the above pic.
I got the animations weights and meshes loading and the textures are added to the content pipeline for now.
I pass the content manager to the model and let it load the textures it finds in the fbx or dae or whatever to its own meshes if they are present in the content folder nothing fancy beyond that it doesn’t search content folders or subfolders ect yet.
I made a custom shader for it and its pretty basic atm but i guess that’s the easier part.
I still can’t see how the mesh transform is supposed to fit in and when either if anyone knows the idea behind how that is supposed to work exactly please let me know, it seems to be a scalar sometimes and other times its just i don’t even know what.

Btw if anyone has some complicated fbx models, that are fully rigged that work as well.

That you know they work.

Basically testing models that they could send me so i can make comparisons i would appreciate it.
Most of what ive found on the net are just regular models with no rigging or the rigging is messed up in multiple viewers.

Im also looking for some models with deformers and or normal / height maps.

I think you didn’t interpolate up to the last keyframe.

Humm if i add a extra frame ill interpolate just barely past it but i get the same thing.

in RiggedModel -> RiggedAnimation -> SetAnimationFpsCreateFrames(…)
TotalFrames = (int)(DurationInSeconds * (double)(animationFramesPerSecond)) +1; // +1

The thing is the original set of of animations for the left hand has only 3 keys rotation and 1 for position.
Which is just making my head spin.

// this is what is in the fbx read in from assimp.

   Channel L_Hand
   (time is in animation ticks it shouldn't exceed anim.DurationInTicks 29 or total duration in seconds: 1.20833333333333)
     Position Keys: 1
       index[0]    Time: 14                secs: 0.583333333333333  Position: {{X:10.82555 Y:5.22715E-07 Z:9.902142E-07}}
     Rotation Keys: 3
       index[0]    Time: 14                secs: 0.583333333333333  QRotation: {{W:0.7862441 X:-0.594564 Y:-0.0645714 Z:-0.1553847}}
       index[1]    Time: 22                secs: 0.916666666666667  QRotation: {{W:0.6790287 X:-0.7253733 Y:-0.0950051 Z:-0.06105447}}
       index[2]    Time: 29                secs: 1.20833333333333   QRotation: {{W:0.7929463 X:-0.6023548 Y:0.05393244 Z:0.07413663}}

// this is the interpolation output i get. w is on the opposite side in the displayed data for assimp.
// looks right.

q : index1: 2 index2: 0 time1: 1.20833333333333 time2: 1.79166666666667 quaternion: {X:-0.6023569 Y:0.0539122 Z:0.07409727 W:0.7929497}

AnimationTime: 1.1251
 q :  index1: 1 index2: 2 time1: 0.916666666666667  time2: 1.20833333333333  quaternion: {X:-0.6420997 Y:0.01135957 Z:0.03567142 W:0.7657065}
 p :  index1: 0 index2: 0 time1: 0.583333333333333  time2: 0.583333333333333  position: {X:10.82555 Y:5.22715E-07 Z:9.902142E-07}
AnimationTime: 1.16676666666667
 q :  index1: 1 index2: 2 time1: 0.916666666666667  time2: 1.20833333333333  quaternion: {X:-0.6226404 Y:0.03269452 Z:0.0549657 W:0.7798902}
 p :  index1: 0 index2: 0 time1: 0.583333333333333  time2: 0.583333333333333  position: {X:10.82555 Y:5.22715E-07 Z:9.902142E-07}
AnimationTime: 1.20843333333333
 q :  index1: 2 index2: 0 time1: 1.20833333333333  time2: 1.79166666666667  quaternion: {X:-0.6023569 Y:0.0539122 Z:0.07409727 W:0.7929497}
 p :  index1: 0 index2: 0 time1: 0.583333333333333  time2: 0.583333333333333  position: {X:10.82555 Y:5.22715E-07 Z:9.902142E-07}

I ignore scaling atm as its not requisite.

that codes in the Riggedmodel RiggedAnimation nested class

which also had this bug that i fixed but didn’t upload.

        public double GetInterpolationTimeRatio(double s, double e, double val)
        {
            if (e < s)
                e = s + e;  // s = s + e;  durrr did that backwards still doesn't fix anything.
            return (val - s) / (e - s);
        }

The original animation output data can be seen in the console by fliping on the bool at the top of the loading class. RiggedModelLoader public bool startupAnimationConsoleInfo = true;

I put the original animation data in the model on purpose so later on if i generate xnb’s ill just keep the original data and rebuild on load or save both didn’t decide not really there yet.

i dunno maybe its not bugged maybe vs just skips the first couple frames or redoes them or something weird.
maybe its this fbx version of the dude.fbx i have has messed up animation frames ?

Do you generate keyframes at time [0] & [Duration] for all bones?
I think those keyframes must be an interpolation between the last and first true Keyframe.

Do you mean generate frame zero from what is in the local node transforms.

I pull them in and set them when i load the models but i don’t add them to the animation keys.
As all of them have at least 1 animation frame 0 already and that starting bind pose has the hand not open.
Is that supposed to be added to the animation sets manually ?
It crossed my mind but it seemed to me that if that was supposed to be in the key frames it would be in there already.

Look at the keyframes of L_Hand.
First one is at 0.5833s and the last one at 1.2083s (Total duration).
How do you interpolate frames from 0.000s to 0.5833s?
I think you have to wrap around time and interpolate with the last keyframe.
If you fill them with the same value from the first keyframe at 0.5833s then the hand ‘jumps’ every time the animation restarts.

I saw a very similar behavior with the frame interpolation of my animation/skinning library.
Once I time-wrapped the interpolation the animation was seamless as with the XNA importer.

1 Like

Ah thanks nkast
I see the time stamps in the animations can start beyond zero.
Im actually pulling the wrong index for all the starting indexs.

Probably the only value i didn’t look at in the output.

interpolationTime: -1.75

name L_Hand
AnimationTime: +0.00
 q :  index1: 0 index2: 0 time1: +0.583    time2: +0.917    interpolationTime: -1.75     quaternion: x: -0.314  y: -0.006  z: -0.304  w: +0.899
 p :  index1: 0 index2: 0 time1: +0.583    time2: +0.583    interpolationTime: -∞        position: +10.826 , +0.00   , +0.00

Ah wait that leads to another question …

Is that right grab the before time index of the animation in this scenario which would actually mean starting with the last frame of the animation ?
Or are we supposed to be manually add in the starting bind pose and insert it into frame 0 for time 0?

Re:edit yep weird the last frame is basically time 0 so i guess that is how it is meant to be.

The nuget assimp.net is like 30mb alone so it’s pretty fat but it should just work.

dude.fbx from the xna skinned model.

victoria hat dance from the xna better skinned model. (hat isn’t working)

I added the ability to just make the total frame time longer.

I put a specular light in the shader for the example, im not using phong im using my own reflection based lighting which is very maluable and doesn’t use a half vector, but its not using any values out of the fbx file to alter the shader to it or anything i just kinda figure meh.

I also have no clue how to actually pull a texture out of a fbx if its embedded in the file itself ?
Well it works for rigged animations atm and it can run multiple animations.

2 Likes

I just about have this thing working at least rotation animation wise i fixed the shader and made another fix to the interpolation so that it can properly loop around by just increasing the total animation time i added in a switch to just directly calculate the interpolated orientations as well directly.

The scaling is a pain though i think i need to scale the whole fbx loaded model back to be unit length the problem is even a fbx depending on how it is made has different early nodes and mesh nodes that have wonky transforms. Like depending on if a model has mesh node transforms or bone node transforms, then the mesh has to be multiplied to the whole transform chain or omitted. Some models seem to have the scale in the animated bone nodes some don’t, but, i have no idea how to programatically detect that.

Open assimp doesn’t seem to distinguish the animation type properly for HasMeshAnimations or HasNodeAnimations.
Like i have this in at the moment but its just a poor hack around.

        private void UpdateMeshTransforms()
        {
            // try to handle when we just have mesh transforms
            for (int i = 0; i < meshes.Length; i++)
            {
                // This feels errr is hacky.
                //meshes[i].nodeRefContainingAnimatedTransform.CombinedTransformMg = meshes[i].nodeRefContainingAnimatedTransform.LocalTransformMg * meshes[i].nodeRefContainingAnimatedTransform.InvOffsetMatrixMg;
                if (originalAnimations[CurrentPlayingAnimationIndex].animatedNodes.Count > 1)
                {
                    meshes[i].nodeRefContainingAnimatedTransform.CombinedTransformMg = Matrix.Identity;
                }
                   
            }
        }

The real problem is the mesh transform is needed for models with animation without armatures and bones.
And the mesh and armature messes up scaling for models without mesh animations but with armature and bones.
I could just remove all the armature nodes i haven’t seen a animated node for them yet, but i dunno if ill need them for some models. Still that would leave scaled bone animation nodes as needing to be made unit length based on the inverted mesh scale only and the rotation would have to remain. Damn it’s a nightmare to load a blender made fbx with scaling.

fbx mesh animated cube.

Well this does work for simple fbx animated models with multiple animations though.
but the above hack around will break if there is more then one animated mesh.
I haven’t made any sort of xnb tool yet as there is no point till i get this into better shape.

If anyone wants to download it and make some suggestions on how to handle the scaling and the mesh vs bone transform chain problem… dl the project flip the console on and take a look.