If you (or anyone else) runs into difficulty’s again with this at a later date post up here and I will share some of my code. I went through all this a while back when implementing a scene graph and found lots of oddities (which I largely have forgotten and didn’t take note of!)
But it mainly related to non uniform scaling and rotation. I’m a bit fussy to have things “right”, and it probably would rarely be an issue in practice so maybe no one cares, but with unity and actually any established render engine I have ever used they support this correctly with skew. I’d be happy to share the base display node class which has much of this stuff in it.
If I remember correctly I had to do a custom matrix decompose method which I can also share.