Refer to this for decreasing memory usage in this context. Are you calling CopyAbsoluteBoneTransformsTo every frame?
CopyAbsoluteBoneTransformsTo