1
\$\begingroup\$

I have an .fbx file containing a an avatar that is standing still.

I followed the steps in this GitHub repository section "Motion Generation" to create an animation for that avatar. What I received were an .mp4 file and a .npy file which contain the movement that I want the avatar to do.

Is there any way to connect between them so that the avatar will move like in these files? The animations are going to be applied to many different .fbx files.

I am sort of looking for something like Mixamo, but with a way of importing an animation and not only an avatar.

\$\endgroup\$
8
  • \$\begingroup\$ MP4? Isn't that a format for video files? I didn't know that those could contain animations for 3d models. Usually it's not a problem to apply animations from one humanoid model in Unity to another, provided that the avatars are set up correctly and that the animations can be imported as Unity animations. \$\endgroup\$ Commented Aug 16, 2022 at 10:27
  • \$\begingroup\$ @Philipp You are most likely correct as I am a complete beginner. I used some model to generate a motion and as a result what I got were a .mp4 file with a video of said motion and a .npy file if that helps. \$\endgroup\$ Commented Aug 16, 2022 at 10:48
  • \$\begingroup\$ Well, a 2d video of a character performing a motion does not include the technical 3d data of the motion itself. I haven't encountered .npy files before. A web search says it has something to do with the Python library NumPy? How exactly did you "use some model" to "generate a motion"? What tools did you use? \$\endgroup\$ Commented Aug 16, 2022 at 10:56
  • \$\begingroup\$ @Philipp I understand. I was using the following repo and followed the steps in the "motion generation" part. github.com/hongfz16/AvatarCLIP. \$\endgroup\$ Commented Aug 16, 2022 at 11:04
  • 1
    \$\begingroup\$ From the description, it sounds to me like these npy files contain the parameters fed into the machine learning model to generate this motion. So it's not conventional skeletal key frames like we'd usually use in a game engine. You'll either need to convert this animation to keyframes in a format Unity understands like fbx, or implement this ML model in your Unity project, so you can feed it parameters and get out poses to apply via script at runtime. Both will take a deeper understanding of this ML model than most gamedevs have — can you contact other AI folks using this model? \$\endgroup\$ Commented Aug 16, 2022 at 11:42

1 Answer 1

Reset to default
1
\$\begingroup\$

Try using their Avatar2FBX tool. Just before the section on Motion Generation, they have this section:

Convert Avatar to FBX Format:
For the convenience of using the generated avatar with modern graphics pipeline, we also provide scripts to rig the avatar and convert to FBX format. See the instructions here.

However I'm a bit skeptical will do everything you need as that page lists animation as TODO & near the bottom it notes:

we currently only supports 'stand pose' and 'zero beta'

Their Avatar2FBX tool appears to be based on other work. If their 'bundled' version doesn't work, you could try looking for a more active or original source with the hopes that it has the functionality you're looking for.

\$\endgroup\$
2
  • \$\begingroup\$ I am a bit confused as they specifically show animated avatars examples at the top of the repo, but there doesn't seem to be a way to actually animate them. \$\endgroup\$ Commented Aug 16, 2022 at 17:14
  • \$\begingroup\$ They mention using the motion libraries Blender, Unity & Mixamo to animate the FBX models. They also discuss motion generation via pose interpolation. I didn't mention those because your question was about conversion to FBX. It's worth bearing in mind that this is research work. The goal is to advance the state of the art more than it is to release a viable, user ready product. \$\endgroup\$ Commented Aug 16, 2022 at 22:24

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.