Nvidia is open-sourcing Audio2Face, its AI-powered tool that generates realistic facial animations for 3D avatars based on audio input. Developers can use the tool and its framework to create realistic 3D characters.

Audio2Face works by analysing the “acoustic features” of a voice, allowing it to generate animation data that it maps to the 3D avatar’s facial expressions and lip movement. It can be used to create 3D characters for pre-scripted content, as well as for livestreams.