Audio to Animation AI-Integration in iClone with Audio2Face

News provided by

Reallusion Inc.

Mar 14, 2024, 10:00 ET

Seamless Solution leveraging iClone, Character Creator, and Audio2Face

SAN JOSE, Calif., March 13, 2024 /PRNewswire-PRWeb/ -- Reallusion is thrilled to announce the seamless integration between Character Creator, iClone, and Audio2Face. This robust connection, empowered by NVIDIA's AI animation technology, revolutionizes multi-lingual facial lip-sync animation production. Not only does this integration bolster NVIDIA's Audio2Face with a versatile cross-application character system, it also enhances facial editing capabilities, enabling users to export character animations to leading 3D engines like Blender, Unreal Engine, Unity, and Omniverse.

NON-LINEAR AI ANIMATION GENERATED BY AUDIO2FACE

  • Lip-Sync Animation and Expressions Straight from Audio

As an AI-powered application, Nvidia Audio2Face (A2F) produces expressive facial animations solely from audio input. In addition to generating natural lip-sync animations for multilingual dialogue, the latest standalone release of Audio2Face also supports facial expressions, featuring slider controls and a keyframe editor.

  • Multi-Language Lip-Sync and Singing Animation

Unlike the majority of English-centric lip-sync solutions, Audio2Face stands out with its exceptional ability to generate animation from any language, including songs and gibberish. Besides the standard AI model Mark, have access to Clair, a new deep-learning model tailored for female characters proficient in Asian languages. Clair's friendly complexion is well-suited to customer interaction.

SEAMLESS CC - A2F - ICLONE INTEGRATION
Two complimentary plugins enable an automated workflow. With just a single click, configure a CC character in NVIDIA Audio2Face, animate it in real-time alongside an imported audio track, and seamlessly transfer the talking animation back to iClone for additional refinement before exporting it to 3D tools and game engines.

  • One-Click CC Character Setup in Audio2Face

The CC Character Auto Setup plugin for Audio2Face is the result of a collaboration between NVIDIA and Reallusion, condensing the manual 18-step process into a single step. By importing a CC character and choosing a training model — Mike or Clair — artists can instantly witness lifelike talking animations synchronized with audio files. Experiment with motion sliders, automatic expressions, and even set keyframes. The finalized animations can then be sent to iClone for additional refinement.

  • Full-Spectrum Animation Refinement using iClone

The free Nvidia Audio2Face plugin for iClone is tailored to receive animation data from Audio2Face. In addition to importing animations, it enhances the liveliness of facial features, resulting in a superior cut suitable for final production.

  • Facial Adjustment by Parts

Animations can be tweaked via a dynamic interface. Adjust various parameters such as expression strengths, head movements, or adding darting eyes to enliven the performance. Enlarge the jaw open range to enhance emotional tension and fine-tune the position of the tongue to mimic precise enunciation.

  • Smoothness Enhancement

Generative AI animation is susceptible to noise, particularly when audio files are captured by low-fidelity devices or within unfavorable environments. Reallusion Audio2Face integration circumvents these limitations by deploying a highly refined noise filter to eliminate jitters and achieve optimal results despite poor audio quality.

RAISING THE BAR
After obtaining a satisfactory animation from Audio2Face, a finishing touch becomes necessary, particularly when faced with emotional shifts or when emphasizing specific mouth shapes at varying levels of dialogue. iClone empowers facial editing, allowing for refined lip sync, the addition of natural expressions, and the incorporation of head movement sourced from mocap equipment.

The integration of Character Creator, iClone, and Audio2Face marks a significant milestone in AI-driven animation technology, offering creators unprecedented source audio flexibility and efficiency in their production workflows. The Character Creator Auto Setup plugin and iClone plugin are now available as free downloads from Reallusion, empowering creators to streamline their animation pipelines and unleash their creative potential.

To learn more about iClone Audio2Face Plug-in, please visit:

Media Contact

Atilas Wang, Reallusion Inc., +886289121028, [email protected], https://www.reallusion.com/

John Martin II, Reallusion Inc., 6156738814, [email protected], https://www.reallusion.com/

SOURCE Reallusion Inc.