Kyutai AURA-1.3B Streams Talking-Head Video in Real Time

Kyutai AURA-1.3B talking-head video model animating a portrait from speech audio

Kyutai has released Kyutai AURA-1.3B, a model that turns a single portrait image and a stream of speech into a talking-head video, one chunk at a time. It is the only member of the AURA family that streams in real time, at 576p. In the research paper, it reaches 20.7 frames per second end to end on a single H100 GPU for 16 Hz playback.

That is 23 times faster than the teacher model it was distilled from. The weights are public, but only for non-commercial use.

How the model works

AURA stands for AUdio-dRiven streaming Avatar. The 1.3B model generates video chunk by chunk and keeps a rolling KV cache, so it can start producing frames before the audio ends.

The released model is a causal student that denoises in 4 steps. It was distilled from a bidirectional 1.3B teacher named AURA-1.3B-Teacher-TalkVerse, which listens to Mimi audio tokens and was itself built by adapting the Wan2.1-T2V-1.3B video model.

The research team comes from Kyutai, Sorbonne Université’s ISIR lab and Google DeepMind. The paper is headed to the British Machine Vision Conference in 2026.

Two-stage training

The larger AURA-5B uses the identical recipe. Each stage adjusts small LoRA adapters (rank 128, alpha 128) and a handful of other modules, chiefly the ones that process Mimi audio.

  • Stage 1 teaches the student to imitate the teacher. It minimizes an MSE loss over 30,000 trajectories the teacher produced for 80-frame clips driven by Mimi audio. Kyutai includes those trajectories in the AURA dataset.
  • Stage 2 applies Distribution Matching Distillation with backward simulation while the audio LoRA stays frozen. The learning rate is 2e-7, and Kyutai publishes the EMA-averaged weights.

Every checkpoint comes in two forms. The lora_only file keeps just the training updates, meant to be layered over the base video model, the teacher or an earlier stage. The merged file folds those updates in and runs by itself.

Running it yourself

The reference code is on GitHub. After cloning the repo and syncing dependencies with uv, a single command takes a portrait image and a speech file and writes an MP4. AURA-1.3B is the default model.

On first run the script downloads the merged stage-2 weights, Mimi for audio features, and the VAE and T5 text encoder that belong to Wan2.1-T2V-1.3B. Each chunk takes 4 sampling steps (timesteps 1000, 745, 498 and 255) and skips classifier-free guidance. Output runs at 16 fps in either 576×576 or 768×512. Options include a text prompt, chunk count, seed and compilation.

With compilation on, the reference code manages about 16 fps on one H100, which matches real-time playback at that frame rate. The 20.7 fps figure in the paper comes from an optimized pipeline, so expect a gap between the two.

Results and limits

On CSIM, the identity-preservation metric, AURA-1.3B beat every method it was compared with on both test sets. The paper has the full tables.

The license is the main restriction. The weights use CC BY-NC 4.0, which allows non-commercial use only. That follows the training data. The student learned from teacher outputs that depended on reference images and prompts taken from Hallo3, which carries CC BY-NC-ND 4.0. The downloaded VAE and T5 encoder keep their own Apache 2.0 license.

Kyutai also lists the other checkpoints, including the stage 1 files and the lora_only versions, as research releases. The reference code does not load them.

Consent and disclosure

The model produces realistic video of real-looking people speaking arbitrary audio. The developers ask users not to impersonate anyone, create non-consensual content or spread disinformation. They say to animate only portraits you have consent to use, and to disclose that the videos are AI-generated.

Real-time speed raises the stakes. A model that keeps pace with live audio can in principle drive a video call, which makes those rules matter more than they would for an offline tool.

What to watch next

The larger AURA-5B shares the recipe but does not stream in real time. Researchers can build on the released trajectories and checkpoints, while commercial teams will need to wait for different licensing terms.

Source: huggingface.co

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.