Expression Transfer

Expression transfer maps your real facial movements onto a digital avatar in real time, making the character mirror your emotions, speech, and micro-expressions faithfully.

Blend Shapes (Morph Targets)

Avatars express emotions through blend shapes — predefined facial deformations that can be combined:

• ARKit standard: 52 blend shapes covering all major facial movements
• Visemes: Mouth shapes for speech (typically 15 phoneme groups)
• Custom shapes: Additional expressions specific to your avatar style

The Transfer Pipeline

1. Capture

   Camera captures your face at 30-60 fps with AI landmark detection running on each frame.

2. Extract Parameters

   AI converts landmarks into blend shape coefficients (0.0-1.0 for each shape).

3. Calibrate

   Map your neutral face to the avatar's neutral state; account for individual facial differences.

4. Apply

   Set blend shape weights on the avatar mesh each frame for smooth, real-time animation.

Calibration Techniques

• Auto-calibration: System detects your neutral face and expression range automatically
• Manual calibration: User performs specific expressions (smile, frown, surprise) to set ranges
• Per-user profiles: Save calibration data for consistent results across sessions
• Range scaling: Amplify subtle expressions for more expressive avatars (common in VTubing)

Common Challenges