Unity > Timeline > Matching Video Performance to Animation Timings w/ Frame Accuracy!

Spring is in the air and new workflows are hatching 🐣! Subscribe for future workflows.

Here’s how to match your prerecorded video performances to event and animation timings within your virtual worlds using Timeline within a Virtual Production pipeline.

PRODUCTION PHASE

We cannot build that which we cannot see. Let’s enable frame accurate video performance previews in Unity using Timeline so we can visualize our work with precision!

1. Browser > Unity Asset Store > Free Default Playables Asset > Add To Assets
2. Unity > Package Manager > Search ‘Default Playables’ > Install
3. Timeline > Add Track > Video Script Playable Track
4. Video Script Playable Track > Add Video Script Playable Asset
5. Inspector > Assign Video Player
6. Inspector > Assign Video Clip

You now have frame accurate video performance previews from Timeline.

Scrub slowly. Fast scrubbing won’t work and isn’t available.

Animate your game objects and VFX to desired effect.

EXPORT PHASE

• Timeline > Video Script Playable Track > Video Script Playable Asset > Inspector > Unassign Video Clip
• Recorder > Export

Notes:

Assigning both a Video Player and a Video Clip to your Video Script Playable Asset is required for frame accurate working previews (if you’re missing one of the two – no previews for you).

And at the same time…

Failure to clear the assigned Video Clip from the Video Script Playable Asset will prevent reliable video playback during export (your video clip will invariably freeze at some inopportune moment during playback whether you are exporting or merely entering Play Mode)…

These are the things I wish anyone had cared to document publicly or cared to teach me personally.

Oh, the hours I’ve lost to the churn of fumbling around In the dark while attempting to unearth a tenable workflow such as I’ve outlined here today, for myself and for you.

Many “gotcha-gremlins” lie in wait to steal our time and sanity when it comes to integrating live action performance with virtual environments believably – together I believe we can clear the path for future storytellers who have yet to arrive and who will have meaningful stories to tell.

What stories will you tell using Unity for Virtual Production?