Why the track sets the edit
A song arrives with a frame already built: intro, verse, chorus, sometimes a bridge and a final chorus. Each part has its own density of events, and the shot gets chosen for that density before anything is generated. Do it the other way, from shots you liked, and the work turns into fitting: a four second scene will not sit inside a two second bar, and the cut shows a jolt.
My order is fixed. The beat grid first, with phrase boundaries every eight bars. Then meaning: what happens in the verse, what in the chorus, where the peak sits. Shots come last, tied to phrases. That plan costs an evening and saves a week.
The grid also shows something easy to miss. Choruses repeat, and the second pass needs variation. The same shot twice reads as a mistake. The variation can be cheap: another angle, another light, a tighter frame.
What the model does with a voice
A lip sync model takes video and audio, then redraws the mouth to match the track.
The sync. labs documentation says lipsync-2 and lipsync-2-pro process video in independent two second chunks and need natural speaking motion in the source. On still frames lip sync does not fire, audio or not. sync-3 works differently: it builds a global understanding of the person across the whole shot and generates every frame at once, so there are no chunk seams. The same page lists face resolution at 512 by 512 for lipsync-2 and lipsync-2-pro, and native 4K on sync-3. Obstructions, profile shots and partly hidden faces are handled by sync-3, while the smaller models struggle there. A temperature setting controls how expressive the mouth movement gets, defaults to 0.5 and exists on the two smaller models.
The practical rule: lip sync needs footage where the person is actually talking. A frozen face will not come alive under a vocal.
The beat grid, in practice
I split the track into eight bar phrases and place two or three scenes in each. At 120 beats per minute a bar lasts two seconds and a phrase sixteen. A scene then runs five to eight seconds, which is a comfortable length for generation: one event per shot.
Next I hand out the jobs. Verses carry story: place, character, movement. Choruses hold the face and the rhythm, because that is where the viewer starts mouthing the words. The bridge can afford a pause and a wide shot, since attention sags there anyway.
I mark the frames that will carry text separately. If a lyric line appears on screen, it needs room, and the lower third is the safe place. Text that jumps around the frame pulls attention away from the mouth.
Where lip sync holds
Lip sync goes on the chorus and on two or three close phrases where the character looks into the lens. The rest is carried by movement, light and cuts.
That is cheaper and more honest. Synchronising a whole song looks mechanical: the mouth moves, the eyes live somewhere else. On the lighter models each chunk is two seconds long, so seams between chunks get more visible over a long run.
My list of shots to avoid for lip sync is short. Profile, a hand near the mouth, a microphone in frame, heavy head movement. The smaller models break on all four, and a redo costs more than a different angle on set.
Music rights come before the edit
The most expensive mistake in a music video happens before the first frame. Music used without permission arrives as a claim.
YouTube's help centre describes Content ID like this: an upload is scanned automatically against a database of audio and visual files supplied by copyright owners. A match produces a claim, and the owner then picks one of three actions: block the video from being viewed, run ads against it and sometimes share revenue with the uploader, or track its viewership. Any of those actions can apply country by country.
For a video this means the rights check happens before the storyboard. A track that cannot be used in paid placement will not be rescued by animation.
Counting shots for three minutes
The arithmetic is simple. Three minutes at 120 beats per minute is 180 seconds, which is 90 bars and roughly 11 phrases of eight bars. Two or three scenes per phrase gives 25 to 30 scenes. That is a few days of generation and one long evening in the edit.
Fewer than twenty scenes and the video starts to drag, with one image held too long. More than forty and nobody reads a single shot, so the piece feels like a highlight reel. The rule from advertising holds: one scene, one thought.
An AI music video is cheaper to rebuild than to shoot, which is why the scene count matters more than the budget line. A single corrected shot does not pull a crew day behind it. If money is the question, the music video budget breakdown walks through what a shoot actually costs.
Vertical version and captions
A song usually lives in two shapes: landscape for a player, vertical for a feed. I build both compositions, because reframing 16:9 into 9:16 crops the edges and kills a wide shot.
Vertical changes the framing. A person fills more of the frame, the background blurs, and movement between cuts reads faster. Short-form video at scale works on the same principle: the first second decides whether anyone stays.
Lyrics belong in a caption file. YouTube's help centre accepts .srt and .sbv in plain UTF-8, and those files carry timing and text without styling. For lyrics that is cheaper than burning type into the picture.
The takeaway
An AI music video begins with a beat grid and a decision about which phrases the character actually sings. Lip sync on the lighter models runs in two second chunks, rights get checked before the first scene, and the scene count comes from the length of the track.
None of this makes a video brilliant. It makes the work predictable, which is what most commissioned pieces lack.
