Three Phases of AI B-Roll: From Manual Beats to Vision-Authored Cutaways

The AI ad platform's b-roll capability evolved through three phases, from manual beat markers to vision-authored cutaways. Phase 1 gave editors manual control, Phase 2 automated timing from reference videos, and Phase 3 generated cutaways with vision-authored prompts. The system uses a BrollBeat data model that supports incremental shipping via feature flags.

B-roll is what makes a UGC testimonial feel edited rather than generated. The presenter talks to camera; cutaways show the product in hand, the label close-up, the lifestyle context. Without b-roll, AI ads read as a single static talking head — technically correct, emotionally flat. On the AI ad platform, b-roll capability did not arrive fully formed. It evolved through three deliberate phases inside the tool's scene planner, each solving the limitations of the last. Phase 1 gave editors control. Phase 2 automated timing from references. Phase 3 generated the cutaways themselves with vision-authored prompts. A power-user feature flag gated the progression so we could ship incrementally without breaking production swipe flows. B-roll is not decoration — it is narrative punctuation. The pipeline had to learn where to punctuate before it could learn what to show. Main-scene generation and b-roll generation are different problems. Person scenes need identity anchoring, motion transfer, and VO sync. B-roll cutaways need product visibility, pacing that matches the A-roll beat, and — critically — no competing human faces that confuse the viewer about who the protagonist is. Early swipe iterations treated b-roll as an afterthought: if the reference had cutaways, copy the timestamps; if not, skip them. Script-first UGC mode made that insufficient. Editors writing original scripts still needed cutaways, but had no reference timing to swipe. The three-phase arc was the structured response. | Phase | Editor action | System responsibility | Requires reference video? | |---|---|---|---| | Phase 1 | Manually mark b-roll beats on timeline | Insert placeholder cutaway slots | No | | Phase 2 | Review auto-placed beats | Extract b-roll timing from reference | Yes | | Phase 3 | Brief + approve generated cutaways | Vision-authored prompts + AI generation | No | Phase 1 shipped inside the tool's scene planner as cutaway beat markers. Editors scrubbed the A-roll timeline and dropped b-roll insertion points — the same mental model as marking ad breaks in a non-linear editor, simplified to click-to-insert. interface BrollBeat { id: string; insertAfterSceneId: string; offsetMs: number; // within-scene offset durationMs: number; // target cutaway length brief?: string; // optional editor hint source: 'manual'; } function insertManualBeat plan: ScenePlan, afterSceneId: string, offsetMs: number, : ScenePlan { const beat: BrollBeat = { id: generateId , insertAfterSceneId: afterSceneId, offsetMs, durationMs: DEFAULT BROLL DURATION MS, source: 'manual', }; return { ...plan, brollBeats: ...plan.brollBeats, beat }; } Phase 1 proved the data model. Every subsequent phase reuses BrollBeat — only the source field and prompt generation logic change. Beats could be inserted between scenes not just mid-scene , which mattered for script-first runs where scene boundaries align with script paragraphs rather than reference cuts. The limitation was obvious: manual placement scales poorly. A twelve-scene ad with three cutaways each meant thirty-six click decisions per run. Editors wanted the system to propose timing, not just slots. Phase 2 activated when a reference video existed. The analyzer extracted b-roll segments from the reference — moments where the camera cut away from the presenter to product or lifestyle footage — and the planner mapped those timestamps onto the generated A-roll timeline. The "swipe" metaphor is literal: b-roll timing swipes from reference to generated plan, adjusted for duration differences between reference VO and generated VO. interface ReferenceBrollSegment { startMs: number; endMs: number; classification: 'product' | 'lifestyle' | 'detail'; } function swipeBrollTiming referenceSegments: ReferenceBrollSegment , generatedPlan: ScenePlan, referenceDurationMs: number, generatedDurationMs: number, : BrollBeat { const scale = generatedDurationMs / referenceDurationMs; return referenceSegments.map seg = { id: generateId , insertAfterSceneId: mapTimestampToScene seg.startMs scale, generatedPlan , offsetMs: sceneLocalOffset seg.startMs scale, generatedPlan , durationMs: seg.endMs - seg.startMs scale, classification: seg.classification, source: 'reference-swipe', } ; } Phase 2 dramatically reduced editor labor on reference-first runs. It did nothing for script-first runs — which is exactly why Phase 3 existed. Phase 3 is the full solution: the system generates b-roll clips, not just timing slots. The pipeline: async function authorBrollPrompt aRollFrame: Buffer, beat: BrollBeat, product: ProductCatalogEntry, editorBrief?: string, : Promise<string { const visionContext = await visionModel.describe { frame: aRollFrame, focus: 'visible product', 'setting', 'lighting', 'palette' , exclude: 'faces', 'presenter identity' , } ; return promptComposer.compose { template: 'broll-cutaway', visionContext, productLock: product.heroImageUrl, editorBrief, constraints: PEOPLE FREE CONSTRAINTS, } ; } Vision-authored prompts solved a subtle quality problem. Text-only b-roll prompts hallucinate context — "woman in sunny kitchen holding product" when the A-roll frame shows a bathroom vanity with cool lighting. Grounding the prompt in the actual frame produces cutaways that match the established scene geography. The most important constraint in Phase 3 is people-free b-roll . If a cutaway shows a human face — even a generic stock-looking person — viewers subconsciously reassign protagonist identity. Scene four's presenter no longer matches scene seven's because the b-roll face became a competing anchor. People-free constraints apply at three layers: | Constraint layer | Failure mode prevented | Fallback | |---|---|---| | Prompt negatives | Model generates presenter lookalike in b-roll | Retry with stronger negatives | | Vision QA face detection | Subtle partial face in background | Regenerate or product-only fallback | | Captioned Drive link validation | Editor-uploaded reference with wrong subject | Block upload, surface error in planner | The captioned Google Drive link fix addressed a specific bug: editors pasted Drive URLs to reference b-roll footage, but the link preview showed the wrong thumbnail. We added caption validation and people-free checks on uploaded reference clips before they entered the prompt chain. All three phases coexist behind a power flag — UGC BROLL ENABLED — with sub-flags for each phase tier. Production reference-first runs stayed on Phase 2 by default. Internal testers and script-first beta users got Phase 3. function resolveBrollPhase flags: FeatureFlags, run: RunConfig : BrollPhase { if flags.UGC BROLL ENABLED return 'disabled'; if run.mode === 'script-first' && flags.UGC BROLL PHASE 3 return 'ai-generated'; if run.referenceVideoUrl && flags.UGC BROLL PHASE 2 return 'reference-swipe'; if flags.UGC BROLL PHASE 1 return 'manual'; return 'disabled'; } Gating also controlled UI surface area. Phase 1 exposed beat markers in the planner. Phase 2 added a "swipe timing from reference" button. Phase 3 added cutaway preview tiles, editor brief fields, and insert-between-scenes handles. Shipping all three UI surfaces at once would have overwhelmed editors who only needed manual markers. A late addition to the gating PR: b-roll beats are not only mid-scene offsets. Editors can insert cutaways between scene boundaries — useful when the script has a natural paragraph break that does not map to a mid-scene timestamp. The planner renders these as bridge segments in the stitch timeline, with J-cut audio overlap from the adjacent A-roll. Phase 3 was not just generation — it required stitch-time and VO pipeline changes: | Metric | Phase 1 manual | Phase 2 reference swipe | Phase 3 AI-generated | |---|---|---|---| | Editor time per run b-roll setup | 8–12 min | 2–3 min review only | 3–5 min brief + approve | | Works without reference | Yes | No | Yes | | Identity confusion incidents | Low no gen faces | Low | 6% → <1% after people-free QA | | Cutaway-script relevance score internal QA | N/A manual slots | Moderate | High vision-grounded | I would define the people-free constraint in the shared prompt library from Phase 1, not Phase 3. We retrofitted it after identity confusion reports, but manual-phase editors were already inserting placeholder slots that Phase 3 later filled with face-containing clips. I would also expose phase tier in the run manifest for debugging. Support tickets saying "my b-roll looks wrong" required log diving to discover whether the run used manual beats, reference swipe, or AI generation. The three-phase b-roll arc is a reusable template for complex generative features: Each phase reuses the same BrollBeat structure. Feature flags gate progression. Domain constraints — people-free b-roll, product lock, captioned link validation — apply regardless of phase. That separation let us ship Phase 1 to all editors while Phase 3 baked in beta, without forking the stitch pipeline.