{"slug": "three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways", "title": "Three Phases of AI B-Roll: From Manual Beats to Vision-Authored Cutaways", "summary": "The AI ad platform's b-roll capability evolved through three phases, from manual beat markers to vision-authored cutaways. Phase 1 gave editors manual control, Phase 2 automated timing from reference videos, and Phase 3 generated cutaways with vision-authored prompts. The system uses a BrollBeat data model that supports incremental shipping via feature flags.", "body_md": "B-roll is what makes a UGC testimonial feel edited rather than generated. The presenter talks to camera; cutaways show the product in hand, the label close-up, the lifestyle context. Without b-roll, AI ads read as a single static talking head — technically correct, emotionally flat.\n\nOn the AI ad platform, b-roll capability did not arrive fully formed. It evolved through three deliberate phases inside the tool's scene planner, each solving the limitations of the last. Phase 1 gave editors control. Phase 2 automated timing from references. Phase 3 generated the cutaways themselves with vision-authored prompts. A power-user feature flag gated the progression so we could ship incrementally without breaking production swipe flows.\n\nB-roll is not decoration — it is narrative punctuation. The pipeline had to learn where to punctuate before it could learn what to show.\n\nMain-scene generation and b-roll generation are different problems. Person scenes need identity anchoring, motion transfer, and VO sync. B-roll cutaways need product visibility, pacing that matches the A-roll beat, and — critically — **no competing human faces** that confuse the viewer about who the protagonist is.\n\nEarly swipe iterations treated b-roll as an afterthought: if the reference had cutaways, copy the timestamps; if not, skip them. Script-first UGC mode made that insufficient. Editors writing original scripts still needed cutaways, but had no reference timing to swipe. The three-phase arc was the structured response.\n\n| Phase | Editor action | System responsibility | Requires reference video? |\n|---|---|---|---|\n| Phase 1 | Manually mark b-roll beats on timeline | Insert placeholder cutaway slots | No |\n| Phase 2 | Review auto-placed beats | Extract b-roll timing from reference | Yes |\n| Phase 3 | Brief + approve generated cutaways | Vision-authored prompts + AI generation | No |\n\nPhase 1 shipped inside the tool's scene planner as cutaway beat markers. Editors scrubbed the A-roll timeline and dropped b-roll insertion points — the same mental model as marking ad breaks in a non-linear editor, simplified to click-to-insert.\n\n```\ninterface BrollBeat {\n  id: string;\n  insertAfterSceneId: string;\n  offsetMs: number;           // within-scene offset\n  durationMs: number;         // target cutaway length\n  brief?: string;             // optional editor hint\n  source: 'manual';\n}\n\nfunction insertManualBeat(\n  plan: ScenePlan,\n  afterSceneId: string,\n  offsetMs: number,\n): ScenePlan {\n  const beat: BrollBeat = {\n    id: generateId(),\n    insertAfterSceneId: afterSceneId,\n    offsetMs,\n    durationMs: DEFAULT_BROLL_DURATION_MS,\n    source: 'manual',\n  };\n  return { ...plan, brollBeats: [...plan.brollBeats, beat] };\n}\n```\n\nPhase 1 proved the data model. Every subsequent phase reuses `BrollBeat`\n\n— only the `source`\n\nfield and prompt generation logic change. Beats could be inserted between scenes (not just mid-scene), which mattered for script-first runs where scene boundaries align with script paragraphs rather than reference cuts.\n\nThe limitation was obvious: manual placement scales poorly. A twelve-scene ad with three cutaways each meant thirty-six click decisions per run. Editors wanted the system to propose timing, not just slots.\n\nPhase 2 activated when a reference video existed. The analyzer extracted b-roll segments from the reference — moments where the camera cut away from the presenter to product or lifestyle footage — and the planner mapped those timestamps onto the generated A-roll timeline.\n\nThe \"swipe\" metaphor is literal: b-roll timing swipes from reference to generated plan, adjusted for duration differences between reference VO and generated VO.\n\n```\ninterface ReferenceBrollSegment {\n  startMs: number;\n  endMs: number;\n  classification: 'product' | 'lifestyle' | 'detail';\n}\n\nfunction swipeBrollTiming(\n  referenceSegments: ReferenceBrollSegment[],\n  generatedPlan: ScenePlan,\n  referenceDurationMs: number,\n  generatedDurationMs: number,\n): BrollBeat[] {\n  const scale = generatedDurationMs / referenceDurationMs;\n  return referenceSegments.map(seg => ({\n    id: generateId(),\n    insertAfterSceneId: mapTimestampToScene(seg.startMs * scale, generatedPlan),\n    offsetMs: sceneLocalOffset(seg.startMs * scale, generatedPlan),\n    durationMs: (seg.endMs - seg.startMs) * scale,\n    classification: seg.classification,\n    source: 'reference-swipe',\n  }));\n}\n```\n\nPhase 2 dramatically reduced editor labor on reference-first runs. It did nothing for script-first runs — which is exactly why Phase 3 existed.\n\nPhase 3 is the full solution: the system generates b-roll clips, not just timing slots. The pipeline:\n\n```\nasync function authorBrollPrompt(\n  aRollFrame: Buffer,\n  beat: BrollBeat,\n  product: ProductCatalogEntry,\n  editorBrief?: string,\n): Promise<string> {\n  const visionContext = await visionModel.describe({\n    frame: aRollFrame,\n    focus: ['visible product', 'setting', 'lighting', 'palette'],\n    exclude: ['faces', 'presenter identity'],\n  });\n\n  return promptComposer.compose({\n    template: 'broll-cutaway',\n    visionContext,\n    productLock: product.heroImageUrl,\n    editorBrief,\n    constraints: PEOPLE_FREE_CONSTRAINTS,\n  });\n}\n```\n\nVision-authored prompts solved a subtle quality problem. Text-only b-roll prompts hallucinate context — \"woman in sunny kitchen holding product\" when the A-roll frame shows a bathroom vanity with cool lighting. Grounding the prompt in the actual frame produces cutaways that match the established scene geography.\n\nThe most important constraint in Phase 3 is **people-free b-roll**. If a cutaway shows a human face — even a generic stock-looking person — viewers subconsciously reassign protagonist identity. Scene four's presenter no longer matches scene seven's because the b-roll face became a competing anchor.\n\nPeople-free constraints apply at three layers:\n\n| Constraint layer | Failure mode prevented | Fallback |\n|---|---|---|\n| Prompt negatives | Model generates presenter lookalike in b-roll | Retry with stronger negatives |\n| Vision QA face detection | Subtle partial face in background | Regenerate or product-only fallback |\n| Captioned Drive link validation | Editor-uploaded reference with wrong subject | Block upload, surface error in planner |\n\nThe captioned Google Drive link fix addressed a specific bug: editors pasted Drive URLs to reference b-roll footage, but the link preview showed the wrong thumbnail. We added caption validation and people-free checks on uploaded reference clips before they entered the prompt chain.\n\nAll three phases coexist behind a power flag — `UGC_BROLL_ENABLED`\n\n— with sub-flags for each phase tier. Production reference-first runs stayed on Phase 2 by default. Internal testers and script-first beta users got Phase 3.\n\n```\nfunction resolveBrollPhase(flags: FeatureFlags, run: RunConfig): BrollPhase {\n  if (!flags.UGC_BROLL_ENABLED) return 'disabled';\n  if (run.mode === 'script-first' && flags.UGC_BROLL_PHASE_3) return 'ai-generated';\n  if (run.referenceVideoUrl && flags.UGC_BROLL_PHASE_2) return 'reference-swipe';\n  if (flags.UGC_BROLL_PHASE_1) return 'manual';\n  return 'disabled';\n}\n```\n\nGating also controlled UI surface area. Phase 1 exposed beat markers in the planner. Phase 2 added a \"swipe timing from reference\" button. Phase 3 added cutaway preview tiles, editor brief fields, and insert-between-scenes handles. Shipping all three UI surfaces at once would have overwhelmed editors who only needed manual markers.\n\nA late addition to the gating PR: b-roll beats are not only mid-scene offsets. Editors can insert cutaways *between* scene boundaries — useful when the script has a natural paragraph break that does not map to a mid-scene timestamp. The planner renders these as bridge segments in the stitch timeline, with J-cut audio overlap from the adjacent A-roll.\n\nPhase 3 was not just generation — it required stitch-time and VO pipeline changes:\n\n| Metric | Phase 1 (manual) | Phase 2 (reference swipe) | Phase 3 (AI-generated) |\n|---|---|---|---|\n| Editor time per run (b-roll setup) | 8–12 min | 2–3 min (review only) | 3–5 min (brief + approve) |\n| Works without reference | Yes | No | Yes |\n| Identity confusion incidents | Low (no gen faces) | Low | 6% → <1% after people-free QA |\n| Cutaway-script relevance score (internal QA) | N/A (manual slots) | Moderate | High (vision-grounded) |\n\nI would define the people-free constraint in the shared prompt library from Phase 1, not Phase 3. We retrofitted it after identity confusion reports, but manual-phase editors were already inserting placeholder slots that Phase 3 later filled with face-containing clips.\n\nI would also expose phase tier in the run manifest for debugging. Support tickets saying \"my b-roll looks wrong\" required log diving to discover whether the run used manual beats, reference swipe, or AI generation.\n\nThe three-phase b-roll arc is a reusable template for complex generative features:\n\nEach phase reuses the same `BrollBeat`\n\nstructure. Feature flags gate progression. Domain constraints — people-free b-roll, product lock, captioned link validation — apply regardless of phase. That separation let us ship Phase 1 to all editors while Phase 3 baked in beta, without forking the stitch pipeline.", "url": "https://wpnews.pro/news/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways", "canonical_source": "https://dev.to/humzakt/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways-539i", "published_at": "2026-08-25 21:04:28+00:00", "updated_at": "2026-08-25 21:44:27.927713+00:00", "lang": "en", "topics": ["generative-ai", "ai-products", "ai-tools", "computer-vision"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways", "markdown": "https://wpnews.pro/news/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways.md", "text": "https://wpnews.pro/news/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways.txt", "jsonld": "https://wpnews.pro/news/three-phases-of-ai-b-roll-from-manual-beats-to-vision-authored-cutaways.jsonld"}}