{"slug": "pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time", "title": "Pika Enact Lets Directors Drag, Speak and Steer AI Video in Real Time", "summary": "Pika unveiled Enact, a research preview that lets users direct AI-generated video live while it plays, using three modes: dragging actors and objects across the frame, giving spoken or typed performance notes, and steering a virtual camera. Pika reports Enact streams audiovisual chunks shorter than one second at 1.8x real time for Live Staging (under 0.4 seconds of response latency on one Nvidia H100), 1.75x real time for Live Actor Direction, and 1.50x real time at 640 x 480 and 24 fps for Live Camera Control (0.51 seconds to first frames on a node with eight Nvidia B300 accelerators). Enact is built on Pika H3 Flex, uses self-forcing distillation plus a sliding memory window to run sessions indefinitely at fixed cost, and is not publicly available or priced; Pika says image quality is lower than its offline version, and none of the performance figures have been independently verified.", "body_md": "# Pika Enact Lets Directors Drag, Speak and Steer AI Video in Real Time\n\nPika's Enact research preview streams AI video and sound in under-a-second chunks, so you can drag actors, talk to them and move the camera while the scene keeps playing.\n\n- Pika unveiled [Enact](https://pika.art/blog/pika-enact?utm_source=alphasignal) , a research preview for directing AI video scenes live while they play.\n- It has three modes: drag actors with a mouse, give actors spoken notes, and steer the camera.\n- Built on [Pika H3 Flex](https://pika.art/pika-h3-flex?utm_source=alphasignal) and streamed in sub-second chunks at 1.5x to 1.8x real-time speed.\n- Uses self-forcing distillation plus a sliding memory window so a session can run indefinitely at fixed cost.\n- Training data comes from simulation: a physics engine and Blender pipelines generate drag and camera-path pairs.\n- Not publicly available or priced; demos via enterprise@pika.art; image quality is lower than Pika's offline version.\n\n[Pika Enact](https://pika.art/blog/pika-enact?utm_source=alphasignal) is a research preview for directing generated video while it plays. A user can drag actors and objects across the frame, issue spoken performance notes, and steer a virtual camera as the system generates synchronized video and audio. The interface replaces the familiar prompt, render, review, and retry cycle with continuous control.\n\nEnact uses [Pika H3 Flex](https://pika.art/pika-h3-flex?utm_source=alphasignal) to generate audiovisual chunks shorter than one second. Each chunk advances the scene, incorporates the latest direction, and carries enough recent context to preserve continuity. All performance figures and quality claims in the preview come from Pika and have not been independently verified.\n\n## Three controls, one continuous scene\n\nEnact exposes three interaction modes covering scene layout, actor behavior, and camera movement:\n\n- **Live Staging:** The user drags a person or object to a precise location while the video runs. The model redraws the subject with the surrounding scene, including pose, lighting, shadows, and the vacated background. Pika says this joint redraw prevents cutout edges and empty regions where the subject previously stood.\n- **Live Actor Direction:** Typed or spoken notes such as “move closer to her” or “make it a kitchen at night” guide AI-controlled actors. They respond in character, retain earlier instructions, and use distinct voices. Calling “Action” converts the rehearsal into a newly generated continuous take.\n- **Live Camera Control:** The user can pan, dolly, or orbit while the scene plays. The mode generates video and audio from an appearance image for the cast and set, plus a rough 3D reference such as an untextured Blender viewport that defines camera motion.\n\n## Subsecond direction, datacenter hardware\n\nPika reports faster-than-playback generation for all three modes, although the company uses different latency measures for staging and camera control.\n\n| Pika-reported Enact performance |  |  | \n|---|---|---|\n| Mode | Generation rate | Latency and hardware | \n|---|---|---|\n| Live Staging | 1.8× real time | Under 0.4 seconds of response latency on one Nvidia H100, including decoding in low-latency mode | \n| Live Actor Direction | 1.75× real time for video and audio | Not disclosed | \n| Live Camera Control | 1.50× real time at 640 × 480 and 24 fps | 0.51 seconds to the first frames on a node with eight Nvidia B300 accelerators | \n\nThe measurements are not directly equivalent. Staging reports response latency, camera control reports time to first frames, and actor direction has no published latency figure. Pika also has not disclosed the staging or actor-direction resolutions, benchmark methodology, concurrent-session capacity, or end-to-end network overhead.\n\nCamera control generates each 0.708-second chunk in 0.473 seconds, leaving limited compute headroom for sustained playback. Pika reaches that rate by caching static appearance inputs across chunks and using a streaming decoder that converts the model’s latent output into frames as generation proceeds.\n\n## A clip model learns to keep rolling\n\nH3 Flex is Pika’s post-trained version of MiniMax H3, adapted after the base model’s initial training. It can generate synchronized video and stereo audio for clips of up to 15 seconds using six denoising steps, with each step serving as an iterative refinement pass. Enact modifies that fixed-length model to support a stateful stream.\n\nThe streaming design combines three techniques:\n\n1. **Causal chunking:** The model attends only to earlier frames, allowing it to emit 0.7-second chunks by default or 0.17-second chunks in low-latency mode without access to future video.\n2. **A bounded memory window:** A key-value cache retains the prompt, source video, initial chunk, and four most recent chunks. Older generated chunks leave the cache, keeping per-chunk memory and compute bounded as the session continues.\n3. **Self-forcing distribution matching distillation:** A fast student model learns to approximate the output distribution of a slower teacher, reducing roughly 20 sampling steps to a small number. During training, the student consumes its own earlier outputs. That exposure teaches it to correct small generation errors before they accumulate across a long take.\n\nLive Staging conditions the model with three frame-aligned inputs: mouse state sampled at 24 fps, a heatmap marking the grabbed location, and the original source video used to preserve the background. Small conditioning modules inject these signals into the base model. Their weights begin at zero, allowing training to add cursor control gradually without immediately disrupting the pretrained behavior.\n\nLive Actor Direction adds a script language model above the audiovisual generator. It reads the latest instruction, opening frame, and session memory, then uses tool calls to produce structured beats for action, dialogue, emotion, gaze, and lighting. Idle beats make actors breathe and blink when no instruction is pending. Hidden hold beats counter the tendency of the opening frame to pull a changing scene back toward its initial composition.\n\n## Simulation supplies the missing training pairs\n\nSuitable real footage rarely contains the control data needed for Live Staging. Training requires paired videos with the same scene and timing, while one version moves a known subject along a recorded cursor path. Pika built a physics-based synthetic data engine to create those pairs.\n\nThe pipeline animates a 3D stand-in along a drag path with simulated weight and momentum, renders a photorealistic result with Pika’s video models, and records every mouse position and button state. The process produces aligned examples without manual labels. It also teaches dragged figures to swing, shift their weight, and settle when released.\n\nCamera control uses a Blender pipeline that records camera paths, untextured motion references, final frames, and spatial audio together. Automated checks reject clipping errors and frames where a character’s identity changes. Simulation gives the model synchronized cursor and camera signals that would be costly and difficult to capture from physical productions.\n\n## Enterprise access, open questions\n\nEnact remains a private research preview with no consumer release date or announced price. Pika directs prospective customers to request a demonstration or discuss an integration through [enterprise@pika.art](mailto:enterprise@pika.art). The company has not announced a public API, SDK, deployment package, or access queue.\n\n| Current availability |  |  | \n|---|---|---|\n| Component | Availability | Published terms | \n|---|---|---|\n| Pika Enact | Enterprise demonstrations and integration discussions | No public pricing or API | \n| H3 Flex weights | Published through Pika’s Hugging Face presence | See the linked model page for licensing details | \n| Hosted H3 Flex | Available from Pika | Starts at $0.01 per output second | \n\nPika has not said that Enact’s real-time control models, synthetic-data pipeline, orchestration layer, or streaming implementation will be released with the H3 Flex weights. Developers also lack published limits for session length, concurrency, staging resolution, actor-direction resolution, input formats, and commercial deployment.\n\nCurrent limitations include lower image quality than Pika’s slower generation system, a published camera-control resolution of 640 × 480, and substantial GPU requirements for the fastest results. The announcement includes no third-party benchmarks, user studies, failure-rate analysis, or comparisons with other interactive video systems.\n\n## Best fit: previsualization and live prototypes\n\nEnact’s present speed, resolution, and access model point toward workflows where immediate control matters more than final-render fidelity:\n\n- Blocking and previsualization for testing positions, timing, and composition\n- Dialogue and performance rehearsals before a higher-quality render\n- Live camera passes over rough 3D layouts without finished lighting or textures\n- Interactive media, games, installations, and demonstrations driven by audience input\n\nAn Enact-style product would require a persistent, bidirectional session that synchronizes controls, generated frames, audio, and model state. That architecture changes the developer workload: latency budgets, streaming transport, session recovery, GPU scheduling, and control synchronization become core product concerns.\n\nFor video-model researchers, Enact provides a concrete recipe for interactive generation: causal chunks, bounded memory, distilled sampling, structured agent instructions, and synthetic control data. Pika has demonstrated those pieces in a continuous audiovisual system, while public access, production economics, image quality, and independent evaluation remain unresolved.", "url": "https://wpnews.pro/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time", "canonical_source": "https://alphasignal.ai/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time", "published_at": "2026-11-18 18:53:56+00:00", "updated_at": "2026-11-18 21:45:47.021477+00:00", "lang": "en", "topics": ["generative-ai", "artificial-intelligence", "ai-products"], "entities": ["Pika", "Enact", "Pika H3 Flex", "Nvidia H100", "Nvidia B300", "Blender"], "also_reported_by": [], "takeaway": "Pika's Enact streams sub-second audiovisual chunks at 1.5x to 1.8x real time by combining self-forcing distillation with a sliding memory window, letting a live directing session run indefinitely at fixed cost.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time", "markdown": "https://wpnews.pro/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time.md", "text": "https://wpnews.pro/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time.txt", "jsonld": "https://wpnews.pro/news/pika-enact-lets-directors-drag-speak-and-steer-ai-video-in-real-time.jsonld"}}