{"slug": "ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol", "title": "LTX-2.5-22B-IC-LoRA AI video studio with codex application protocol", "summary": "AIVS, a desktop application from developer andrewarrow, converts a human-readable screenplay into an AI-generated video with speech by routing the text through Codex's app-server protocol to produce a timed shooting script, then rendering each shot's video and audio locally with LTX-2.5 on an Apple Silicon Mac via MLX. In a saved example run dated 2026-10-26, Codex divided a six-beat desert-road scene into two shots totaling 21 seconds (10 seconds and 11 seconds), and Python validated that the plan covered the source beats in order, preserved dialogue, and fit duration and model token limits before FFmpeg assembled the output. The app's Rust desktop front end and Python video pipeline keep inference and speech verification local, while Codex planning uses the user's configured Codex service and login.", "body_md": "## small.mov\n\nAIVS turns a human-readable screenplay into an AI-generated video with speech.\nWrite a scene, cast its characters, and choose **Generate video**. The app uses\n**Codex through its app-server protocol to turn the screenplay into a shooting\nscript**, then uses **LTX-2.5 to render each shot's video and audio locally** on\nan Apple Silicon Mac.\n\n```\nScreenplay + cast\n       ↓\nCodex app-server → shooting script (timing, staging, dialogue, camera, sound)\n       ↓\nPython validates the plan → LTX-2.5 renders connected shots\n       ↓\nSpeech and video checks → FFmpeg assembly → video.mp4\n```\n\nThe desktop app is Rust; the video pipeline is Python using MLX on Apple's GPU. Codex planning uses your configured Codex service and login. Video inference and speech verification run locally.\n\nThese files come from run `2026-10-26_04-12-06-092171_1035559519`.\nCopies live in the repository so you can read them without a local render:\n[screenplay.txt](https://github.com/andrewarrow/aivs/blob/main/docs/examples/desert-road/screenplay.txt) →\n[shooting-script.txt](https://github.com/andrewarrow/aivs/blob/main/docs/examples/desert-road/shooting-script.txt).\n\nThe input describes the story in terms a writer understands: a setting, actions, characters, and dialogue. Here is the original text:\n\n```\nINT. DESERT ROAD - DUSK\n\nA two-lane road and cuts through red rock. An old station wagon sits on the shoulder, its hood raised.\n\nSUE, 30s, turns a battered postcard over in her hands. JOE, late 20s, emerges from behind the engine.\n\nJOE: \"Good news. It's definitely the car.\"\n\nSue almost smiles. She looks down the empty road.\n\nSUE: \"My dad said you could see the ocean from here.\"\n\nJOE (looking around): \"He was an optimistic man.\"\n\nFADE OUT.\n```\n\nThe app sends the screenplay, cast descriptions, ordered story beats, and\n[director instructions](https://github.com/andrewarrow/aivs/blob/main/python/shooting-script-instructions.txt) to Codex.\nA **beat** is one source action or dialogue line. Codex divides those beats into\ntimed shots and specifies where people stand, what they do, how the camera sees\nthem, and what should be heard.\n\nIn this example, it produces **two shots totaling 21 seconds**:\n\n| Shot | Duration | Source beats | Planned action | \n|---|---|---|---|\n| 1 | 10 seconds | 1–4 | Establish the wagon and postcard; Joe steps into view and delivers his first line; Sue almost smiles and looks down the road. | \n| 2 | 11 seconds | 5–6 | Sue delivers her line about the ocean; Joe looks around and replies; the scene ends on a quiet reaction. | \n\nAn excerpt from the actual first shot shows the added physical detail:\n\nSUE stands on the gravel screen left beside the front passenger door, holding a battered postcard in both hands. JOE stands on the ground screen right alongside the engine bay, empty hands clear of the engine, upper body partly screened by the raised hood. His feet and route along the near fender remain visible.\n\nThe plan also directs sound and the end of the action:\n\nFinish on this quiet reaction with environmental sound only: dry wind, faint cooling-engine ticks, and gravel crunch synchronized with JOE's step.\n\nThe [full shooting script](https://github.com/andrewarrow/aivs/blob/main/docs/examples/desert-road/shooting-script.txt) includes\nexact dialogue, appearance and voice descriptions, and start/end continuity notes\nfor each shot. This translation is the core of AIVS: turning a short human scene\ninto explicit, timed instructions for a video model.\n\nThis saved example is a historical planning artifact, not a guarantee of visual quality or a template for every current rule. For example, its second shot calls for a tighter framing; the current renderer carries forward the previous frame, and current director instructions require reframing to develop through camera movement within a scene.\n\nPython checks that the plan covers the source beats in order, preserves dialogue,\nand fits duration and model token limits. It fills dialogue placeholders with the\nsource lines and saves both `shooting-script.json` for the renderer and\n`shooting-script.txt` for people to inspect. Invalid plans get one correction\nattempt through Codex before the run fails.\n\nLTX-2.5 generates video and speech together in **4–12 second shots at 1280×704,\n24 fps**. Within a scene, each shot after the first starts from the previous\nshot's final decoded frame to carry forward faces, clothing, positions, and props.\nA new scene resets that frame reference. The model receives each shot's complete\nprompt; the separate start/end notes support continuity review.\n\nWhisper transcribes each take locally and the app compares the recognized words\nwith the source dialogue. FFmpeg assembles the shots and applies the final fade\nwhen requested. The assembled result must pass speech, decoded-frame, and\naudio/video timing checks before it is published as `video.mp4`.\n\n- An **Apple Silicon Mac with Metal support** and**Rust 1.92+** .\n- **Homebrew** , so setup can install`uv` , FFmpeg, and`whisper-cpp` if missing.\n- A **Hugging Face access token** for downloading models.\n- The **Codex CLI** , signed in and available on`PATH` (or set`AIVS_CODEX_PATH` ).\nCodex is required for screenplay planning; the chat panel is optional.\n- **50 GiB of free disk space** for first setup. Video model downloads are about**43 GB** ; speech verification also caches a roughly**488 MB** Whisper model.\n\nFrom this checkout:\n\n1. \nCreate `.env` if it does not already exist, using[.env.example](https://github.com/andrewarrow/aivs/blob/main/.env.example) as a template. Set`HF_API` to your Hugging Face token.\n2. \nStart the app: \n\n```\ncargo run\n```\n\n3. \nLet setup prepare Python 3.12 in `.venv` , install the pinned packages, download\nmodels, and verify the GPU. Downloads reuse the cache. Use**Check again** after adding a token, or**Retry setup** after fixing a setup error.\n4. \nIf needed, choose **Codex → Sign in with ChatGPT** and finish in your browser.\nAn existing Codex login is reused.\n5. \nEdit the screenplay and cast every role. Drag a gallery character onto a role, or use its **Uncast / played by** control. For the sample, try**Ava as Sue** and**Eli as Joe** . Casting keeps the script's role names and dialogue intact.\n6. \nChoose **Generate video** .**Activity** shows planning and rendering progress.\nWhen complete, play the result in the app or choose**Show files** .\n\n**Stop** interrupts the render and its child processes; closing AIVS also stops\nactive work. Partial outputs remain available for inspection.\n\nBy default, the current directory is the workspace. To keep scripts, caches, and outputs elsewhere, pass an existing directory:\n\n```\ncargo run -- /path/to/workspace\n# To select a specific Codex executable:\nAIVS_CODEX_PATH=/absolute/path/to/codex cargo run\n```\n\nPut `.env` in the selected workspace. Keep this checkout's `python/` directory\nalongside the source when building; it contains the runtime pipeline.\n\nScripts and cast assignments autosave to `screenplay.json` in the workspace.\n**⌘/Ctrl+S** saves immediately. Click text to edit; use the element selector,\n**+ Insert**, and **Edit** for formatting, insertion, deletion, and role renaming.\n\nUse **File → Import…** to choose a `.fountain` screenplay. Import replaces the\ncurrent screenplay and clears its cast; **Undo** restores both. Scene headings,\naction, character cues, parentheticals, dialogue, and transitions become editable\nelements. Centered text and lyrics retain their text in action/dialogue elements;\nsimultaneous dialogue is imported in source order. Title-page metadata, outline\nsections, notes, omitted passages, page breaks, and emphasis formatting are omitted.\nThe original Fountain file is preserved, and the imported screenplay autosaves to\n`screenplay.json`. Full-length scripts can be imported; video generation still\nrequires a script within the existing 100,000-character limit.\n\n- **Enter** creates the next screenplay element;**Shift+Enter** inserts a line break.\n- **Tab** in dialogue adds a parenthetical instruction. On an empty action, it\nchanges the element to a character cue;**Shift+Tab** selects the previous type.\n- **Backspace** on an empty element steps backward through its formats, then\nremoves it. One blank scene heading remains when the script is cleared.\n- **⌘/Ctrl+Z** and**⌘/Ctrl+Shift+Z** undo and redo edits, including casting.\n\nThe gallery supplies saved appearance and voice descriptions. The GUI uses these for a scene's first shot, then carries frames forward. Solo gallery portraits are not supplied as separate identity references to an ensemble render.\n\nFollow the main path: **UI → Python entry point → Codex planner → shot renderer**.\n\n| File | Responsibility | \n|---|---|\n| [src/app.rs](https://github.com/andrewarrow/aivs/blob/main/src/app.rs) | Desktop workspace, setup gate, and optional chat UI. | \n| [src/screenplay.rs](https://github.com/andrewarrow/aivs/blob/main/src/screenplay.rs) | Screenplay editor, casting, and document persistence. | \n| [src/video.rs](https://github.com/andrewarrow/aivs/blob/main/src/video.rs) | Starts setup/render workers, handles cancellation, and reads Python progress events. | \n| [python/video.py](https://github.com/andrewarrow/aivs/blob/main/python/video.py) | Pipeline entry point; `generate_screenplay()` coordinates planning and rendering. | \n| [python/shooting_script.py](https://github.com/andrewarrow/aivs/blob/main/python/shooting_script.py) | Parses story beats, calls Codex, validates the plan, and saves shooting scripts. | \n| [python/shooting-script-instructions.txt](https://github.com/andrewarrow/aivs/blob/main/python/shooting-script-instructions.txt) | Director guidance for staging, timing, continuity, and sound. Start here to improve planning. | \n| [python/shot_renderer.py](https://github.com/andrewarrow/aivs/blob/main/python/shot_renderer.py) | Renders sequential shots, carries frames forward, checks speech, and assembles the result. | \n| [python/quality_backend.py](https://github.com/andrewarrow/aivs/blob/main/python/quality_backend.py) | Local LTX-2.5/MLX generation and model setup. | \n| [python/speech_validation.py](https://github.com/andrewarrow/aivs/blob/main/python/speech_validation.py) ,[python/validation.py](https://github.com/andrewarrow/aivs/blob/main/python/validation.py) | Speech matching and media checks. | \n| [src/codex.rs](https://github.com/andrewarrow/aivs/blob/main/src/codex.rs) | Codex account login and optional chat integration. | \n| [src/main.rs](https://github.com/andrewarrow/aivs/blob/main/src/main.rs) ,[src/playback.rs](https://github.com/andrewarrow/aivs/blob/main/src/playback.rs) | Native egui/winit/wgpu window and macOS AVKit playback. | \n\nAIVS launches `codex app-server` and exchanges newline-delimited JSON over\nstdin/stdout using the [official Codex app-server protocol](https://learn.chatgpt.com/docs/app-server).\nThe planner in `python/shooting_script.py` initializes a connection, creates an\nephemeral read-only thread, and starts a turn with the screenplay and a JSON\noutput schema. It streams activity into the UI, then validates the returned plan.\n\nThe Rust client in `src/codex.rs` handles login and optional chat separately.\n**Show chat** opens that panel; chat threads use `workspace-write` in the selected\nworkspace, with network access disabled and `approvalPolicy: never`. Reconnecting\nstarts a fresh chat. The Codex CLI owns credentials, and AIVS inherits its model\nand reasoning defaults.\n\nChange story direction in the director instructions. Keep Python responsible for source fidelity, timing bounds, and rendering contracts rather than rules tied to particular story props.\n\nEach run writes to `outputs/<timestamp>_<seed>/` in the workspace:\n\n| Artifact | What to inspect | \n|---|---|\n| `screenplay.txt` ,`screenplay-request.json` | Original script and cast bindings. | \n| `shooting-script.txt` ,`shooting-script.json` | Readable directions and the validated machine plan. | \n| `shooting-script-request-*` ,`shooting-script-response-*` | What the planner sent to Codex and received back. | \n| `shooting-script-events-*.jsonl` ,`shooting-script-errors-*.txt` | Live protocol events and diagnostics, including failed or stopped planning. | \n| `shots/001/` ,`shots/002/` , … | Individual takes and their reports. | \n| `speech-recognition.json` ,`speech-transcript.txt` ,`speech-validation.json` | Speech evidence for individual shots and the final assembly. | \n| `render.json` | Seed, status, prompts, timing, runtime details, and errors. | \n| `video.mp4` | Final validated assembly, present after a successful render. | \n\nFor planning work, you can skip video inference after completing setup. From the checkout using its default workspace:\n\n```\n.venv/bin/python python/video.py render \\\n  --script-file docs/examples/desert-road/screenplay.txt \\\n  --cast SUE=ava --cast JOE=eli \\\n  --plan-only\n```\n\nThis still calls Codex and uses the cached tokenizer. It generates a new plan,\nwhich may differ from the saved example. Remove `--plan-only` to render as well.\n`--shooting-script /path/to/shooting-script.json` reuses a plan after revalidating\nit against the current script and cast.\n\n```\ncargo test\ncargo clippy --all-targets -- -D warnings\ncargo fmt --check\npython3 -m unittest discover -s python -v\n```\n\nThe suites cover editing, casting, setup, planning, cancellation, media validation, Codex protocol handling, and responsive UI layout. Optional integration checks:\n\n```\n# Reuse an existing model cache while testing setup in a temporary workspace.\nAIVS_TEST_CACHE=/absolute/path/to/.cache/huggingface \\\n  cargo test live_setup_installs_python_then_skips_installation_on_the_next_launch -- --ignored\n# Use an existing ChatGPT login for a small live Codex request.\ncargo test live_app_server_streams_a_complete_reply -- --ignored\n```\n\nTiming is estimated, and generated actions, faces, voices, and lip sync still need human review. Frame continuity helps within a scene but does not lock identity across new locations. Whisper can mishear dialogue; a matching transcript does not prove the correct speaker or a convincing performance. Speech mismatches stop the run and leave completed takes available for review.\n\nThe editor has no print pagination or Final Draft import/export. Other renderers\nand separate speech synthesis are outside the main workflow. An opt-in\n[character consistency experiment](https://github.com/andrewarrow/aivs/blob/main/experiments/character-consistency/README.md)\nexplores portrait-guided starting images and fixed dialogue audio; it has separate\nsetup and limitations.\n\n`.env`, `.venv`, `.cache`, `screenplay.json`, and generated `outputs/` stay out of Git.\nThe adapted Studio source retains its **GPL-3.0-only** license; see [LICENSE](https://github.com/andrewarrow/aivs/blob/main/LICENSE).", "url": "https://wpnews.pro/news/ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol", "canonical_source": "https://github.com/andrewarrow/aivs", "published_at": "2026-10-26 13:18:49+00:00", "updated_at": "2026-10-26 13:41:35.537503+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "ai-tools", "ai-agents", "large-language-models"], "entities": ["AIVS", "Codex", "LTX-2.5", "Apple Silicon", "MLX", "FFmpeg", "andrewarrow", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol", "markdown": "https://wpnews.pro/news/ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol.md", "text": "https://wpnews.pro/news/ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol.txt", "jsonld": "https://wpnews.pro/news/ltx-2-5-22b-ic-lora-ai-video-studio-with-codex-application-protocol.jsonld"}}