Building a Local Agent Run Production Studio Developer Owain began building a local "Agent Run Production Studio" in early 2026 to automate video production — research, writing, images, animation, narration and assembly — on self-hosted hardware, following his December 2025 post on turning a tinybox green v2 into a private AI home server running local models through Ollama. The project orchestrates multiple inexpensive models including DeepSeek and GLM under a self-hosted or low-cost model harness, with Codex used as the coding agent for the end-to-end build, and is intended to support swapping models on the fly. The longer-term goal is a feedback loop that uses video platform analytics APIs to compare titles, thumbnails and retention against competing videos and feed those observations into the next production round; the author describes the post as a starting point, not a finished system. Building a local Agent Run Production Studio Back in December 2025, I wrote about turning the tinybox into a private AI home server https://owain.bearblog.dev/turning-a-tinybox-green-v2-into-a-private-ai-home-server/ . I had local models running through Ollama, a chat interface and access through Tailscale. Near the end of that post, I wanted to add agents that could get on with longer tasks. This project is one attempt at doing that. My plan is to build a system that takes a video idea through research, writing, images, animation, narration and assembly. Eventually, I want it to bring me a finished film and the material around it, ready to review, then prepare a long-form upload and a set of Shorts. There's quite a lot hidden in that sentence, as it turns out. I am not sure yet if the final system will be my own harness end to end which is where this first experiment essentially ended up or offload the harness to something like Hermes with access to local tools I build. But I want to test multiple versions. The idea came from seeing a few AI-generated videos get more than a million views. Some were slop https://www.youtube.com/watch?v=-5nPRWkl0Ps&t=3s ; others were slightly better historical or architectural videos https://www.youtube.com/watch?v=qDSRO9v-yRg . I wanted to see what it would take to automate something like this while keeping the generation on hardware I control. History already looked crowded, but it seemed like a useful proving ground: a repeatable format where I could compare the output with other videos, learn whether the system and models were any good, and later try other subjects, also I wanted to see what a fully autonomous feedback loop would look like for this considering the cheap models available for evaluation available and data that the analytics APIs for video platforms provide. There were a few things I wanted to try. I hadn't run image and video models locally, or built orchestration around inexpensive, capable language models such as DeepSeek and GLM, and I want to see how far I can get using MANY model types together orchestrated by a self hosted or cheap model, I also wanted to make tsure the system could be built around me swapping models on the fly but the core system being able to handle that. I also wanted to try Codex as the coding agent. I hadn't used it to build a project end to end before Claude 4 for life . The longer-term ambition is to see whether a local agent run system could create something people would spend their time watching using local generation models without any input from me. That's the ambition, anyway; it doesn't mean the system is there yet. The other part of the idea was a feedback loop. I didn't want automation to stop when a video had been assembled. Over time, I want the system to research what is working in the same subject area, then compare that with its own results: which titles and thumbnails earn attention, where viewers keep watching or leave, what people say about the films, and which kinds of shots seem to hold a sequence together. I've never run a video channel, on YouTube, TikTok or elsewhere, so its an area my own opinion won't taint the expermiment too much. This feels like somewhere a research agent and feedback loop could prove extremely valuable. Those observations could inform the next round of topic research, writing and production. The value would be in turning each release into evidence for the next one, instead of making every creative decision from scratch. That needs reliable analytics and enough videos to see patterns; a handful of views can't tell me why something worked, and a high click rate on its own doesn't make a good film. This post describes the starting point, not the finished system. Across a few posts, I want to improve both the films and the automation and models that makes them: use audience responses and viewing data to decide what to investigate and where to spend time improving, test changes to scripts, shots, titles and thumbnails, then make the useful parts of that learning repeatable in the pipeline. Some of that may eventually be automated, so the system can carry what it learns into later productions. The first version could make a complete short film, but it still needed me to review and correct work throughout. The journey from that base towards a system that can learn and improve with each release, while making better films and avoiding AI slop, is what I want to document here. The first version needed attention at almost every stage. For this part of the series, I'm going back to the October setup: checking the machine, testing the models and getting the first complete production through it. I'll work through the local models, controlling the process from Codex and the later OpenRouter operator over a few posts. The films are a useful reason to build the system, but I'm just as interested in what it takes to get agents doing useful work by themselves on hardware I control. I'm keeping the channel itself separate from these posts. Part of the project is finding out whether the system can produce something people choose to watch and whether the economics make sense. Sending readers from here to the channel would make those results harder to interpret. The engineering is what I want to share here. Getting the machine into a usable state The video project's hardware checks are dated 17 October 2026. The machine has three RTX 5090s, each with 32 GB of memory; one is reserved for other work. That's the configuration behind the measurements in these posts. Plenty of GPU memory, then. The slightly annoying detail is how it's divided up. The three cards weren't one shared 96 GB pool. For this setup, the useful model was three independent 32 GB devices. A workflow had to fit on the card running it, and the scheduler had to avoid putting incompatible workloads on the same device at the same time. That pushed the design towards running separate generation jobs across the cards. A film contains lots of shots, so there was useful parallel work available without having to distribute a single generation across all three GPUs. I was also thinking about how this could scale to multiple channels or themes later, keeping the system usefully occupied. Those are the versions from that setup, rather than an installation recipe for today. What mattered was having a known working combination and checking that it survived a restart. A server that works until its next reboot isn't a particularly good foundation for unattended rendering. Checking real work rather than inventory Before downloading a pile of models, I wanted to know that all three cards could actually do work. The checks ran a bf16 matrix multiplication on each GPU, then loaded all three together for 90 seconds. The cards sustained roughly 221–224 TFLOPS in that test, with maximum temperatures between 67°C and 73°C. The recorded limiting factor was the configured power cap. The driver and devices also survived three reboots. That gave me a baseline. If a model was painfully slow later, I had something to compare it with before blaming the model or disappearing into driver troubleshooting. The power measurements mattered too, as I pay the electricity bill at home where the GPUs are hosted. At full load the three cards were drawing around 1.7 kW between them. That changes how I think about a retry. A repeated render uses both machine time and electricity, and a weak review process can consume quite a lot of both. I wanted to know how long a usable clip took, how many attempts ended up in the bin and how often I had to step in. Those turned out to be more useful questions than how quickly I could make the GPU graphs move. Giving the video system its own space The machine already had services on it, including a self-hosted Hermes setup I use. The video project needed its own runtime and model services so that changing a generation dependency didn't require rebuilding unrelated tools. I built the application with Codex, mainly GPT-6.1-Sol at medium reasoning effort. I won't go too deep into this part as software stacks in apps like these I view as secondary now models can do all the heavy lifting and to be clear outside of networking I did not look at a line of application code. The stack is FastAPI, PostgreSQL, Redis and Celery. Model serving lives separately: ComfyUI instances for image and video generation, a dedicated local language-model service and a speech service. These all run in containers on the TinyBox alongside my other services. If interesting I can do another post on this piece but my focus for this project is models and agents running locally for value not the video app its self The application keeps track of production. Workers pick up the jobs, local text, video, animation, video and audio model services generate the media and FFmpeg puts it together. It's a fairly ordinary application stack around some very large model files. It will go through more iterations, but this was enough to get a video through the door. The production workflow I wanted to take one idea all the way to a watchable video and get it published to test the generation and feedback, so the pipeline follows the stages a small production would need anyway: research, script, storyboard, images, animation, narration and assembly. Each stage leaves an artifact the next one can use. That gives me a place to check the work before I spend time generating more media. The models have different jobs along the way. Language models help organise research, draft the narration and describe the shots. Image models turn those shot descriptions into reference frames. A video model adds movement to an image the pipeline has already selected. A speech model reads the approved narration. Ordinary application code keeps track of versions and timing, while FFmpeg combines the pieces. The model names and why I chose them can wait for the next post. Throughout the video pipeline the system is generating useful artifacts that later stages can refer to to keep all of the pieces consistant and make sure we are telling a full story and allow a review agent at the end to compare the initial research against the final product. Research gives the story somewhere to start A history video needs more than an interesting subject and a confident narrator. I wanted claims connected to sources before they turned into script lines. The research stage gathers the supplied material and sources, then organises it into claims, references and uncertainties. The language model can help structure that material, but its output does not make a claim true. Source checking and editorial review still matter. Keeping research separate also gives later stages something specific to draw on. The script can refer back to claims, and visual planning can inherit limits about what the evidence actually supports. The script turns research into something you can follow The script stage takes the research and turns it into a sequence a viewer can listen to. A language model is useful for producing a first draft, but the draft still needs an editorial pass for accuracy, pacing and whether the story makes sense when spoken aloud. The approved script becomes the production reference. Later work should follow that version, rather than a newer draft that happens to exist. Otherwise narration, visuals and subtitles can quietly end up describing different versions of the film. The storyboard connects each line to a picture Once the narration is in place, the storyboard breaks it into shots. For each shot it records the line being spoken, the visual idea and how the camera or subject should move. It also gives the sequence timing, so the film has a plan before image and video generation starts. The model helps describe what a shot might look like. The application works out shot order and duration from the script. That division is useful: a language model can suggest a view of a street, but I don't want it inventing the total running time by adding up seconds in its head. Images are the first visual check An image model creates candidate reference frames for each shot. I can compare those candidates, select one and send a rejected image back for another attempt. It is much cheaper to notice a wrong building or a modern vehicle at this point than after generating a moving clip from it. This stage is also where an attractive picture can be misleading. The model can blend places, add details the script never mentioned or get the period wrong. The storyboard and research provide constraints, but the generated image still needs to be looked at. Animation adds motion to a selected image After choosing a reference image, an image-to-video model generates a short clip. Giving it an image first anchors the shot: the model has a composition to work from, rather than having to invent the whole scene and its movement at once. The clip then needs its own review. Movement can introduce new objects or distort details that were fine in the still image. The image approval is not a promise that the video will be right. Narration gives the timing a real voice A text-to-speech model reads the approved script and creates audio for each narration section. Generating speech after the script is settled means the soundtrack follows the wording the film is actually meant to use. The measured speech also helps with timing. The storyboard provides a starting plan, then assembly can fit the shot sequence around the audio rather than relying on a guessed speaking rate. Assembly makes the film Finally, FFmpeg combines the clips, narration, any music, subtitles and the planned timing into an export. This stage is ordinary media processing rather than another generative model. That makes it a useful place for deterministic checks: does the file decode, do the audio and pictures line up, and is the result the expected length and format? Passing those checks tells me the export is technically intact. It doesn't tell me whether the story is clear or every image is accurate, so watching the result remains part of the process. The stages are separate because their outputs need different kinds of judgement. Sources support claims, an approved script controls the narration, images are checked before animation, and the assembled export can be checked as a complete file. That structure also makes it easier to replace one model when a better one becomes available without redesigning the whole production process. A simplified view of the early system looked like this: Private dashboard | v Application API ---- PostgreSQL | v Redis and task workers | +---- GPU 0 generation service +---- GPU 1 generation service +---- GPU 2 generation service +---- CPU assembly and checks This meant the dashboard didn't need to know how an image model was loaded. It could request a job and inspect its state. Changing a model workflow didn't need to change what a project, shot or approved asset meant. Keeping those boundaries made it easier to experiment with models without redesigning the whole production process. The storyboard review screen from a later project, captured on 8 November. I can compare image candidates, see which one was selected and inspect the resulting clip. This is the interface the pipeline grew into. Having the candidates next to each other made the review much more practical, this is a step we will hand off to an operator model at a later stage but this laid down the APIs and process for later. A model could give me three plausible images, but I still needed somewhere to decide which one belonged in the film. The green outline records that choice instead of leaving it in a chat message. The first milestone was a complete short film By the end of the first day I had ther first video through the pipeline fully locally. That version produced a short film through research, scripting, storyboarding, image generation, animation, narration and assembly. It also had a dashboard for reviewing candidates and inspecting jobs. Here's an excerpt from that first completed test film, pulled from the project's original export. It's about 22 seconds, with the original narration and music. The images and movement were generated locally; A frame from the same early film. Seeing it assembled was a much better test than looking at individual model outputs. The camera movement, narration and cuts had to work together. It also made it harder to ignore the details that still needed review. To be clear, this is not the quality I want the final films to be but proved that end to end using local models I could go from autonomous research all the way to an AI Generated video. The final orchestration agent will need a lot more tools to create different kinds of content before we get to a video I would be happy with. Despite this, this video and 5 other along ith shorts have been published so far to test the upload, analytics and feedback loops. Getting through all those stages exposed issues that a single impressive model output couldn't. The script needed editing. Images could look convincing while being geographically wrong. Timing needed rules. Generation needed retries, and the retries needed a history. A finished export was useful evidence that the components could work together. It was still a long way from a system I could leave to make production decisions on its own. One really nice thing the model came up with was the techncial review dahsbaord which processed the video through a full techncial QA to make sure all of the audio levels, aspect ratio etc. hit Youtubes standards, also giving me a view of the GPU hours used in total for the video and the cost for the video in electricity based on some estimations and my current tariff. From an export to a YouTube package Getting a video file out of FFmpeg is only part of finishing a production. It still needs the material around it: a title, thumbnail, description, captions, chapters and the smaller bits of copy that help introduce it. The package screen came later than that first October film; the screenshots here show a November version of the system. The package starts from the approved export and its shots. It proposes several title angles, then builds thumbnail variants from selected frames, changing the crop, layout, wording and accent colour. I can compare the candidates and choose a combination. The scores and suggestions help organise the choices; they don't tell me which one an audience will prefer. The description can include the opening hook, chapters and source list, alongside tags and category. It also prepares a pinned-comment suggestion, a Community post and material for Shorts. The comment and Community text are there for me to review and copy; they aren't posted automatically. All of this still needs a human read for accuracy and tone. The package offers different title angles and keeps the candidate text with its rationale. Title wording and project details are blurred in this copy. Several thumbnail layouts can be compared against the same approved image. The actual wording and project details are blurred here. The package also prepares supporting copy. Project-specific tags and text are blurred in this crop. The Shorts are cut from the film's approved clips and narration, with a vertical crop, a spoken hook and captions. Reusing approved material gives them a starting point, but the crop and shorter sequence still need their own review. Four rendered vertical Shorts in the publishing package, with identifying text covered The later package includes vertical cuts alongside the main film. Titles and on-screen wording are covered in this publication copy; these are screenshots of rendered previews, rather than playable Shorts. The same later production records generation time and energy. GPU energy is measured; wall energy and electricity cost are estimates. These figures belong to this later film, and electricity alone is not its total production cost. YouTube was the first platform I connected because I already knew the YouTube APIs and how to set them up in Google Cloud. I enabled the Data API and Analytics API on a Google Cloud project, then connected the channel through Google's OAuth flow. That gave the production app a direct route from an approved export and package to an upload, and a way to bring performance data back later. The upload uses YouTube's resumable upload API to transfer the video in chunks. Once the video exists, the system can set the chosen thumbnail and upload the caption track. The package and upload are tied to the same approved export so I don't accidentally attach metadata for one cut to another. The system can send the prepared package to YouTube privately. The channel label is outside this crop; public release remains a separate decision. The API connection also lets the system retrieve channel performance. The Data API gives it current video details and lifetime counts; the Analytics API adds daily views, watch time, average view duration, likes, comments, shares and subscribers gained. It can also fetch audience-retention data and line that curve up against the film's shots. That gives me a way to ask whether viewers tend to leave at a particular point, then look at what the film was doing there. It is a clue for the next edit, not proof that a single shot caused someone to leave. Impressions and click-through rate still need a separate reporting integration, so I can't claim the system has that full view yet. For now, the system uploads through the API as private. I review the upload and use YouTube Studio for any public release. The sync runs every six hours, but analytics can lag by one to three days, and one video won't give a dependable pattern. As more releases accumulate, the package can use its history of title angles and thumbnail layouts to inform new candidates. That is the beginning of the feedback loop I want to build through this series: publish, collect evidence, decide what seems to be working, and improve the next production without letting a metric replace editorial judgement. Next I'll get into the models and the pipeline around them and what tools I give to the agents to begin making videos beyond generated content. Since the first upload to YouTube on the 19th October we have: 5 AI Generated Full Videos Published 25 Shorts Auto Cut from the videos Published 12 Subscribers 10,112 Total Views 44.4 hours total watch time 10,265 Lifetime views 12 Subscribers