I gave an LLM agent write access to my cloud drive. Three bugs taught me how to constrain it.

A developer built Mediary Scout, a self-hosted tool that uses an LLM agent to search, transfer, and verify media files in cloud storage. Three critical bugs revealed that LLM agents cannot be trusted to self-limit; deterministic gates must enforce constraints on searches, transfers, and deduplication before irreversible actions occur.

I wanted a media library that knew the difference between what should exist and what does. Most automation I tried picked one side. Some tools search well and never track what you already have. Others move files and assume the move worked. I wanted the gap between those two things to be the thing the software acted on. So I built Mediary Scout https://github.com/fancydirty/mediary-scout . You name a movie or a show. An LLM agent searches your indexers, transfers the best match into your own cloud drive, then reads the drive back to confirm what landed and what is still missing. It runs self-hosted. You bring your own drive, your own model, your own metadata key. There are desktop builds for Mac and Windows if you just want to double-click and run it, and a read-only demo https://demo.mediaryscout.app if you want to watch one acquisition play out first. The drives it speaks today happen to be Chinese cloud storage 115, Quark, GuangYaPan . That detail does not matter for the rest of this post. The part that took real work was different: handing an LLM tools that move and delete files, and stopping it from doing something dumb with them. Three bugs taught me most of what I now believe about that. The web app does almost nothing interesting. It writes a row to a Postgres queue and returns. A long-running worker picks up the row and starts a sandboxed agent. The agent gets a small set of tools: search resources, transfer a candidate, list a directory, move files into a season folder, mark episodes as obtained. Every tool runs through a deterministic workflow that owns the actual side effect. The agent proposes. The workflow decides whether the proposal is allowed, performs it, and reads the world back. That split is the whole design. The model is the part I cannot fully predict, so it gets the smallest possible blast radius. The deterministic code around it holds every irreversible action and every check. When I violated that split, things broke. They broke in the order below. A twelve-episode season needs, at most, one good complete pack. My agent decided to cover it with eleven overlapping season packs, after running sixteen separate searches to find them. One planning pass spent 6.8 minutes inside the model loop. The transfer loop then tried to receive all eleven packs before deduplication ran. 115 has a per-operation API budget that exists to stop exactly this kind of hammering. The run tripped it PAN115 RATE LIMIT and died. The whole acquisition failed because the agent was being thorough. My prompt had a line that said overlap was safe because a later step would dedup. That line was a lie. Dedup ran after the transfers that spent the budget. The cleanup I promised the model never got a chance to happen. I spent an afternoon rewording the prompt to ask for restraint. The model ignored all of it. Asking a model to be economical works about as well as asking water to be less wet. The fix lived in deterministic code, not in English. A greedy set-cover function trimToMinimalCoveringCandidates computes the fewest packs that cover every wanted episode and drops the rest before any transfer happens. A second gate caps distinct searches at eight and dedups identical queries at the tool boundary, so the same keyword can only hit the provider once. The model can ask for sixteen searches. It gets eight, and the redundant ones come back as cached snapshots. The lesson generalizes past my project. If your agent produces a number how many times to search, how many things to select, how many calls to make , put a deterministic ceiling on that number. Do not trust the model to limit itself, and do not trust a future step to clean up the mess. The gate has to sit between the model and the irreversible action. Coverage is the question of which episodes I have. The honest answer requires looking at the real files in the drive. I kept trying to compute it mechanically instead, and my own design notes kept telling me to stop. I wrote a re-read that hit the drive after a mark to "verify" the file was present. I wrote a filename parser that guessed episode numbers from titles. I wrote a check that called a movie acquired if the directory had any video file. Each one felt reasonable. Each one put a mechanical guess in the seat where the agent's judgment belonged. The discipline I landed on draws a hard line. The agent decides coverage by inspecting real files, after it has moved and flattened them into place. It then declares which episodes it got. That declaration is a plain statement with no file IDs and no hidden re-read. The system records it and moves on. The system never counts files on its own to second-guess the agent, and it never parses a filename to decide truth. The bookkeeping side is just as strict. I have whatever the agent marked. I should have whatever the metadata says aired. The missing set is the subtraction between those two. A scheduled sweep only wakes the agent for shows the subtraction says are incomplete. Thousands of finished shows never get scanned, because there is nothing to scan for. The transferable point: let the model judge the messy, real-world question, and make it judge against the actual world rather than its own narration. Keep your deterministic bookkeeping separate and dumb. Trouble starts the moment those two responsibilities bleed into each other. This one cost me the most pride. Users behind a slow model reported that the inline progress bar looked empty during an acquisition. I opened a PR. The bar's phase mapping had a step in the wrong band, so I fixed the band math and shipped it. Still empty. Second PR. The progress events fire only when a tool call happens, and one search was taking ninety-four seconds of a three-minute run, so the bar froze between events. I added a client-side trickle that animates between server updates. Shipped it. Still empty. The author of the bug report kept insisting there was no green at all, that anything above ten percent would be visible by eye. I had "verified" my two fixes by reading style.width off the element and seeing a sensible percentage. The number was correct every time. The third PR found it. The fill was a <span . A span defaults to display: inline , and width has no effect on an inline box, so the fill collapsed to zero by zero pixels and painted nothing. My demo component used a <div and worked fine. I copied its logic into the production badge and swapped the element type without thinking about it. style.width reported five percent, then twenty-one percent, then thirty-four. getBoundingClientRect reported a width of zero the entire time. I had been reading the value I set, not the pixels the browser drew. The two had nothing to do with each other for a collapsed inline element. The fix was one CSS line. The lesson was larger than the fix. Test the artifact the user sees, not the proxy that is convenient to read. A green bar is green pixels on a screen. It is not a string in a style attribute. I trusted the proxy for three rounds and let a confident-sounding number lie to me. The pattern under all three fixes is the same. A probabilistic core, wrapped in deterministic plumbing that owns every irreversible action, gates every number the model emits, and verifies against the real world instead of the model's account of it. The agent in Mediary Scout is small on purpose. It reads search results and picks. It cannot decide that a run is done, cannot count files to fake coverage, cannot blow past a search budget, cannot touch a directory outside its sandbox. Those constraints live in code the model never sees and cannot argue with. Everything I trust about the system, I trust because a deterministic function checked it, not because the model said so. If you are building agents that do real, irreversible things, that is the part worth most of your attention. The model will surprise you. Your job is to make the surprises cheap. The project is open source and self-hosted at github.com/fancydirty/mediary-scout https://github.com/fancydirty/mediary-scout , with a live demo at demo.mediaryscout.app https://demo.mediaryscout.app and desktop builds at mediaryscout.app https://mediaryscout.app . The drive backends are domain detail. The plumbing is the part I would reuse anywhere. One practical note, since the drives it speaks today are Chinese and most readers won't use them. If this pattern is useful to you and you'd want your own drive supported Google Drive, Dropbox, a regional one , the drive layer is a self-contained plugin behind a brand registry: a client plus a transfer executor. I'd welcome a PR and I'm happy to help scope one. That's the payoff of keeping the model's blast radius small. The boundaries end up clean enough that adding a drive is bounded work, not surgery.