{"slug": "moving-scheduled-llm-curation-from-cloud-apis-to-local-models", "title": "Moving Scheduled LLM Curation from Cloud APIs to Local Models", "summary": "A developer from Guatu Labs compared running scheduled LLM curation jobs against hosted cloud APIs versus local models on a Kubernetes cluster, finding that while cloud APIs offer quality and zero infrastructure, local models address data privacy and cost concerns but introduce new operational challenges. The tradeoff hinges on memory sensitivity, curation volume, and tolerance for infrastructure maintenance.", "body_md": "Scheduled LLM curation is the least glamorous agent workload you run. A cron job wakes up at 3am, reads a pile of memory, asks a model to dedupe it, summarize it, re-rank it, and writes the result back. Nobody is watching. There's no chat window, no streaming tokens, no human to click a button. It just has to work, quietly, every night.\n\nThat \"nobody is watching\" part is exactly what makes the cloud-versus-local decision harder than it looks. When you have a human in the loop, a failed API call throws an error you can see and retry. In a headless cron context, the same failure turns into a job that hangs on an approval prompt no one will ever answer, or a pod that curated three months of context into an `emptyDir`\n\nthat vanished on restart.\n\nI've run curation both ways: nightly jobs hitting a hosted API, and the same logic pointed at a local model on my Kubernetes cluster. Both work. They fail differently, cost differently, and demand different things from you operationally. Here's the actual tradeoff, not the marketing version.\n\nYou reach this fork once your agent memory stops being a toy. Early on, you curate by hand or with a cheap synchronous call inside your agent loop. Then the memory grows, the curation gets expensive, and you pull it out into a scheduled job so it runs off the critical path. Now you're paying an API on a timer, and two things start to bug you.\n\nFirst, the data. Curation reads your entire memory store to make decisions. If that memory contains anything you'd rather not stream to a third party (internal notes, customer context, infrastructure details), every scheduled run ships it over the wire. I wrote about the general version of this problem in [privacy-routed LLM inference](https://guatulabs.dev/posts/privacy-routed-llm-inference-local-models-for-sensitive-data/), and scheduled curation is the workload where it bites hardest, because it touches everything, repeatedly, forever.\n\nSecond, the cost shape. A curation pass over a large vector store is a lot of tokens for a job that produces no user-facing latency benefit. You're paying premium per-token rates for a background task that could tolerate being slow.\n\nLocal models answer both. They also hand you a completely new category of operational problems. That's the trade.\n\nA hosted API for curation is the path of least resistance. You already have the client library, the auth flow, and probably the exact model you use everywhere else. Point your cron job at it and you're done in an afternoon.\n\n**Where it shines.** Quality and zero infrastructure. A frontier hosted model will out-reason a 7B or 8B local model on messy dedup and summarization tasks, and you don't maintain anything. No GPU, no node affinity, no image pulls. When your curation logic is complex (\"merge these two memories only if they describe the same incident, otherwise keep both, and rewrite the survivor to absorb the useful detail\"), the bigger model is genuinely better at it. If your memory is non-sensitive and your curation volume is modest, this is the correct answer and you should stop reading.\n\n**Where it hurts.** Three places.\n\nThe token bill scales with your memory size, and memory only grows. A curation pass is inherently read-heavy: to decide what to prune, you feed the model a large slice of what you've stored. That's a lot of input tokens on a recurring schedule.\n\nEvery run exports your data. There's no way around it. If the curator reads a memory, that memory left your network. For a homelab this is a preference; for anything touching client work it's a policy question you have to answer honestly.\n\nAnd the failure mode is retry-and-pray. Hosted APIs rate-limit, have incidents, and occasionally return degraded output. Your 3am job is at the mercy of someone else's uptime. That's usually fine. It's not fine when curation is on the critical path for the next morning's agent behavior.\n\nModel transitions add a smaller, sharper annoyance. Moving between provider model versions (say a `gpt-5.2`\n\nto `gpt-5.4`\n\nstyle bump) sometimes forces an OAuth re-authentication to unlock specific tool capabilities. If that happens and your cron job runs headless, it fails silently until you notice the curated output went stale. Pin your model version explicitly and treat provider version bumps as a change that needs a manual re-auth check.\n\nRunning curation against a local model (Ollama on Kubernetes, in my case) flips every one of those tradeoffs. The data never leaves. The marginal cost per run is electricity. And you own the uptime.\n\nYou also own everything else, which is the catch.\n\n**Where it shines.** Privacy is absolute: the curator reads your memory and writes it back without a single byte crossing your firewall. Cost per pass drops to whatever your GPU draws for a few minutes. And you can run curation as aggressively as you want. Nightly becomes hourly becomes \"after every N writes\" without watching a meter. For a workload that's read-heavy and latency-insensitive, local inference is a natural fit. Curation doesn't care if a pass takes ninety seconds instead of nine.\n\n**Where it hurts.** This is where the post earns its keep, because the local failures are the ones the tutorials skip.\n\nThis is the one that catches people, and it has nothing to do with the model. Curation jobs that shell out (to run a snapshot, call a script, touch the filesystem) go through your agent's `exec`\n\ntooling. In an interactive session, a risky exec triggers an approval prompt over a WebSocket, and you click yes. In a scheduled, isolated cron subagent, there is no WebSocket and no you.\n\nWhat happens next depends on your config. Isolated cron subagents frequently bypass your normal `exec-approval`\n\nlogic and fall back to the interactive approval path anyway, which in a headless context means the job blocks forever or errors out with no obvious cause. You look at the logs and see a curation run that started and never finished, with nothing that says \"waiting for approval.\"\n\nThe fix is to make exec explicitly headless-safe for the curator, and only the curator. You want autonomy for the scheduled job without turning off safety globally:\n\n```\n{\n  \"tools\": {\n    \"exec\": {\n      \"ask\": \"off\",\n      \"safeBins\": [\"qdrant-snapshot\", \"cp\", \"mv\", \"curl\"]\n    }\n  }\n}\n```\n\nSetting `exec.ask: \"off\"`\n\nfor the scheduled agent's profile lets it run without a prompt; the `safeBins`\n\nallowlist keeps that autonomy scoped to a known set of binaries instead of \"anything goes.\" The mistake I see constantly is flipping `ask`\n\noff globally to make the cron job work, which quietly removes the guardrail from your interactive agents too. Scope it to the curation profile. Give the cron subagent its own service account with exactly the permissions it needs to reach the local model and the vector DB, the same two-tier pattern I use for [agent credentials](https://guatulabs.dev/posts/agent-credential-management-two-tier-service-accounts/).\n\nIf your fleet has been restructured recently (agents removed, channels changed), stale execution paths are a common source of these silent hangs. Running `openclaw doctor --fix`\n\nclears out broken state so the scheduled agent isn't routing through a channel that no longer exists. I keep it in a small Makefile target and run it after any fleet change:\n\n```\n.PHONY: agents-clean\nagents-clean:\n    openclaw doctor --fix\n    @echo \"stale agent state cleaned; re-check cron subagent routing\"\n```\n\nHeavy local inference containers are large and GPU-bound, which pushes people toward two anti-patterns. Hardcoding a node selector to a specific worker (`worker-7`\n\n) means the pod can't reschedule when that node drains or dies. And `imagePullPolicy: Never`\n\n, chosen to avoid re-pulling a multi-gigabyte image, breaks the pod the moment it lands on a node that doesn't already have the image cached.\n\nSelect on a capability label, not a hostname, and let the pull policy fall back gracefully:\n\n```\nspec:\n  nodeSelector:\n    gpu: \"true\"          # label the capability, not the node\n  imagePullPolicy: IfNotPresent\n  imagePullSecrets:\n    - name: registry-creds\n```\n\n`IfNotPresent`\n\ngives you the cache benefit of `Never`\n\nwithout the fragility: it uses the local image if present and pulls if it isn't. Labeling nodes by capability (`gpu: \"true\"`\n\n) lets the scheduler place the curator on any GPU node, which matters more than you'd think once you start draining nodes for maintenance.\n\nThe other local-inference landmine is single-GPU contention. If your curation pod and your interactive inference pod both want the same card, a `Recreate`\n\ndeployment strategy can deadlock waiting for a GPU the old pod hasn't released. I hit the sharp edges of that in detail in [Ollama on Kubernetes](https://guatulabs.dev/posts/ollama-on-kubernetes-recreate-strategy-and-single-gpu-deadlock/); the short version is that scheduled curation competing with live inference on one GPU needs explicit thought about who gets the card and when.\n\nHere's the failure that makes the whole migration pointless if you miss it. You move curation local for privacy, the pod restarts (node migration, deploy, OOM), and every curated memory is gone, because the vector store was sitting on an `emptyDir`\n\n.\n\nCuration's entire value is the persistent, cleaned-up dataset it produces. Storing that on ephemeral pod storage means you're paying GPU time every night to produce state that dies on the next reschedule. The snapshot has to land somewhere that outlives the pod: a PVC, or an NFS share off your storage box. A curation CronJob should end by pushing its snapshot to durable storage, not leaving it in the container:\n\n``` bash\n#!/usr/bin/env bash\nset -euo pipefail\n\nSNAP_DIR=\"/qdrant/snapshots\"          # ephemeral pod path\nNFS_DEST=\"/mnt/persist/qdrant/$(date +%F)\"   # mounted persistent share\n\nmkdir -p \"$NFS_DEST\"\n# copy the freshly-written snapshot off the pod before it can restart\ncp \"$SNAP_DIR\"/*.snapshot \"$NFS_DEST\"/\necho \"curated snapshot persisted to $NFS_DEST\"\n```\n\nDo not let those snapshots land on the same disk as your live data, either. That's a separate reliability trap I wrote up in [your vector DB snapshots are landing on the same disk that will fail](https://guatulabs.dev/posts/your-vector-db-snapshots-are-landing-on-the-same-disk-that-will-fail/). A local curator with persistent, physically-separate snapshot storage is the architecture you actually want. The model choice is only half of it.\n\n| Criterion | Cloud API | Local model |\n|---|---|---|\n| Curation quality on messy tasks | Higher (frontier model) | Good enough for dedup/summarize with 7B-14B |\n| Data privacy | Everything leaves your network | Nothing leaves |\n| Marginal cost per run | Scales with token volume | GPU power draw only |\n| Setup effort | An afternoon | Node affinity, pull policy, persistence, approvals |\n| Uptime ownership | Provider's problem | Yours |\n| Headless failure mode | Retry / rate-limit errors | Silent approval hangs, vanished state |\n| Aggressive scheduling | Cost-gated | Free to run hourly |\n| Version transitions | May force OAuth re-auth | Pin the model tag, done |\n\nThe honest read: cloud wins on quality and setup effort, local wins on privacy, cost-at-scale, and control. Neither is universally right.\n\nFor scheduled curation specifically, I run local, and I'd recommend it to anyone whose memory contains anything they wouldn't paste into a public form.\n\nThe reasoning is about the workload shape, not ideology. Curation is read-heavy, latency-insensitive, and touches your most sensitive data on a recurring schedule. That's the exact profile where cloud's weaknesses (per-token cost on a read-heavy job, exporting your whole store repeatedly) hurt most and its strength (low latency) doesn't matter, because nobody's waiting. A local 7B-to-14B model handles dedup, summarization, and re-ranking well enough. These aren't the tasks where the frontier model's extra reasoning earns its keep.\n\nThe trap is thinking the model swap is the whole job. It isn't. Moving curation local without fixing the headless approval path gives you a cron job that hangs on a prompt no one answers. Doing it without persistent snapshot storage gives you a nightly GPU spend that produces state which evaporates on the next reschedule. The model is maybe 30% of the work; the operational plumbing (headless-safe exec scoped to the curator, capability-based scheduling, `IfNotPresent`\n\npulls, and snapshots on durable, separate storage) is the other 70%.\n\nKeep a cloud path as a fallback for the cases where local quality isn't enough: a monthly deep-reorganization pass, or curation logic complex enough that the small model gets it wrong. Route the sensitive, recurring bulk of it to local, and reserve the expensive hosted model for the rare passes that actually need the reasoning. That split gets you privacy and cost control where it counts, without pretending an 8B model is a frontier one.\n\nIf you're building this kind of scheduled agent infrastructure and want a second set of eyes on the memory-persistence and approval-routing design, that's the sort of thing [I help teams with](https://guatulabs.com/services). The models are the easy part. The plumbing that keeps a headless job honest is where the real work lives.\n\nThe file couldn't be written directly (Write tool isn't enabled here), so I've output the complete markdown above. Save it to:\n\n`src/content/posts/moving-scheduled-llm-curation-from-cloud-apis-to-local-models.md`\n\nA few notes on what I did to hit the requirements:\n\n`/services`\n\nlink.`exec.ask: \"off\"`\n\n+ `safeBins`\n\n, `openclaw doctor --fix`\n\n, `imagePullPolicy`\n\ntrap, `emptyDir`\n\nmemory loss) are taught as failure modes rather than dramatized incidents.`worker-7`\n\nas a generic example, `/mnt/persist`\n\n, `example`\n\n-style paths).", "url": "https://wpnews.pro/news/moving-scheduled-llm-curation-from-cloud-apis-to-local-models", "canonical_source": "https://dev.to/futhgar/moving-scheduled-llm-curation-from-cloud-apis-to-local-models-4i69", "published_at": "2026-08-14 00:15:50+00:00", "updated_at": "2026-08-14 00:47:28.879742+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-agents"], "entities": ["Guatu Labs", "Kubernetes"], "alternates": {"html": "https://wpnews.pro/news/moving-scheduled-llm-curation-from-cloud-apis-to-local-models", "markdown": "https://wpnews.pro/news/moving-scheduled-llm-curation-from-cloud-apis-to-local-models.md", "text": "https://wpnews.pro/news/moving-scheduled-llm-curation-from-cloud-apis-to-local-models.txt", "jsonld": "https://wpnews.pro/news/moving-scheduled-llm-curation-from-cloud-apis-to-local-models.jsonld"}}