TL;DR:We built a one-shot HTTP worker that talks to Notion through MCP. Version one worked. Version two got cheaper and more readable, then failed in a new way. The harness was fine. The tool surface, the model, and the prompt were not the same problem, and we kept treating them as one.
We keep seeing the same pitch: put an agent in the cloud, give it tools, let it live in Slack, let it remember you. That is a product. It is not the product we needed.
We needed something dumber and more useful. Another service should be able to say "read this Notion page, write a summary somewhere, stop." No chat history. No personality that accretes over weeks. No always-on process. If nobody is calling it, it should cost nothing.
We started calling that shape a one-shot agent. One HTTP request. Tools for that request. A JSON result. Then the instance can go away.
This is the path we actually walked: first working version, what it got wrong, the Markdown fork, and the cheaper tricks that mattered more than swapping frameworks.
The first real task was almost boring. Once a week, pull a skill write-up from Notion, extract what mattered, and append it to a digest page. Callers would name pages in English. They would not paste Notion ids. If a name was ambiguous, the agent should refuse to write rather than guess.
If that loop is wrong, people stop trusting write-back. If it is expensive, nobody schedules it. If it needs a human to babysit a terminal, it is not a system.
So the constraints were social as much as technical:
Question you will probably ask: why not a cron script that hits the Notion API directly?
Because the task changes every call. This week it is a weekly digest. Next week it is "list in-progress rows and do not write." We did not want a new Python file per job. We wanted one worker that takes a prompt plus a list of tool servers.
Common pitfall: starting from a coding agent (shell, files, memory, a learning loop) and trying to shrink it into a request/response worker. You spend months deleting features you never wanted.
The loop is short on purpose.
caller
| POST /run { system prompt, user prompt, which tools }
v
one-shot agent (starts if needed, dies when idle)
| model + tool loop
v
MCP server for Notion
| integration token
v
Notion API
|
+--> JSON back: result, tools used, tokens, cost
A few choices fell out of that picture.
HTTP, synchronous. The caller waits. A weekly digest can wait two minutes. We did not want a job queue for v1.
The agent is not the Notion client. Notion (and later Slack, GitHub, whatever) lives behind MCP: a small server that exposes tools. The agent image stays thin. The Notion token never enters the agent process as "the caller's header."
Search first. Callers say "Weekly Skill," not a 32-character id. The model searches, picks a title match, and stops if it cannot.
A cheap default model behind an OpenRouter-style gateway. We did not want a Claude bill on every internal trigger. The first default was a very cheap DeepSeek flash model, on the order of a few cents per million tokens.
Decision: keep the harness thin (validate request, attach tools, run the loop, return JSON). Put intelligence in the model, the tool server, and the prompt. If quality is bad, we change those three before we change the loop.
That last sentence sounds obvious. We still almost violated it.
Notion's hosted connector is built for Claude Desktop and similar clients. It wants OAuth. We wanted an internal integration token in a secret store and a server we could run next to the agent. Self-hosting the official Notion MCP was the boring path that matched that.
The "correct" cloud version is: every caller has a service account, mints a short-lived token, the platform checks it. That is great when all callers live in your cloud. It is miserable when the next caller is a script on a laptop or a third-party job.
We shipped a shared API key on the agent instead. Cloud ingress is open. The app checks the header and fails closed. Onboarding is "here is a key." Revoke is "rotate the key."
Tradeoff: you lose per-caller identity until you grow a key set. We accepted that for v1.
When quality wobbled, the tempting move was "use a real agent": something with memory, skills, a terminal, a personality that improves. We looked hard at that family (the persistent, self-improving runtimes people mean when they say they want Hermes-class agents).
They are good at being companions. They are the wrong shape for "POST, work, return, stop." They want state on disk. They want to stay up. They would make our Cloud Run bill and our threat model worse for a problem we did not have.
Decision: do not swap the loop because the model cannot name a tool. A thin loop plus OpenRouter already is the harness. Quality lives one layer up.
This one is ideological, and it showed up in a real request.
We had a generic worker. Someone sent a specific Notion page and a specific question ("summarize in-progress tasks"). The easy fix is to stuff that page name, that Status property, that persona into the system prompt.
That prompt then becomes unusable for the next job. The agent is no longer one-shot and generic. It is a weekly-skill bot wearing a generic coat.
The pattern we now follow: two layers.
Claude Desktop users barely write layer 1, because a strong model infers it from the tool list. A cheap model will not. Layer 1 has to be explicit, and it still must not contain this week's page title.
The first MVP was honest and a little ugly.
search
, retrieve page
, append blocks
, and friends).It worked. Search found pages by title. A digest got appended. A successful run we logged was on the order of 40,000 input tokens, a few hundred output tokens, and about a quarter of a cent.
For a weekly job, that is fine. For something you want to call all day, you start staring at the 40k.
The failure was not "it does not work." The failure was "it works in a way that does not travel."
JSON is a terrible reading format for a summarizer. Claude's own Notion connector feels good because it reads Markdown. We were stuffing block trees into a cheap model and asking it to sound like a colleague. Input tokens were the tax. Quality was the interest.
The cheap model was good enough for named tools. That hid a landmine. When every tool has a clear name, flash-class models can finish in a handful of steps. We thought we had "the model problem" solved. We had only solved it for a flat tool list.
Deploy was a human ritual. Build three images on a laptop, remember linux/amd64
on Apple Silicon, push, hope. One arm64 image and Cloud Run will not start. That is not an agent problem. It still blocks the agent.
We almost taught callers the wrong lesson. Version one accepted a vague system prompt. Official tools are discoverable. Callers copied that prompt to the next server. Then nothing worked, and it looked like the new server was broken.
Question you will probably ask: why not raise the step cap when a run dies?
Because a confused loop is a money printer. Twelve steps is a fuse. If the model is guessing tool names, more steps buys more guesses. Fix the prompt and the model first. Raise the cap last, and only for jobs you have already seen succeed in fewer rounds.
We wanted reads that look like documents, not like API dumps. We source-reviewed two community MCP servers that return Markdown. Both were MIT, both spoke Streamable HTTP, both kept the Notion token in the environment.
| Awkoy-style (two meta-tools) | Flat Markdown server (~40 named tools) | |
|---|---|---|
| How the model calls Notion | ||
execute(operation, payload) plus a schema helper |
||
| Direct tool names | ||
| Schema size in the prompt | Tiny (hundreds of tokens) | Large |
| Notion 429 handling | Built-in pacing and retry | You own it |
| Risk | The model must name the operation | |
| Tool-list overload |
Awkoy's trick is the interesting one. Instead of 40 tools, it exposes two. Every real action is an operation name inside a payload. The tool schema that lands in the prompt shrinks from something like 17,000 tokens to something like 400.
That is a real cost win if the model can dispatch. There is almost no public evidence of people driving that two-tool surface from a flash-class model. Desktop Claude does it because Claude can. We were about to find out what DeepSeek flash does instead.
Decision: do not replace version one. Run both. Callers pick a catalog name. Official named tools stay the simple path. Markdown stays the cheap-token path. Another scale-to-zero container is roughly free when idle.
We pointed the same /run
body at the Markdown server: same generic prompt, same flash model, use: markdown_notion
instead of use: official_notion
.
Two things happened, in order.
The official server exposes search as a tool the model can see. The Markdown server does not. It exposes two verbs. The model has to say search_pages
inside execute
. Our prompt never said that. Flash guessed, asked for schemas, retried, and burned 12 steps.
A clean Markdown run is 3 to 5 steps. A confused one is 12 and an error.
Before: named tools + vague prompt + flash = success.
After: two meta-tools + the same prompt + flash = step_limit_exceeded
.
The worker did not get worse. The instructions no longer matched the tools.
get_page_markdown
renders the page body. An inline database shows up as a stub: a title and an id, not the rows. The model would say "there is a database" and stop. That looks like a product bug. It is a missing rule.
Rows only come from query_database
. If the user asked about tasks, status, or items, we have to query. We also cannot assume the property is called Status
. Real databases are named State
, Stage
, a checkbox, a board.
Common pitfall: writing a "working" prompt for one database (Status = In progress
) and putting it in the system prompt. The next caller has different properties. The agent filters into empty and looks broken.
Version two's actual upgrade was not a new framework. It was:
Quality jumped because the model could finish the loop, not because we grew a memory subsystem.
We also stopped shipping images from laptops as the happy path. Merge to main builds and deploys. Terraform still owns env and IAM. The pipeline owns tags. Those two fighting each other is its own footgun (ignore image changes in Terraform, never apply Terraform from the image job).
These are the optimizations that paid rent. Not a toolkit dump. A short list of bets.
1. Pay for tokens where they leak.
Official JSON reads were the 40k-input problem. Markdown plus a tiny tool schema attacks that directly. Extra round-trips from a weak dispatcher can eat the savings. Measure both.
2. Price the model to the tool surface, not to the brand.
Flash is fine when tools have names. Meta-dispatch wants a model that can emit query_database
on the first try. We would rather spend ~$0.21 per million input tokens and finish in four steps than spend ~$0.07 and fail at twelve. Failed runs are not cheap. They are delayed work plus a retry.
3. Keep idle at zero.
The worker and each MCP server scale to nothing. Notion's API did not add a monthly line item. The LLM bill is the bill.
4. Put a fuse on the loop.
Steps, tool calls, wall clock. Return a structured error, not a 500 with a stack trace. Log a trace id. The caller should be able to paste that id, not a screenshot of Cloud logs.
5. Split "how to use tools" from "what to do."
This is the one we would tattoo on the repo. It is also how you keep one worker generic while still running a weekly digest.
6. Search by name. Refuse to guess ids.
Callers should not hold Notion's internal ids. If two pages match, stop. Write-back to the wrong page is worse than a failed run.
7. Do not confuse "simpler implementation" with "fewer moving parts in the prompt."
A two-tool MCP is simpler for the server. It is harder for a cheap model. Simplicity moved. We had to put it back in English.
Rough numbers we actually use as a gut check (prices move; re-check before you budget):
| Piece | Order of magnitude |
|---|---|
| Official MCP success, flash | ~40k input, ~$0.002 to $0.01 |
| Markdown tool schema vs official tool list | ~400 tokens vs ~17k |
| Flash vs V3-class DeepSeek (input / 1M) | ~$0.07 vs ~$0.21 |
| Idle Cloud Run | ~$0 |
| Notion internal integration | $0 API |
None of this is a promise. It is where the pain is.
More tools on the same worker. Slack, GitHub, meeting transcripts. The catalog should grow without a new harness. If we cannot add a server without rewriting /run
, the one-shot idea failed.
A harness that is actually seamless. Today the "harness" is still a pile of conventions: which catalog name, which model, which primer. We want a structure where a caller says the job and the worker picks a sane tool server and a sane model. Disposable should feel like calling a function, not like configuring an IDE.
A way to learn without becoming a companion. Persistent agents learn by writing skills to disk and living forever. We still want one-shot. The interesting version is: after a good run, store a primer delta or a eval fixture somewhere external. The next cold start can load it. The instance still dies.
Use it on real internal work. Weekly digest was the wedge. The point is other systems posting jobs: standups, research write-backs, "what changed in this database." If we only ever curl it ourselves, we built a demo.
Compare answers without a human reading every one. Judging the final response is the slow part now. We need a fixture shape: same prompt, same Notion snapshot (or a recorded tool trace), two models or two primers, a score that is not "the author liked it." Until that exists, we will keep arguing from anecdotes and burning evenings on single traces.
Start with a named-tool MCP and a cheap model. Prove search, read, write, and "do not guess." Then, and only then, switch the read path to Markdown and spend a little more on the model that has to dispatch.
Keep the worker boring. Make the prompt boring in the right way: how tools work, not what this week's page is called.
And if a run starts saying "there is a database," believe the stub. Query the rows. The page was never the table.