From Slack message to staged blog post — research, draft, edit, and all. Here’s how I designed and deployed Quill, and what I learned about building agents that actually ship in production.
The problem I was trying to solve #
Our DevRel team publishes content regularly: blog posts, tutorials, announcements. The process was always the same. Come up with an idea, research it, write a draft, copy-edit it, get a header image ticket into Jira, save the draft to Confluence so others can review it, then eventually stage it to WordPress. Four or five tools, a lot of context-switching, and plenty of things that could fall through the cracks.
I wanted to build an agent that could handle the whole pipeline. You drop a topic in Slack, Quill does the rest: research, draft, copy-edit, save to Confluence, open a Jira ticket for a header image, and stage to WordPress as a draft, ready for a human editor to schedule and publish.
That agent is Quill. This is the story of how I built it.
The first design question: what should the AI decide vs. what should be hardcoded? #
Before writing a single line of code, I had to think through one of the most important questions in agent design:
What parts of this workflow should be deterministic, and what parts should be non-deterministic?
If you’re new to agents, here’s a simple way to think about it:
- Deterministic workflows always produce the same output for the same input. They’re predictable, auditable, and easy to test. Think: saving a file, creating a Jira ticket, scheduling a post on a specific date.
- Non-deterministic workflows use an AI model to generate output that varies based on context, creativity, and the prompt. Think: writing a blog post, brainstorming ideas, copy-editing for tone.
Getting this split right is what separates an agent that actually works in production from one that surprises your team in bad ways.
Quill’s deterministic workflows (the “just do it” parts)
These are the parts where I did not want the AI improvising:
- Saving to Confluence. Always saves to the same space, same parent page. No creativity needed.
- Creating a Jira ticket. Always creates in the marketing project, always tagged as a header image request, always links back to the Confluence draft.
- Staging to WordPress. Always creates a draft, never publishes. Predictable, auditable, reversible.
- Finding the next available publish slot. Pure scheduling logic: Tuesdays and Thursdays at 8am PT, skip US holidays, one post per day. No LLM needed here at all.
- Checking blog coverage. Searches blog.postman.com to see if a topic has already been covered. A yes/no lookup.
These are API wrappers that run consistently. If something goes wrong, it’s easy to trace the cause. Trading for a deterministic workflow means you don’t waste tokens or money on unnecessary LLM calls.
Quill’s non-deterministic workflows (where the AI earns its keep)
These are the parts where an LLM actually adds value:
- Writing a draft. Quill calls Claude with a dedicated system prompt that encodes Postman’s voice, banned words, SEO requirements, and target length (1,200 to 1,600 words). The output will differ each time, and that’s by design.
- Copy-editing. Another dedicated Claude call with a style guide and SEO pass. Returns auto-applied changes and flags risky rewrites for a human to review.
- Blog idea generation. Parallel web searches across multiple signals, scored against five weighted criteria. Eight to twelve ideas, ranked by urgency.
Each of these tools makes its own isolated Claude API call with its own system prompt. I didn’t route everything through a single mega-prompt. That would have made the behavior harder to control and harder to debug.
Planning the APIs #
Once I knew what the agent needed to do, I mapped out the integrations. Here’s the dependency chain I was working with:
| Integration | What Quill uses it for |
|---|---|
| Anthropic API | Write draft, copy-edit, generate ideas (three separate Claude calls) |
| Tavily | Web search for research and coverage checks |
| Confluence REST API | Save drafts, read existing pages |
| Jira REST API v3 | Create header image tickets in the Marketing project |
| WordPress REST API | Stage posts as drafts, read editorial calendar |
| Slack (via Astro) | Surface as a chatbot the team can talk to |
One useful decision: I used a single shared Atlassian service account for both Confluence and Jira. One API token, one identity, write access to the drafts space and create-issue permission on the marketing project. This keeps the auth surface small.
Testing the non-deterministic parts with Postman AI Request #
At the heart of Quill are three core tools: blog_ideas
, blog-write
, and copyedit_draft
. Before wiring everything together, I needed to validate the skills and system prompts behind each one.
This is where Postman’s AI Request was really useful. I used it to test my Anthropic API calls in isolation before embedding them in the agent. The workflow looked like this:
- Write the system prompt for
blog-write
: Postman voice, banned words, SEO frontmatter, 1,200 to 1,600 word target. - Run it against a dozen different blog topic inputs in Postman to see how the output varied.
- Identify patterns where the model drifted from the style guide.
- Iterate on the prompt until the outputs were consistently in the right range.
- Repeat for
copyedit_draft
andblog_ideas
.
The key insight from testing non-deterministic workflows: you’re not testing for a single correct answer. You’re testing for a range of acceptable answers. Postman made it easy to save these prompts so you can reuse them in the future to evaluate changes, pick a different model when Anthropic announces one, and run them at volume to spot when the model starts drifting in the wrong direction.
Note:Testing deterministic tools is easy. Did it create the Jira ticket? Did it save to Confluence? Pass/fail. Testing non-deterministic tools is more like reviewing a sample of outputs and asking: “Would I be comfortable if my team saw this?” That’s the bar.
Building locally with Astro AI #
Once the prompts were solid, I scaffolded the agent using Astro AI. Astro AI is the platform I used to deploy Quill into production, but it also gives you a clean, opinionated starting point on day one.
Scaffold the project
A new agent is one command:
ast project create my-agent --model anthropic
The generated harness includes starter source code, an astropods.yml
blueprint, and two files that describe the project to coding agents:
AGENTS.md
explains the project structure, the spec format, and the conventions for writing a proper Astro Pods agent. Any coding agent (Claude Code, Copilot, Cursor, and so on) can read this to understand how to implement and extend your agent correctly.CLAUDE.md
contains Claude Code-specific instructions for working in this project.
Together, these two files mean you can hand the repo to a coding agent and it already knows the house rules. That’s a big deal when you’re iterating fast.
Configure your agent for local development
Next, set the credentials and API keys your agent needs (Anthropic, Tavily, WordPress, Confluence, Jira, and so on):
ast project configure
This walks you through each required env var and stores them so both local runs and production deploys pick them up.
Run your agent locally
Start the local development environment:
ast project start
The agent boots in a local container. I could talk to Quill in the local playground, watch the tool calls fire, and fix things before ever touching production.
The astropods.yml file
The astropods.yml
file is the blueprint for the whole agent. It’s what Astro uses to understand what the agent is, what it needs, and how to deploy it. It declares the runtime, the env vars Astro should inject (Anthropic, Tavily, WordPress, Confluence, and Jira credentials), the adapters (Slack and web), and the list of tools Quill exposes: everything from web_search
and check_blog_coverage
through the three Claude-backed tools (write_draft
, copyedit_draft
, blog_ideas
) to the Confluence, Jira, and WordPress integrations.
That file is the single source of truth. Astro reads it and knows exactly what to provision, what secrets to inject, and how to expose the agent (in this case via Slack and a web interface). There’s no infrastructure YAML to manage separately.
Pushing to a blueprint and deploying #
Once local testing was solid, shipping Quill was two commands.
First, push the local project up as a blueprint:
ast blueprint push quill
Then deploy it:
ast blueprint deploy
From there, I connected Slack in the Astro dashboard: deployed agent, Integrations, Slack, connect workspace. Anyone on the team could now drop a message to Quill in Slack and the whole pipeline was live.
You can see the full blueprint, including the repo layout and architecture, at astropods.com/postman/quill. If you want to deploy your own version, there’s a one-click deploy button right on that page.
What I can see now that it’s running #
This is the part I underestimated before I started: observability matters enormously for agents in production. When Quill goes sideways, and it does sometimes, I need to know exactly where and why.
Astro gives me tracing out of the box. Here’s what I actually look at:
- Token usage. Prompt vs. completion tokens, per request and over time. Useful for spotting when a prompt is ballooning unexpectedly, and for forecasting API costs as usage grows.
- Tool calls. Which tools fired, what they returned, and where they failed. This is the most important one. If
save_to_confluence
returned a 403, I can see that immediately. Ifwrite_draft
produced something short, I can see the raw Claude response. - Latency. End-to-end and per span. Writing a draft takes the most time (LLM call plus long output). Saving to Confluence is nearly instant. This breakdown helps me know what’s actually slow vs. what just feels slow.
- Request volume. Traffic patterns and error rates over time. How often is the team using Quill? Which requests fail most often?
- Network traffic. Payload sizes across every external call. Useful for catching cases where Quill is sending way more data to an API than necessary.
If you’re building agents and you’re not tracing them, you’re debugging in the dark. An agent with 12 tools and 3 LLM calls has a lot of surface area for things to go wrong. Tracing is what turns a mystery into a five-second diagnosis.
What I’d tell someone starting today #
A few things I wish I had internalized earlier — one of them I almost didn’t include because it sounds like a product pitch, but it genuinely changed how fast I could ship:
- Separate your deterministic and non-deterministic workflows early. The clearer that line is, the easier everything else becomes: testing, debugging, building trust with your team.
- Give each LLM tool its own system prompt. Don’t route everything through a single prompt. Isolated prompts are easier to test, easier to tune, and easier to reason about when something goes wrong.
- Test your prompts before you embed them. Use Postman’s AI Request (or something like it) to run your prompts at volume and validate the range of outputs before wiring them into the agent.
- Use a platform that handles the infrastructure. Astro AI let me focus on what Quill actually does rather than how to containerize it, manage secrets, and wire up Slack. That’s not yak-shaving I wanted to do.
- Ship with observability from day one. Don’t add it later. You’ll want it the first time something breaks in production and your team is waiting on you.
Quill is a teammate and this blog is written by both myself and Quill.
That’s genuinely how I think about it. Quill joined our team, picked up the blog workflow, and just got to work. Every time someone drops a message in Slack, Quill turns it into a staged draft without anyone touching four different tools. It has already saved us hours per post.
We’re still training it. There’s more we want to teach it about how we work, and persistent memory across sessions is coming. But that’s true of anyone new to the team. You don’t wait until someone is fully onboarded to let them add value.
What makes Quill easy to trust is that we can check its work and coach it when something’s off. It’s not a black box. It’s a teammate who’s still learning, but who shows up every single time.
If you want to bring Quill onto your team, the blueprint is public at astropods.com/postman/quill.
Resources #
- Quill blueprint on Astro AI: astropods.com/postman/quill - Astro Pods documentation: docs.astropods.com/welcome - Explore other agent blueprints: astropods.com/explore - Postman AI Agent Building: postman.com/product/ai-agent-building - Anthropic Claude API documentation: docs.anthropic.com
Built with Astro AI, Anthropic Claude, Postman AI Request, and Mastra.