TL;DR: I finished the Future AWS Agent Engineer nanodegree with three graded projects, each a customer support system for a fictional online store: one chatbot, then one agent, then five agents working together. Each project was harder to build than the one before, and each one gave me fewer problems. Here's how the three compare, what changed between them, and why the last one, which scored 120 out of 120, still got sent back for more proof.
My first project for this nanodegree went in on deadline day in August. It was a chatbot with one system prompt, graded by a rubric written for a product AWS had already stopped offering to new customers. It passed. I spent most of that day worried it didn't count, because I hadn't typed the code.
My second project went in a month later, built in one session. It was one agent that could look up orders, process refunds, and search a product catalog, with an ownership check in code so it wouldn't show you someone else's orders. It also introduced me to a database that bills by the hour just for existing.
My third project went in on a Sunday night in October. It had five agents, two models, three knowledge bases searched at the same time, and a safety filter on every model call. It went from Udacity's untouched starter files to a perfect score on its test suite in under four hours.
Each project was harder than the one before, and each one gave me fewer problems. I didn't expect that, and I wanted to understand why, so this post compares the three side by side.
The AWS AI & ML Scholars program runs with Udacity. I was in the Agent Developer track, and its nanodegree, Future AWS Agent Engineer, has three graded projects. All three are customer support systems for a fictional online store. Because the problem stayed the same, I could compare the three builds directly, each one with more tools than the last.
All three ran on Amazon Bedrock AgentCore, AWS's managed service for the parts of an agent you would otherwise build yourself: calling the model, remembering the conversation, running tools, and hosting the whole thing.
Project one was Nova Trivium, and I wrote about it in Future AWS Agent Engineer? I Didn't Write the Code. Does It Count?. A trivium is where three roads meet. One system prompt sent every message down one of three paths: file a bug report, answer from a pasted FAQ, or redirect to a human.
Project two was Nova Adiutrix, written up in Nova Adiutrix: my second agent built my first project's to-do list. An adiutrix is a helper. One agent tracked orders, processed refunds, searched a product catalog stored in a knowledge base (a managed store the agent can search by meaning instead of exact words), and refused to show you someone else's orders.
Project three is Nova Concilium. A concilium is a council. Five agents, each a specialist, with an orchestrator deciding who speaks.
Status: submission README for the nd905 Course 4 final project — complete
with the full 120/120 test run and both X-Ray screenshots recorded below. This
file is the submission README; the original Udacity starter README is preserved
in the git baseline commit 7ea6508 (unmodified starter). Account ID and
runtime ARN are intentionally omitted (redacted for public sharing).
Nova Concilium (Latin for "council") is a five-agent customer-support system for the fictional e-commerce company NovaMart, built on Amazon Bedrock AgentCore and the Strands Agents SDK. An Orchestrator agent routes each request to specialist worker agents, retrieves grounded policy answers with parallel multi-agent RAG across three Knowledge Bases, enforces a Bedrock Guardrail, keeps shared state in DynamoDB with optimistic locking, and is
observable end-to-end through CloudWatch and X-Ray. NovaMart's support team handles order status, returns and refunds, and policy questions by…
Here is how the three compare:
The row I'd read first is "How I knew it worked." The testing got stricter with each project, and the stricter the test, the more I trust everything else in that column.
At the bottom of the Nova Trivium README, I wrote a section called "Not built, and why." It listed three things I left out: retrieval, guardrails, and structured output.
Nova Adiutrix built two of them without my planning it. I noticed halfway through, rereading my old README. That post ended with this: "The third item on that 'Not built, and why' list was guardrails. Still not built. I will let you know how project three goes."
In Nova Concilium, guardrails were a graded requirement. Amazon Bedrock Guardrails is a filter that checks what goes into and comes out of the model. Ours blocks competitor talk, price haggling, and legal threats, and it masks personal details. The first sign it worked came from a test that wasn't about it. A customer's email address came back in a test transcript as {EMAIL}.
That finished the list. I didn't pick the order the items got built in. The course did.
All three builds had problems. What changed was where the problems came from.
Here is where each earlier lesson landed:
The change I'd credit most happened before any code got written.
On the earlier projects, I handed Kiro a build spec. This time I had Claude, Anthropic's AI model, write a knowledge base first: one markdown file holding every course page, the rubric, the starter repo, and a table of every place those sources disagreed with each other. There were 13. Then, before any spec, Kiro read the starter files and checked all 13. Eleven resolved against a file. The other two had no file to check: a set of names the code hadn't defined yet, and the due date.
Checking the files before writing the spec came straight out of Nova Trivium, where I planned around a starter I had never fully listed. Every value also had to name the file it came from, or get marked UNVERIFIED.
The work split three ways, the same as before. Kiro wrote the code. Claude (Opus 5.5) wrote the knowledge base and reviewed each stage of Kiro's spec against the starter files. I directed, did the Console work, took the screenshots, and made the calls.
Claude's reviews caught four problems in Kiro's requirements. The folder wasn't a git repo yet, so two checkpoints couldn't run. One test was scheduled before the deploy it depends on. The guardrail tests needed example prompts that no file supplied, so Kiro would have had to make them up. And two agents could only be checked after the deploy, so Claude suggested checks that could run earlier.
The first plan had Kiro stop after every single task and wait for my approval. Early on, I told Claude to stop over-constraining the build. I appreciate thoroughness, but I needed to keep moving.
We cut the stops down to four, the moments that needed me or cost money: before I created the knowledge bases in the Console, before the first deploy, before any git push, and before teardown. Everything else, Kiro ran straight through, pasting output as it went.
The whole run from bare starter files to a perfect score happened after that change, in under four hours. I think the fewer stops are a big part of why it went that fast.
One of the three graded test scenarios asks: how much are 5 items at $29.99 with 10% off?
The exact answer is $134.955, so it has to be rounded. In the latest run, the orchestrator worked it out as $134.96. Then it handed off to the Communication agent, which rounded the discount to $15.00 first and wrote $134.95. The final reply said $134.95. In an earlier run of the same test, the final reply said $134.96.
The rounding rule in the starter code sits in the orchestrator's instructions. The Communication agent, which writes the reply the customer sees, rounded early anyway. As far as I can tell from the logs, the orchestrator's math never made it into the shared record the agents read from, so the Communication agent did the math again its own way.
The test passed anyway. When we wrote the acceptance check, Claude pointed out that the answer depends on rounding mode and suggested accepting either $134.95 or $134.96, since the graded behavior was that the guardrail didn't block a math question. I agreed. Accepting either answer fit the rubric, and it's also why a one-penny disagreement between two of my agents got through every test. Nobody noticed until I read the logs for this post.
This is my second penny this month. In Eight Minutes to Live. Was It Cursor, or Was It the Spec?, a tip calculator I built with Cursor split a 1,437.50 bill three ways into 479.17 each, which adds up to 1,437.51. The extra cent floated because my spec never picked a rule for it.
The comments on that post did more for me than the post did. flatin.pt suggested keeping money in whole cents and handing out the remainder one cent at a time in a fixed order, so the shares always add up. Matt Busel suggested treating the spec as a hypothesis instead of a contract, and updating it every time a check finds a gap. Vlad Zoff suggested keeping the acceptance checks separate from the build, so whoever builds the thing can't redefine what it's judged against.
Vlad's point lands closest to this penny. Loosening the check to accept either answer was a reasonable call for the rubric, but it was still the people building the thing changing the test. Both pennies came from the same gap: in the tip calculator, no rule said who gets the extra cent, and in Nova Concilium, no rule said whose number is the final one. If I follow Matt's advice, the next spec names one agent as the owner of any calculation and has it save the result to the shared record before anyone writes to the customer.
I submitted Nova Concilium that night. The reviewer sent it back less than an hour later.
The feedback was kind. It said the code was a good example of modular, well-documented agent code. Then it asked me to fully deploy my solution, including all the Knowledge Bases and the AgentCore deployment, and to include screenshots of each. Everything it asked for was already deployed and running in my AWS account. The reviewer had no way to see it.
The written deliverables were the code, the knowledge bases, the populated config file, a screenshot of the test score, and a screenshot of the X-Ray service map, which draws every request's path through the agents.
I submitted every one of them. No page asked for Console screenshots or test-run output.
The gap was still mine to catch. The original requirements Kiro wrote said every checkpoint's output gets saved to an evidence folder. When we sped up, the results went into the README as text. Claude didn't flag it in review, and I approved the submission without checking that line. Human approved means the miss is mine.
There's a second lesson in that miss. The rubric says the knowledge bases were "created in the AWS Console" and the system was "deployed to AgentCore Runtime." A reviewer can't log into my account, so for those two lines, a screenshot is the only proof they get. My checklist asked whether each thing was done. It needed to ask whether someone outside my account could see that it was done.
I added the Console screenshots and the test-run output and resubmitted. It passed, and on October 27 the certificate arrived.
Before my course access ended, I had Claude help me turn every lesson into notes: four markdown files, one per course. I wanted to take the knowledge with me, into future builds, instead of losing it when the classroom closes.
The first post in this series asked whether directing agents counts as learning, when I never typed the code.
Three projects in, I can name what I learned. I learned where the knowledge lives, where the rules get enforced, and how to write a check that an agent can't talk its way past. On the last project, I learned that a perfect score and a passing submission are different things.
The difficulty went up every time. The building got easier every time. The lessons table is my evidence for why. Seven lessons, each one learned when something broke, and each one built into the next project as a check. That's the skill these three projects gave me: making the next build break in fewer places. I would like to hear from anyone who has built the same thing three times and seen something else.
The certificate says it counts. And..so do I.
Quick context if you are new here. I work in the California courts, running court operations for the county. I started building with AI in July 2025 and I have been learning in public ever since. I do not write the code. I direct, the agents generate, I validate and decide. I build the Clew Suite, a set of civic tech tools for making complex systems easier to inspect. That is the lens I am writing from.
Built with AI assistance in October 2026. Kiro wrote the code, and Claude (Opus 5.5) wrote the knowledge base, reviewed the spec, and helped draft this post. All three repos are public, so check them before you quote me.
AI Assisted. Human Approved. Powered by NLP.