What happens when you move beyond AI-assisted coding and start handing more of the software delivery process to AI agents?
Our engineering team, which is building Syntasso Kratix Enterprise (SKE), set out to find out. We built an agentic software development workflow around our existing engineering practices, put it to work on a substantial real-world refactoring, and set ourselves an ambitious target: delivering one epic per week.
The results surprised us. AI dramatically accelerated delivery, but it also changed what engineers needed to understand, review, and ultimately own. Here’s what we learned building what we now call our daylight factory, and how we're handing off even more work to AI agents. Keep reading, and we'll even explain why we didn't call our approach a dark factory...
Motivation #
"My goal is to get this team to be able to deliver one epic per week."
That was the announcement our VP of Engineering shared, just before going on holiday for two weeks. Leadership at its finest.
As you can imagine, the team was a bit puzzled. We pair-program, follow TDD to the letter, and like to understand and discuss exactly how a piece of the codebase should behave. An epic is usually a set of 5 to 10 stories that delivers one or more pieces of functionality to the end-user. Our stories are usually sliced into one to two days of work, and most of the team spends a big chunk of their time with customers.
Just how would we deliver one epic a week? You guessed it (I mean it’s also in the title of the article). The solution for everything and anything these days, from deciding what to cook for dinner to growing healthy plants to creating workout routines and more: Artificial Intelligence. How can we rely more heavily on AI in our workflows to build not only faster, but better?
What did we (and the AI) do? #
Before handing everything over to HAL and hoping it doesn’t revolt, we wanted to build enough guardrails and processes to delay the revolution. Or, more precisely, we didn’t want to build crap code that would break in customers' hands. We set out to research and learn: what’s out there, where the industry is moving, and how to build reliable software factories.
You’d be surprised again: no one knows (or at least no one agrees). People are trying things, creating things, selling things, but there is no consensus. General trends seem to be moving toward spec-driven development, multi-agent orchestration, and building enough harnesses to make agents produce better results faster. Birgitta Böckeler’s talk, State of Play: AI Coding Assistants, gives an excellent overview of what is needed to live the dream. Greg Isenberg told us all we needed to know about how AI agents & Claude skills work.
Most people also agreed that we needed an agentic loop: a plan-build-measure-learn cycle that our agents go through to deliver software (our CEO loved it; the entire company is built on build-measure-learn loops on a Miro board). For each step in our loop, we decided to build skills and tooling to support it. We already had loads written up on our engineering practices, definition of done, and personas for whom we build… it was a matter of building the system so our AI agents could deliver using them.
The big portal unification #
“You know that big integrations refactor we keep postponing? Give us two weeks, and we’ll make it happen” is what Sapphire and I told our Head of Product to try to convince her of basically giving us free rein to build the AI engine, at the same time that we tried it out on a relatively complicated epic: a big unification of the different (Cortex, Backstage, Port) Portal integrations you can have with SKE.
“You crazy cats have two weeks”.
We were on. Calendars fully blocked for two weeks. Time to codify.
We split our skills into three layers.
Layer 1: Reference skills
Skills that encapsulate reference documentation. Includes skills like:
-
personas: who we build for, what they already know, what they would expect from the product.
-
architectural-principles: how we design software, what the tenets of Kratix are.
-
definition-of-done: when we know something is completed.
Layer 2: Practice skills
Skills that encapsulate our ways of working:
- syntasso-way: our system of work, the flow of value, feedback loops.
- engineering: how we build and deliver software (TDD, CI/CD, etc.).
- shaping: problem framing, journeys, appetite, options.
Layer 3: Authoring skills
Skills that produce or change a living artefact, the core of our day-to-day development:
- adr-writer: summarise a conversation into an ADR for human review.
- verify: the verification pipeline: behavioural checks, persona evaluators, adversarial critics vs ADRs; issues raised as stories.
- deliver-story: the main way to deliver a story end to end. Encapsulates and orchestrates most of the skills we have, invoking them when required. Includes a mechanism to self-improve.
We started using the skills. We produced an ADR; from the ADR, we generated the Epics; from the epics, the vertical slices (a.k.a. stories). For each story, we used /deliver-story to drive the implementation.
Results: AI FTW #
We did it!
We shaped the work across three epics and delivered all three in a couple of weeks. One integration is still pending, and we'll tackle it next, but we kinda proved that our VP’s dreams are possible.
Lessons learned from working with AI #
The architecture is the artefact
The lesson we learned first: the code itself is no longer “ours”, but the architecture is. Claude is excellent at throwing out ideas and generating possible designs, but we still needed to keep a firm pulse on the design direction. Left to its own devices, Claude would have produced a design that wouldn’t work for our customers. Although we tried to provide relevant persona context, it still made wrong assumptions or chose a more complicated path than we would design ourselves.
It was also an interesting exercise in knowing what context the team needs to hold. AI agents can produce code much faster than we can read, and even when we review every line carefully, it still doesn’t stick: our ability to recall what a particular function or file does is hindered. Instead, our context and understanding become deeply rooted in the system's architecture: the components and their interactions. We found that with that knowledge, we can still confidently talk about how the software works, even without working on the code ourselves.
It does speed up the process
The data says it all: a massive refactoring, folding multiple products into one, adding new functionality, delivering a brand-new pipeline stage, and updating docs. All in about two weeks. Customers started using it and quickly got their Portal integrated with Kratix. The team rapidly got up to speed with the new system. The steps in our skills actually detected a few bugs and rough edges and prevented us from shipping them. I can say with confidence we wouldn’t have delivered it all as fast as we did without this new AI-supported process.
The full machinery is not always necessary
The skills we developed can be quite slow sometimes. We realised that using the full machinery sometimes feels like feeding pigeons with slow-fermented sourdough: it surely does the job, but it’s much cheaper and faster to give them the run-of-the-mill white loaf (or better, feed them nothing, ask me some day how I feel about pigeons).
It didn’t take us long to simplify the skills, experiment with a different order of operations, and try other strategies to make it fast when we thought the change would be relatively small.
Continuous Improvement is (still) fundamental
At the end of the two weeks, we had two new pieces of software: the new portal integrations controller itself, and all the AI skills and harness we developed to deliver it.
As the team started using the skills in earnest, our velocity did go up, but lots of new issues were then raised:
- Comments are very verbose
- It makes exploratory work slower
- What does load-bearing even mean?
- One of our skills produced new issues when non-critical problems were found in the review stage; one story we delivered produced 4 new issues. In a week of using the skills we had, we created about 20 new issues; it was a bit overwhelming!
- We didn’t have a system to verify a change in the skill: deciding if something was an improvement or a regression was based on opinion.
And many more. And that’s exactly what we expected. Although the skills themselves included a self-improving loop, the human in the mix also has agency to add what AI lacks (for now): taste.
Since the end of the experiment, we have already:
- Tried different models for different types of tasks
- Run a bake-off of skills vs just Claude, and used the results to harden our own skills
- Written more skills, thrown some skills out, brought in some evals
- Had a long philosophical debate on the nature of engineering and if we can force a non-deterministic text generator to generate the exact same text twice.
But overall the experiment was a success. The team adopted it, used it, got annoyed with some of the rough edges, and started iterating and improving as we went. I’d say the skills we have today are much better than the ones we had just a few weeks ago. We are also much more knowledgeable about how they work.
What’s next in the Syntasso Daylight Factory #
We are still not setting the agents free while we sleep. We still like to observe while Claude burns tokens. And that’s why we are calling it a daylight factory instead of a lights-out factory: we want to see what is happening and understand why so we can keep improving. And we will get there.
The Gas Town future is looming. We’re building confidence in the harness and in the skills so we can release it and turn our CEO’s dream of managing fewer engineers into a reality. It’s going to be great.
In all seriousness, AI agents have helped tremendously in speeding up our development. In the last couple of months, we have produced more product features than we can really remember. And with this new power come new responsibilities (shout-out to Spider-Man's Uncle Ben): listening to our customers, prioritising work that matters, and making sure we continue to own and understand the product architecture are more important than ever.
Exciting times, and we’re looking forward to shipping all the great ideas our team has in our products, SKE and SKA__,__ to make your platform even more delightful!
If you tried a similar approach, found something interesting in here, or have tips on how to stop Claude from writing like the author of a BuzzFeed article (three bugs, and the last one is going to surprise you), we would love to hear from you! If you are building a dark factory, investigating platform orchestrators, or just looking to learn more about AI and platforms, please contact the Syntasso team. You can also learn more about Syntasso Kratix Enterprise (SKE__)__ on our website and in the SKE docs__.__