We ported 65 OSS projects with AI

Lightbend, the company behind the Akka SDK, announced it used AI models Claude Sonnet and Opus with its Akka Specify toolkit to port 65 open-source software projects in 99.3 hours, with 57 of the ports showing improved lines of code or performance. The company reported that a legacy RAG system with 200,000 lines of code was reimplemented in 48 hours, reducing to 9,000 lines and improving eval accuracy from 64% to 95%. Lightbend attributed the gains to the Akka SDK's opinionated structure and the delivery harness, which had a bigger impact on consistency than model capability.

We Ported 65 OSS Projects With AI We created a self-improving, spec-driven delivery harness to test whether frontier models can port complete systems. TL;DR - Complete, unattended rewrites of existing systems that pass the original's unit and integration tests are possible. - Low effort models were more efficient than high effort models. - The delivery harness and its structure had a bigger impact on consistency than model capability. - We saw a LOC or performance improvement on 57 of 65 ports. We see a potential for AI to write and maintain entire systems without human involvement. In March, we released Akka Specify https://doc.akka.io/reference/specify/index.html , a spec-driven delivery toolkit for creating new systems with the Akka SDK https://doc.akka.io/sdk/index.html . It takes an opinionated approach to defining and implementing an AI system—inclusive of APIs, orchestration, agents, streaming data, and stateful memory. We observed a few things after the release of Akka Specify: - Our solution architects were completing POCs in hours instead of weeks. - Our account executives were shrinking delivery times by many months in customer proposals. - A customer upgraded a codebase written in Kalix to Akka in a few hours. - We redesigned a legacy RAG system with 200K lines of code generating 64% eval accuracy reimplementing it in 48 hours shrinking the LOC to 9K while improving eval accuracy to 95%. We attributed these gains to the dramatic improvement in intelligence with frontier models, the improved reasoning structure that comes with AI harnesses, and structural constraints baked into the Akka SDK. The Akka SDK has an opinionated structure that imposes one way, and only one way, to do each thing, such as how one service should reliably invoke another regardless of its location. These restrictions enable us to provide a guarantee that if the AI system compiles and passes tests locally, then it's production-ready without further code modifications. Akka embeds its clustering, elasticity, and resilience runtime on every build, producing a full-stack system while not intermingling resilience and scalability concerns within application code. These SDK opinions have made Akka particularly well-suited to leveraging AI to rapidly generate complex, production-ready backend systems. In a world where the hyperscalers, Palantir, C3.ai, and the frontier model providers are building sizable Forward Deployed Engineer FDE armies, we wondered: why can't AI write and maintain entire systems without human involvement? So we put it to the test. We rewrote 65 open-source software projects using Sonnet, Opus, and Akka Specify in 99.3 hours total. The Two-Tranche Approach We did this in two tranches: Span: A pass across all 65 projects that would create a fully mapped specification and implement a slice of up to 10% of the project's surface area. We did this on OSS projects that were both well-suited to an Akka port and those that were not Go libraries, Rust modules, systems that didn't have a need for backend services . Deep dives: Complete implementation of all of the project's use cases for those projects where we identified places where Akka could have a significant positive impact: systems with a Python backend, limited persistence, orchestration, scheduling, streaming or distributed fundamentals, feasible scope, and a baseline of a quality that could be adequately measured before/after. Unless stated otherwise, all tables and graphs are the results from the 65 initial ports done together and do not include the additional data from the 10 fully completed ports. While dramatically increasing the total tokens consumed, there were few places where the trends in the completed 10 differed from the trends in the intial tranche of 65 ports. Our Experimental Goals Specification structure: Akka Specify takes a specification as its input. We wanted to explore whether specifications should have a structure, schema, and maybe even typed behavior so that systems can more reliably be first-pass generated. Context file gaps: Akka Specify uses the Akka context files for creating a design and guiding an implementation. We wanted to identify and hopefully close gaps within those files that cause faulty initial implementations of greenfield systems. Exit Conditions: Delivering a "system" into production is more than code. It's also the harnesses, testing, and organizational requirements that must be satisfied. Akka Specify can enforce Exit Conditions that must be proven by pre-release auditors. We wanted to incorporate best practices from different OSS projects into our list of Exit Conditions and auditors. Runtime performance: We wanted to see if the Akka runtime could improve the performance of various OSS projects, and whether embedded durable execution in Akka could create higher forms of resilience for some projects. Model and effort impact: Whether there was a significant impact in terms of time to complete, accuracy, and token cost when switching between different Anthropic models and levels of effort. Summary Metrics Table 1 Totals and extremes of the 65 initial tranche | Metric | Value | |---|---| Total time | 99.3 hours | | Mean time per port | 92 min median 76 | | Fastest port | | activepieces/activepieces https://github.com/TylerJewell/activepieces-akka — 292 min opus/high agno-agi/agno https://github.com/TylerJewell/agno-akka — 0.08:1 1,524 → 127 getredash/redash https://github.com/TylerJewell/redash-akka — 2.17:1 542 → 1,176 langgenius/dify https://github.com/TylerJewell/dify-akka — 143,333× faster ⚠ Netflix/metaflow https://github.com/TylerJewell/metaflow-akka — 0.01× 100× slower Table 2 Summary of the 10 fully implemented ports | Port | Model / effort | Tokens | Steps | Time | LOC original | LOC port | LOC ratio | Perf original | Perf port | Perf ratio | |---|---|---|---|---|---|---|---|---|---|---| | ActivityWatch/activitywatch https://github.com/ActivityWatch/activitywatch glanceapp/glance https://github.com/glanceapp/glance lfnovo/open-notebook https://github.com/lfnovo/open-notebook dgtlmoon/changedetection.io https://github.com/dgtlmoon/changedetection.io linkwarden/linkwarden https://github.com/linkwarden/linkwarden umami-software/umami https://github.com/umami-software/umami getredash/redash https://github.com/getredash/redash louislam/uptime-kuma https://github.com/louislam/uptime-kuma permitio/opal https://github.com/permitio/opal Steps. The number of agent-LLM interactions in order to complete a port. LOC ratio. Port divided by original, scope-matched; below 1× the rebuild shipped fewer lines for the same behavior. Perf ratio. Original divided by port, in each report's own units, so a row is self-consistent but two rows are not comparable; above 1× the rebuild is faster. The Methodology We ported 65 OSS projects that were considered popular large numbers of GitHub stars, strong community interaction online, and had a backend . We picked OSS projects in three different flavors: Popular AI OSS projects —particularly those that focused on context, memory, and orchestration. Many of these projects are anchored in Python without durable execution, and we expected to see significant performance gains. Some frameworks chosen were direct competitors to Akka. We also included some popular AI harnesses like DeepSeek's new plug-in-only framework. Popular networking libraries —particularly those that were built in Go and Typescript. We expected overlap with Akka concepts, and Akka could offer a different structure to such concepts and would likely cause a performance degradation. We wanted to see how well AI would transform a library into a framework where concepts are similar, but the interface would deviate. Popular SAAS OSS projects —with rich, complex Web-based user interfaces. These systems have structured APIs which should be well-suited to doing a complex port. We licensed the code in every port to be Apache licensing, except where we copied over specific GUI interfaces verbatim, and those were licensed however the original project licensed the code with appropriate attribution. The Delivery Harness We created a delivery harness that would run in a self-improvement loop. Setup: We will create a new repository, setup observability, and ensure all discovery and build tooling is installed. Discovery: We execute code analysis, domain model analysis, schema analysis, and runtime interrogation of the existing system in order to generate a series of specifications that define the behavior and conditions of the system itself. Port: We define a series of exit conditions derived from the specification and then leverage the Akka Specify planning, tasking, implementation, building, and testing capabilities to author the system with Akka best practices, and then execute the Akka review to ensure that it matches the design standard expected. Benchmark: We execute both systems with a single runner that measures successful execution of all test suites, counts lines of code and assess end user latency performance. We draft a README and push up to the GitHub repository. Improve: We record what failed in discovery or areas where the implementation had to deviate from the specification, and then speculate on techniques we can use to improve the discovery engine or specification standard for the next iteration. We would rewrite any API, CLI, and MCP surface adhering to the original's design as closely as possible. For any project with a Web GUI, we brought that GUI over wholesale, but modified the way the GUI accesses endpoints to take advantage of Akka server-side events and other streaming capabilities. We did not port mobile, Slack, WhatsApp, or other types of app interfaces that would require human configuration to properly test for validity. After completing a detailed discovery phase, AI would generate a detailed specification and implementation plan. Claude was required to use Akka Specify for the implementation leveraging Akka components with a focus on: - Minimizing total LOC. - Eliminating functional redundancies. - Avoid using non-determinism when unnecessary ie, an agent for the sake of an agent should be avoided . - Optimizing for performance such as using CQRS to separate writes from reads . - Leveraging the embedded Akka persistence layer of event-sourced entities and key-value entities, workflow for orchestration, views for read models, and durable state replicated across nodes before attempting to import an external database, graph store, or vector store. We were pleased to find that after all ports had finished, none of the projects needed to import an external persistence engine. The port was only considered complete and successful if the source system's unit and integration tests would pass when run against the new Akka system. Additionally, Claude was required to run /akka:review after every port which executes a variety of design, performance, and code structure auditors. Claude was required to resolve every identified issue: Table 3 The auditors every port had to pass | Auditor | What it checks | |---|---| | Serialization & state integrity | Ensures persisted data can always be read back correctly across restarts and deployments. Covers stable type-name labels, pure event-replay functions, and safe entity initialization. | | Endpoints & security | Confirms endpoints are declared correctly and don't leak sensitive operations to the public internet. Access-control rules must be present and not overly permissive. | | Workflows | Checks that workflows use the current APIs, keep step vs. command return types straight, and give AI steps enough time to finish. Prevents silent breakage and constant 5-second timeouts. | | Agents | Agents must be stateless so concurrent requests don't corrupt each other. No fields holding data between calls. | | Views | Verifies read-model wiring: right annotation placement, right handler for the source type, correctly wrapped list results, and no null fields. Wrong wiring means the view silently never populates. | | Error handling | Errors must use the framework's supported types so they can travel across nodes cleanly. Custom exception classes must be static to serialize. | | Payload & state size | Enforces 1 MB limits on payloads, state, and events, plus 1 KB on timer inputs. Bigger data breaks replication and stalls the entity — store large assets externally by reference. | | Code quality & safety | Bans blocking I/O in handlers, shared mutable state between components, and hardcoded secrets. Each causes stalls, race conditions, or leaks. | | PII & data sanitization | Personal data must not appear raw in logs, error messages, endpoint responses, or LLM prompts. Sanitize or omit before it leaves the entity. | | Serialization conventions | Sensible defaults for serialized types: name sealed-interface variants, use Optional for missing fields, return new records for state changes, and declare all Protobuf event types. Keeps schemas evolvable and readable. | | Architecture conventions | Enforces a clean 3-layer DDD structure with framework-free domain code, business logic in domain objects, and standard naming for entities, commands, events, and views. | | Endpoint conventions | Every endpoint declares access rules, returns API-specific types not raw domain , and uses the sync .invoke style with the HttpResponses helpers. Access context comes from requestContext . | | Workflow & agent conventions | Failing steps have compensations, AI steps have retry caps and long-enough timeouts, transitions use method references, and each agent has an intentional model, memory, and error-fallback choice. Keeps workflows predictable and AI costs under control. | | Consumer & idempotency | Ensures duplicate deliveries and retries don't cause double-effects: dedupe tokens on mutating commands, bounded dedupe state, deterministic tokens on external calls, and infallible compensations. Also: no fat events, load big assets just-in-time, enable sanitization when PII is present. | | Testing conventions | Uses the right testkit per component type, Awaitility for async view projections, httpClient for endpoint integration tests, and TestModelProvider to avoid real LLM calls. Integration test class names end in IntegrationTest. | | Error handling conventions | Return values from component calls aren't silently discarded, CommandException is caught and mapped to proper HTTP responses, and unexpected exceptions are handled instead of leaking as bare 500s. | | Design review | Higher-level design observations: no hot/god entities, right entity granularity, bounded state, right-sized events, parallel workflow steps, appropriate use of workflows and views, no deep sync call chains or circular dependencies, and clear aggregate boundaries. | The conversion must run human unattended. During the analysis phase of any source system, if there was any ambiguous functionality or behavior AI was instructed to get clarity by running the source system's integration tests. Eventually, in trial runs, Claude invented a form of adversarial testing that it would use to determine whether ambiguous behavior could be deterministically applied, or whether it was indefinitely non-deterministic. Claude was instructed to vary its choice of models and efforts randomly. After a few of the initial runs consumed a lot of time on Opus xhigh, we removed xhigh from the consideration list for the remaining runs. Each resulting port was placed into a GitHub repository and given a detailed README with instructions on how to use it, how to re-run the unit and integration tests, and how to rebuild it from scratch using Akka Specify. The full list of projects we converted is in the Appendix. Answer parity measures how often the port returned the same result as the original on the same input. Ports cluster at 100%, and the ports below it are the ones whose remaining gap is recorded in the port's own benchmark report. Analysis Model and Effort Performance We expected Opus to be quicker as it should get more of the analysis right on the first pass. But what we found was that effort was a better barometer of speed. Since we added guardrails over time, we could ensure that there was adequate output accuracy ie, the project wasn't done until all unit tests passed regardless of model and effort. When taking into account efficiency an aggregate score of total round trips to the model, total wall clock time to complete, and total tokens consumed , both Opus and Sonnet showed similar efficiency until they hit xhigh effort. But, while they were largely the same efficiency, you still paid more total tokens for higher efforts than you did for lower efforts. Complexity-Adjusted Efficiency The dataset was large enough where we could predict the efficiency that was expected given the total complexity of the original project. Complexity here is measured by the total LOC of the original project, using that as a cheap predictor of total complexity. When conversions were efficient, they were anywhere from 50-90% more efficient than expected, but when they were inefficient, they had some significant outliers where they could be 150-300% less efficient than expected. LOC and Performance by Project Category It seems that the type of application that was ported was a predictor of whether Akka could improve its LOC and its performance. Big SAAS applications and large complex frameworks saw significant performance boosts due to Akka event-sourced entities, embedded durable execution, and in-memory durable data. Table 4 Median LOC and performance change, grouped by what the project is | Category | Ports | Median LOC ratio | LOC change | Median perf. ratio | Perf. change | Mean parity | |---|---|---|---|---|---|---| | Application | 32 | 1.00× | +0% | 5.60× | +460% | 96.80% | | Framework | 11 | 0.57× | −43% | 6.30× | +530% | 98.05% | | Library | 14 | 1.06× | +6% | 3.10× | +210% | 98.82% | | Infrastructure | 6 | 0.74× | −26% | 0.53× | −47% | 97.56% | | Tool/CLI | 2 | 0.75× | −25% | 0.53× | −47% | 100.00% | LOC ratio. Port lines divided by the lines of the source slice it replaced; below 1.00× the port is smaller. Performance ratio. Source latency divided by port latency; above 1.00× the port is faster. Parity. Share of benchmark cases where the port returned the same answer as the original. Guardrails and Efficiency Over Time Over time, after each port was completed, the engine would self-assess whether there were additional checks that it should do when working on the next port. These would show up as additional auditors and guardrails. And, as you'd expect, as you add more exit conditions, auditors, and guardrails, the ports become less efficient. Steps vs. Tokens It wasn't obvious to us before running the experiment, but after looking at the data, it does seem intuitive that younger, smaller models need more round trips "turns" to go through all of its analysis and work before it signs off on the final outcome. You can see from the data that, generally, sonnet needed a lot more turns than opus, and these extra turns typically led to more tokens being consumed in aggregate. Since the cost per token is cheaper in sonnet vs. opus, this may be a moot point in terms of total cost of a port, but for those who are hosting their own models with spare GPU to execute more inference, the extra turns of a smaller weight model may be worth the cost. Performance Outcomes We had hoped to show that the Akka runtime and durable execution engine introduce significant performance gains for each project. That happened in quite a few places, but there were significant degradations in others. We expected to see Akka abstractions create smaller code bases, but we were surprised to find that the amount of shrinkage could be predicted based upon a project's original LOC size. Residual is the gap between what a port cost and what its original size predicted it would cost. These ports came in furthest under that prediction, and every one of them ran on sonnet at low effort. Table 5 Ports that came in furthest under their predicted cost | Port | Model / effort | Residual | Efficiency | Original LOC | Turns | Minutes | Tokens | |---|---|---|---|---|---|---|---| | agno-agi/agno https://github.com/agno-agi/agno apify/crawlee https://github.com/apify/crawlee umputun/remark42 https://github.com/umputun/remark42 assafelovic/gpt-researcher https://github.com/assafelovic/gpt-researcher Efficiency score. Geometric mean of the port's turns divided by the median turns across all ports, its active wall-clock minutes divided by the median minutes, and its tokens divided by the median tokens. Unitless. A score of 1.0 is an average port; lower is more efficient. These ports ran furthest over the cost their original size predicted. Four of the five ran on opus, and the two most expensive ran at high effort. Table 6 Ports that ran furthest over their predicted cost | Port | Model / effort | Residual | Efficiency | Original LOC | Turns | Minutes | Tokens | |---|---|---|---|---|---|---|---| | MemTensor/MemOS https://github.com/MemTensor/MemOS louislam/uptime-kuma https://github.com/louislam/uptime-kuma teableio/teable https://github.com/teableio/teable AsyncFuncAI/deepwiki-open https://github.com/AsyncFuncAI/deepwiki-open Residual. The port's efficiency score minus the score an ordinary least-squares fit predicts from the original slice size. A positive residual is a port that cost more than its size accounts for. The ports that shed the most code all replaced source that carried its own orchestration, persistence, or transport layer. Table 7 Ports that reduced LOC the most | Port | Model / effort | LOC saved | Source slice LOC | Port LOC | LOC ratio | |---|---|---|---|---|---| | teableio/teable https://github.com/teableio/teable plastic-labs/honcho https://github.com/plastic-labs/honcho permitio/opal https://github.com/permitio/opal agno-agi/agno https://github.com/agno-agi/agno Source slice LOC. Lines in the part of the original the port reproduces, rather than lines in the whole project. Pairing a port against a whole project reads as a claim the ratio never made. The ports that grew all replaced small source slices, where the record and state declarations Akka requires are a larger share of the total. Table 8 Ports that increased LOC the most | Port | Model / effort | LOC saved | Source slice LOC | Port LOC | LOC ratio | |---|---|---|---|---|---| | novuhq/novu https://github.com/novuhq/novu infiniflow/ragflow https://github.com/infiniflow/ragflow glanceapp/glance https://github.com/glanceapp/glance linkwarden/linkwarden https://github.com/linkwarden/linkwarden LOC saved. Source slice lines minus port lines. A negative figure is a port that is larger than the slice it replaced. When you rebuild a program on Akka, two things happen at the same time, pushing the LOC count into different directions. Why LOC Shrinks With Akka A lot of what's in a big open-source project isn't the actual feature—it's the plumbing. This includes things like: - Threads that pass work around so nothing gets stuck. - Locks that prevent two things from stepping on each other. - Files-on-disk to remember what happened last time. - Little safety checks to make sure the data isn't garbage. Akka comes with all of that built in. So the more plumbing capabilities that were in the original, the better Akka would do. Why LOC Increases With Akka Akka asks you to be explicit about a few things that little scripting languages let you fudge. In other words, Akka depends upon Java which is strongly typed and you can see a long history of writings from different vendors on the benefits and consequences of typing . In a small Python or JavaScript project you write one line that mutates an object and it just works. In Akka you write a SomethingHappened record, a SomethingCommanded record, a SomethingState record, and a bit of code that says "when this thing happens, the state goes from A to B." For big projects that's a rounding error; for tiny projects that structure ends up as a lot of unnecessary overhead. Performance Gains The ports that gained the most speed all replaced source that reached a database or an event loop on every request. Table 9 Ports that improved performance the most | Port | Model / effort | Performance ratio | |---|---|---| | permitio/opal https://github.com/permitio/opal QuantumNous/new-api https://github.com/QuantumNous/new-api pocketbase/pocketbase https://github.com/pocketbase/pocketbase linkwarden/linkwarden https://github.com/linkwarden/linkwarden Performance ratio. Source latency divided by port latency on the same benchmark. dify. The 143,333× figure compares a cached in-memory read against a source path that re-ran an LLM call, so it measures two different amounts of work. The ports that lost the most speed all ship something the source did not: durability, an HTTP surface, or a boundary between components that the original crossed in-process. Table 10 Ports that worsened performance the most | Port | Model / effort | Performance ratio | |---|---|---| | knadh/listmonk https://github.com/knadh/listmonk openstatusHQ/openstatus https://github.com/openstatusHQ/openstatus infiniflow/ragflow https://github.com/infiniflow/ragflow deepstreamIO/deepstream.io https://github.com/deepstreamIO/deepstream.io Performance ratio. Below 1.00× the port is slower than the original. A ratio of 0.01× is a port that takes 100 times as long. We believe that these elements cause Akka to offer a performance improvement: Your data is in-memory and durable, persisted transparently without a developer coding database semantics. Most projects hit a database on every request—issue a SQL query, wait for the disk, parse the result, close the connection. In Akka, the "row" you'd query is an in-memory object called an entity. A request that reads or updates one entity is a plain field access. That single change turns a millisecond-per-query database call into a microsecond field read. Most of the biggest speedups in the data—1,000×, 350×, 240×—are this: the source was hitting SQLite or Postgres per request, whereas Akka was reading from memory. Writes are durable without being slow. Every change is recorded to an append-only event log so nothing survives a crash gets lost—but the recording happens asynchronously and batched, off the request path. The caller gets its answer while the durability write happens behind the scenes. You get "safe" for the price of "fast." No locks or contention. Each entity is single-threaded: only one message at a time, in order, no shared mutable state between requests. Most fast-looking Python and Node code has hidden costs from asyncio.Lock, Mutex, and defensive "check-then-act" guards. Akka single-writer entities make all of those unnecessary. Streaming is back-pressured and non-blocking end to end. Data flows through the system without buffering and without threads blocking on each other. Throughput is limited by the slowest stage rather than contention. Services are clustered from within. When you scale from one node to a hundred, the same code runs—messages route to whichever node holds the entity, and there's no database in between coordinating anything. Most projects reach for Redis or an external message queue to scale; Akka has that inside the runtime. JVM JIT. After a warmup pass the hot paths run at essentially native speed. Interpreted Python and Node can't catch up on tight inner loops. And there doesn't seem to be any correlation to whether improving LOC would cause a performance increase or not. About Spec-Driven Delivery Spec-driven delivery SDD is the process of using specifications to define a system and use it as a foundation for its delivery. A side-effect of SDD is the avoidance of technical debt and loss from tribal knowledge migration. When implemented with appropriate guardrails, an SDD system can ensure that it maintains itself against all current best practices, avoiding the introduction of technical debt, and further, no project tribal knowledge is lost when a team member migrates to another project or someone new who is unfamiliar with the project starts to engage. If the specifications are complete and the engine which generates systems from them comprehensive, then knowledge is captured and system maintenance is mostly automatic. There are a number of spec-driven development open source toolkits available. Akka Specify is based upon speckit https://github.com/github/spec-kit . We've extended it by: Structured Projects We impose an opinionated way that every project must be structured. This structure incorporates everything needed to manage code, tests, evaluations, red teaming, documentation, specifications, and harness configuration. Environment Validation A development environment has pre-requisites that need to be installed and properly configured. Some of those are Akka dependencies, and others are defined by an organization. We provide setup and configuration that is AI-aware so that any laptop can be readied for any kind of project. Build & Deploy Build, test, and deployment are Akka-aware. This includes an understanding of the build tooling, dependencies analysis, running unit and integration test harnesses, and executing microservices locally within clusters. It automates the setup of any remote dependencies or mocks that may be needed such as calling into 3rd party APIs or other microservices that are running in a staging environment rather than locally. And, if you are not tied into CI/CD, it knows how to authenticate against an Akka platform environment, package a service as a Docker image, deploy it, set up routes, and validate that it's accepting traffic. Self-Documenting Akka documents the domain model, interaction model, entity model, schemas, and component hierarchy. Akka Context Akka Context is a local, offline mirror of the Akka documentation, core concepts, and principles including theory tied to distributed systems, real-time streaming, brokerless reliable messaging, operations concerns, production readiness, and architecture design considerations. Exit Conditions & Auditors Akka Specify can optionally be executed in an enforced mode, which maps every rule or requirement in a specification to an auditor that executes within an AI harness and must successfully pass before returning to the user. Exit conditions can enable a higher level of guarantee on quality, but adding more exit conditions leads to more auditors which can extend build cycles. Exit conditions have to be satisfied as part of the initial build, before a project is review ready, or before a project is releasable. Enterprises can enforce their own exit conditions, and finding the right balance of conditions that must be satisfied on every build vs. satisfied before the final release is challenging. We have built a large repository of conditions that we use on various projects. Table 11 Exit conditions by type and category | EC type | Category | What it asks | |---|---|---| Productdoes it work? | Functional | Does the software work? | | Data integrity | Is data consistent and correct? | | | Reliability | Is it dependable and fault-tolerant? | | | Performance | Is it fast and resource-efficient? | | | Security & compliance | Is it protected and regulation-conformant? | | | Operability | Can it be run, observed and managed? | | | Experience | Is it usable and satisfying? | | Projectis the codebase healthy? | Documentation & training | Can knowledge transfer to others? | | Repository hygiene | Is the codebase clean and organised? | | | Pipeline & scanning | Do CI/CD and automated checks pass? | | | Code & test health | Quality metrics and test coverage | | | Environment & dependencies | Config and library management | | | Enterprise integration | System interoperability | | | Infrastructure & deployment | How it is provisioned and shipped | | Processwas the build honest? | Process integrity | Did the AI build stay honest and transparent? | | Build efficiency | Development speed and resource use | | | Content & brand governance | Standards compliance | | | Agentic AI safety | Safe autonomous agent behavior | A Specification Standard Most spec-driven development toolkits are indifferent to the structure of the specification. We've extended speckit https://github.com/github/spec-kit to accept an Akka Specification which has a structure that leads to better requirements creation and faster authoring. The more structure we give to a specification, the faster a delivery cycle can complete. Table 12 The sections of an Akka Specification | § | Section | What it holds | |---|---|---| | 1 | Scope | What the rebuild covers and what it explicitly excludes, each exclusion with a reason | | 2 | Domain model | The nouns and their relationships. opal splits this into Policy, Data, Identity/scopes, Fleet | | 3 | Deterministic contract | The numbered rules, grouped by capability | | 4 | Open decisions | Places the original has no settled answer or gives two; the port picks one and says so | | 5 | Conformance | A table binding every rule to source and test | Discovery Most systems, even though being ideated from scratch, are based upon an existing system. Being able to understand an existing system and derive specifications that AI can understand accelerates delivery. No single technique is sufficient for discovery. It is a collection of techniques that span deep analysis of the existing system, identifying entry points, finding dead code, tracing imports and their usage, comparing ported systems with original systems, chaos engines, adversarial testing, code analysis, schema analysis, file analysis, delivery ordering, evaluation, input analysis, temporal analysis, experimentation, and so on. This effort was, in part, designed to help us advance the intelligence of our discovery tooling. We studied all of the situations where the porting harness started looping due to mistakes or issues in the specification. Many improvements were automatically added into the harness by the harness itself. But, after discussing it internally, we identified five additional things that we will consider adding to our discovery engine to improve specification quality. Interface enumeration. Extract the source system's complete interface — routes, flags, config keys, env vars, scheduled jobs, event types, migrations — into a manifest before specification begins, and require every entry to resolve to a spec rule or a recorded out-of-scope decision. It identifies interface surface the specification has not accounted for, in either direction. Test-suite ingestion. Parse the source project's own test suite, plus its closed issues and release notes, and treat each assertion as a candidate rule requiring a mapping or an explicit rejection. It identifies behavior the original's authors considered contractual that the specification does not state. Copy-time provenance. Instrument the pipeline's copy path so that taking a file from the clone appends its destination, source, and content hash to the manifest as a side effect. It identifies reused material, and distinguishes files shipped byte-for-byte from files copied and then modified. Differential oracle. Run the source system alongside the rebuild behind a proxy that sends each request to both and compares the responses, active throughout construction rather than only at measurement. It identifies behavioral divergence at the moment it is introduced, and — once fed generated inputs — the cases that weren't initially considered. Adverse-condition observation. Degrade the running original on purpose: withdraw its dependencies one at a time, control its clock, and issue concurrent conflicting writes. It identifies error handling, retry and scheduling semantics, and concurrency behavior, none of which appear during normal operation. Lessons Learned We had a few goals when we set out on this experiment. Here is what we found. 1. Specifications Must Be Structured Data Specifications should have structure, schema, and typed behavior, unambiguously. Every port that ran through the harness produced a spec before writing any code. What we watched across all iterations is that the structure of the specification determined whether a first-pass implementation was correct. Ports whose specs enumerated their claims, cited them to evidence, and typed their answers this input produces that output; this state transitions to that state generated code that matched behavior on the first pass. Ports whose specs were prose—"the system should handle X gracefully"—generated code that was plausible and wrong, and the errors surfaced two to four steps downstream when the port was already committed to a design or a structure. The rework cost of the second kind is measurable. The most expensive ports in the run—the ones that spent 250+ minutes and burned 400M+ tokens—were disproportionately ports whose early specs left decisions implicit. We added a rule that "any claim about a class needs a list" midway through the conversions and every port after that paid slightly more per iteration but had fewer turns before getting to accuracy and compliance. 2. The Akka Context Files Have Gaps Across all ports, the model's initial implementation choices were high-quality on the components the Akka context files describe in detail—entities, workflows, endpoints, views—and low-quality on the choices between them. Which effect goes in which component, whether a workflow should orchestrate or a consumer should react, when a KVE is enough and when an event-sourced entity is required—these are the questions where first-pass code was most often rebuilt in the review step. 3. Exit Conditions Capture What "Shipping" Means Exit Conditions do capture what shipping a system means, and a structured harness can act as the record of what those conditions ought to be. The delivery harness that we generated that standardized how a port should work—setup, discovery, port, benchmark, improve—is the closest thing we have to a portable definition of "system delivered." Every guardrail we added mid-run sweep leftover processes, refuse an empty screen, close the rendering gate, warn on a hung tool at four heartbeats turned into an Exit Condition that needed to be audited on a future project. This reinforces our thinking around providing an enforced mode for Akka Specify where we allow enterprises to define exit conditions from across a variety of different dimensions, integrate the auditors with internal tooling that knows how to measure these conditions, and to enforce them as part of the project's build and test cycle. 4. The Akka Runtime Delivers on Performance and Resilience Across the ports where we could time the same slice of work on both sides, the port is dramatically faster whenever the original relied on a database, a language interpreter, defensive locking, or hand-rolled scaffolding—which is most of them. In-memory entities beat SQL queries. Single-threaded actor mailboxes beat locks. Async journal writes give durability without paying for it on the request path. The measured range—from ~1.3× on already-fast Java originals to 1,000×+ on database-bound or event-loop-bound originals—is entirely explained by how much plumbing the original was carrying that the port doesn't need. Where the port looks slower on the benchmark, the cause is nearly always that the port ships things the source didn't: durability, HTTP surface, or cross-component boundaries. You could argue that this isn't a fair port, that we are porting an apple and turning it into a turnip, and that it is pretty fair, so it's hard to draw strong conclusions from that. On resilience: yes. Every port ships event-sourced durability by default. Every port survives a mid-request crash. Every port ports cleanly to multi-region without changing the code. These aren't things that many of the originals do out of the box. The runtime's embedded durable execution is a step-function improvement in resilience across every category we tested. 5. Model and Effort: Latency vs. Cost Sonnet is roughly twice as fast wall-clock as Opus per port. Opus uses about 40% fewer tokens per port. Complexity-adjusted efficiency—our composite of steps, time, and tokens normalized to the median port—puts the two models within 8% of each other. Choosing the model is a latency-versus-token-cost tradeoff, not an efficiency-versus-inefficiency tradeoff. Effort matters more than model. Moving from low to high effort roughly doubles the composite efficiency score inside either model. That's a bigger swing than switching models at any effort level. The most efficient cells in the matrix—opus/low and sonnet/low, effectively tied—do the same-quality work at a fraction of the higher-effort cells' cost. In this run there is no evidence that higher-effort settings produced meaningfully more grounded ports; they just cost more. Improvements to Akka Specify Based upon our experience in executing these ports, we productized the project discovery and conversion process within Akka Specify. We created a /akka:port command that will execute a clarification and analysis cycle against an existing code base to generate a specification that Akka Specify will then create a new system from. You have the flexibility to choose between 15 different types of discovery probes and provide an indication of what level of depth the probes should execute against. After the system has been recreated, the analysis probes are rewritten as Exit Conditions that describe the system's behavior as invariants. After conversion, the system enters a state of maintenance, as if it were any other project and detaches itself from the definitions of the originating project. You can also write your own discovery probes that generate insights that can be captured within the spec. You can learn more about the discovery and porting mechanism in the Akka docs https://doc.akka.io/reference/specify/index.html . Future Experiments These results give us a baseline for comparison. We want to run a few variations against it for additional insight. Porting with the tools switched off: We port the same project again, with the same model and the same effort setting. This time we take away the tools that read the original code first. The AI gets the original project and Akka, and nothing else. We want to see how much those tools were helping. We think the port will start giving wrong answers before it starts getting bigger, because the tools that help most are the ones that say no: test census , runtime errors and mutation probe . An AI that tries harder cannot do their job for them. Porting onto a different framework: We port the same project onto a framework that forgets its work when something crashes and does not store data for you. Everything Akka hands over for free has to be written by hand instead. All that hand-written code shows up as extra lines. We have seen this happen already: glanceapp/glance came out 44% bigger because it had to rebuild Go's template library itself. Running it tells us how many lines the platform saves. Building from the specification alone: We hand a brand new AI the specification, SPEC-001, and nothing else. That AI never sees the original project and has no copy of it to read. Then we run the same tests against whatever it builds. Running it tells us whether the specification is good enough on its own. Every port we have run so far let the AI read the original code, so none of them answered that question. Breaking the specification on purpose: We take a single rule out of a specification. Then we check that the matching code disappears from both the original and the port, and that nothing else disappears. Until we do this, we have not shown that the measurement can go down at all. One Overall Finding If there is a single thing to take from 65 ports, it is that the interesting variable in this system is not the model, not the effort, and not the runtime—it is the discipline of the specification and the auditors. The ports that failed did so because the spec left decisions implicit or the auditors weren't yet strict enough to catch a class of mistake. The ports that succeeded did so because the spec forced enumeration and the auditors refused to sign off on anything less. The platform's biggest lever going forward is upstream of the model: it's in making the spec a data structure and the Exit Conditions a versioned, category-aware library. We've started to see Akka Specify as intelligence that can execute an AI-assisted delivery lifecycle without human intervention—not just single systems, but the infrastructure and audit layer that makes them production-ready. Appendix: The Ported Projects Each port has its own repository, with a README covering how to run it, how to re-run the unit and integration tests, and how to rebuild it from scratch using Akka Specify. Table 13