{"slug": "teaching-llms-to-one-shot-complex-backends-at-scale-report-3", "title": "Teaching LLMs to one-shot complex backends at scale, report #3", "summary": "Red Planet Labs reports that large language models (LLMs) are now consistently one-shotting complex backend challenges, including an unbalanced social graph, a social graph with fanout, and a Slack-like chat backend, as part of a project to one-shot the entire Matrix specification. The company says the models achieve this with only general programming knowledge and no task-specific guidance, countering claims that LLMs are only good for prototyping.", "body_md": "See the [first report](https://blog.redplanetlabs.com/teaching-llms-to-one-shot-complex-backends-at-scale-report-1/) in this series in this series for context on our goal of teaching LLMs to one-shot complex backends at scale. To summarize, we’re working towards one-shotting a scalable, fault-tolerant, and high-performance implementation of the entire [Matrix spec](https://matrix.org/), which is orders of magnitude more difficult than any backend task LLMs have demonstrated so far.\n\nSince the last report, LLMs are now consistently one-shotting these new challenges:\n\n- Build a highly scalable\n[unbalanced social graph](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/unbalanced-social-graph)that can support balanced fanout for a Twitter-like service. - Build both the\n[social graph and fanout](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/social-graph-and-fanout)for a Twitter-like service. - Build a\n[Slack-like chat backend](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/chat-app)supporting user registration, rooms, messages, threads, “recent threads you’re involved in” view, presence, unread counts, and pagination.\n\nThese are on top of the previous challenges of [bank transfers](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/bank-transfer-module), [fanout with a prebuilt social graph](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/fanout), [time-series analytics](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/time-series-module-hard), and an [auction service](https://github.com/redplanetlabs/rama-ai-learn/tree/master/challenges/auction-module).\n\nThe unbalanced social graph challenge is a tricky distributed systems problem requiring the LLM to reason about scalability. The social graph plus fanout challenge requires the LLM to reason through two difficult problems at the same time, and passing it required teaching it to decompose the problem into smaller steps so it could focus. The chat challenge is a pile of intertwined functionality, each of which is not difficult but requires proper decomposition.\n\nLike all the challenges, the only guidance given to the LLM is general knowledge about programming Rama and general principles of architecting backend systems. No specific information is given on how to approach these particular problems. Many say that LLMs are good for prototyping and mocking, but not for autonomously building production-grade systems with tight performance constraints. That is not what we’re finding through this project.\n\n## Teaching LLMs how to reason about scalability\n\nThe unbalanced social graph problem requires a solution that not only balances computation across the cluster, but also achieves proportionally more throughput with increased resources.\n\nThe most naive approach is to store all of someone’s followers on a single partition. This means reading all of their followers requires reading them all off that one partition. In an unbalanced social graph, where there can be a 1000000x difference in follower counts between celebrities and average users, this causes fanout to be severely unbalanced whenever a celebrity posts.\n\nThe obvious way to fix this is to spread someone’s followers across all partitions, like by choosing the target partition by the hash of the follower ID. This balances processing, but it fails badly in terms of scalability. The problem is the cost of getting a non-celebrity’s followers. In this scheme, reading someone’s followers requires doing a read on every partition. So if you scale up the cluster by doubling the number of partitions, you’re also doubling the number of reads that has to be done to read someone’s followers, regardless of how few followers they have.\n\nThe correct approach is for people with more followers to store their followers across progressively more partitions. If you set the threshold to 1000 followers, this means someone with 100 followers would store all on one partition, someone with 5000 followers would store on 5 partitions, and someone with 100M followers would store on all partitions. This is not hard to implement with Rama and just requires thinking about the problem correctly.\n\nAt first, the LLM kept making one of two mistakes. Sometimes it went with the second naive approach, stopping after addressing the balance problem. Other times it used a single threshold, storing followers on one partition below it and spreading across all partitions above it. That’s on the right track, and it works fine on a heavy-tailed distribution where almost everyone is below the threshold. But on a medium distribution, such as where lots of users have 5000 followers, those followers get spread across all partitions when they only needed 5.\n\nWhat fixed it was making the LLM compute instead of rationalize. The unit it computes with is the seek, the random disk read that starts any lookup. A seek costs about 0.5ms while iterating sequentially afterwards costs a few microseconds per element, so counting seeks is a good proxy for the total cost of the operation.\n\nThe planning doc now requires that count:\n\n“Seeks/op” is the\n\ntotal number of tasks the operation reads from, summed across the whole cluster, one local read (seek) per task touched. A read that fans to all N tasks costsN, even if some tasks’ local slice is empty (the read is still dispatched there). This is a TOTAL across tasks, do NOT count per-task or divide by the task count (a per-task number falls as N grows for any design and measures nothing).\n\nThat table gets filled in at three cluster sizes, N = 1, N = 16, and N = 128, with a row per category of data weighted by how often operations hit it. Then one rule decides the design:\n\nIf weighted seeks grow substantially from N=1 to N=128, the partitioning is INEFFICIENT, redesign. A good design keeps weighted seeks roughly flat as N grows.\n\nThe second mistake fails that test on their own arithmetic, and the LLM rejects them without being told anything about social graphs.\n\n## Learning to decompose\n\nThe need for a decomposition step showed up on the social graph plus fanout challenge. The LLM had already solved each problem on its own, but given both problems at once it got overwhelmed and reverted to incorrect solutions, such as storing followers across all partitions for every user for the social graph.\n\nTo enable the LLM to be able to focus, we introcuced a decomposition step at the beginning of the workflow to build the solution in stages. Each stage runs the full planning, plan validation, implementation, implementation validation, testing, and test validation workflow.\n\nThe first attempt gave each stage a self-contained mini-spec, with its operations and the relevant workload numbers copied in. However, this didn’t work well. The decomposing agent would often say too much, anchoring the subsequent agent, such as prescribing extra implicit constraints (such as boundaries between stages) or even specifying how things must be designed.\n\nThe fix was to stop producing specs. A stage now produces only a scope, and the original spec stays the only spec:\n\nA stage’s scope duplicates NOTHING from the full spec, no restated requirements, no copied numbers, no paraphrases. Where a scope must invoke a requirement, it points at it. A scope names purpose, never design. It never prescribes schemas, keys, partitioning, placement, topology types, dataflow, or any other mechanism choice.\n\nEvery build cycle reads the entire original spec. The scope only says which operations that cycle owns and which state it must figure out, named by role. Portioning is governed by one rule:\n\nThe purpose of decomposition is focus: a build cycle should face ONE hard problem at a time. When the problem contains multiple hard parts, you MUST separate them. A problem with a single hard core stays ONE stage. Parts too trivial to deserve a build cycle fold into the part that uses them. Order the stages so later ones build on the state earlier ones create.\n\nThe drawback to decomposition is runtime. Each stage runs the entire cycle, so a six stage decomposition means six full workflows back to back. The chat app challenge decomposes to six or seven stages, and where the social graph challenge takes 60 to 90 minutes, the chat app challenge takes 4.5 to 8 hours. Ultimately this is a necessary cost, as without decomposition an LLM cannot solve these challenges completely.\n\n## Model effectiveness\n\nWe did many experiments with different combinations of models and efforts for planning and implementation. For the social graph plus fanout challenge, Fable/high for planning and Opus/low for implementation consistently passes, whereas Opus/high for planning and Opus/low for implementation usually gets one thing wrong.\n\nFor the chat app challenge, Fable/high plus Opus/low is able to pass in about 4.5 hours, whereas Opus/high plus Opus/low find the right solution but take more than 8 hours. What we see during code gen with Opus/high doing the planning is a significant amount of additional time finding mistakes in its original design and correcting them.\n\nWe did a few runs with Sonnet doing implementation, but it made significantly more mistakes than Opus or Fable. We need to do more experiments with Sonnet to fully understand the types of mistakes it makes and why.\n\n## What’s next\n\nA high priority now is testing with models from different providers, like OpenAI, Moonshot, and xAI. I expect lots of surprises in how they perform better or worse than the Anthropic models.\n\nWe’re also looking to expand challenges in two ways. The first is to push LLMs more on advanced distributed algorithms, and we’re thinking of graph algorithm challenges for this. Distributed loops and distributed intermediate state are powerful capabilities of Rama, and it will be interesting to see what it takes to get LLMs to reason with those to solve tough problems.\n\nThe second type of challenge we’re looking to build are much more complex ones whose implementations realistically require a lot of decomposition into multiple namespaces and many helper functions. Even though the chat app challenge has a lot of functionality in it, it’s still too concise to exercise this sort of coding.", "url": "https://wpnews.pro/news/teaching-llms-to-one-shot-complex-backends-at-scale-report-3", "canonical_source": "https://blog.redplanetlabs.com/2026/08/12/teaching-llms-to-one-shot-complex-backends-at-scale-report-3/?utm_source=rss&utm_medium=rss&utm_campaign=teaching-llms-to-one-shot-complex-backends-at-scale-report-3", "published_at": "2026-08-12 17:39:20+00:00", "updated_at": "2026-08-12 17:46:26.187460+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["Red Planet Labs", "Matrix", "Rama"], "alternates": {"html": "https://wpnews.pro/news/teaching-llms-to-one-shot-complex-backends-at-scale-report-3", "markdown": "https://wpnews.pro/news/teaching-llms-to-one-shot-complex-backends-at-scale-report-3.md", "text": "https://wpnews.pro/news/teaching-llms-to-one-shot-complex-backends-at-scale-report-3.txt", "jsonld": "https://wpnews.pro/news/teaching-llms-to-one-shot-complex-backends-at-scale-report-3.jsonld"}}