{"slug": "what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re", "title": "What if the problem with AI writing novels isn't context length, but how we're using memory?", "summary": "A developer argues that the difficulty of getting AI to write long-form fiction is an architecture problem rather than a context-length limitation, proposing that novels be treated as structured projects with retrievable state instead of one giant conversation. The approach separates story canon from raw ideas, applies version-control-style tracking to plot and character facts, and supplies the model only the relevant information for the chapter at hand.", "body_md": "I keep seeing the same argument whenever people talk about AI writing long-form fiction:\n\nAI can't really write a 100,000-word novel because eventually the context becomes too large, details get lost, and the model starts contradicting itself.\n\nAnd I think we might be looking at this as an **LLM problem when it's actually an architecture problem.**\n\nHear me out.\n\nWe've already solved something surprisingly similar in software engineering.\n\nIf I'm working on a 200,000-line codebase, I don't need to hold the entire codebase in my head every time I want to fix one function.\n\nThe project has files, folders, documentation, dependencies, state, tests, Git history, configuration, and architecture decisions.\n\nI pull the information I need, make a change, test it, and move on.\n\nThe entire codebase exists.\n\nBut I don't need the entire codebase in my **active working context** at the same time.\n\nSo why does AI-assisted writing often work more like this?\n\n\"Here, Claude. Here's the entire 100,000-word novel. Now remember all of it forever and write Chapter 47.\"\n\n😂\n\nMaybe we need to stop treating a novel as one giant conversation.\n\nInstead of thinking about a novel as one enormous block of text, imagine treating it as a **structured project**.\n\nThe chapters are the actual story.\n\nThen you have separate information about:\n\nThe important distinction is that the model doesn't need to read everything every time.\n\nIt needs the **right information for the current task**.\n\nFor example, if I'm writing Chapter 48, Claude probably doesn't need Chapters 1–47 sitting in its context.\n\nIt might need:\n\nThat's a much smaller context.\n\nBut the information from Chapter 12 hasn't disappeared.\n\nIt's simply stored somewhere else and retrieved when it's relevant.\n\nThat's a fundamentally different way of thinking about the problem.\n\nThis is where I think things get much more interesting.\n\nImagine I'm halfway through a novel and suddenly think:\n\n\"Wait... what if John's brother is actually working for the antagonist?\"\n\nThat's not canon.\n\nIt's just an idea.\n\nSo the system should treat it differently from an established fact.\n\nMaybe it starts as a raw idea.\n\nLater, I develop it:\n\n\"Actually, that would explain why John disappeared in Chapter 7.\"\n\nNow it's becoming a serious possibility.\n\nEventually I decide:\n\n\"Yep. This is canon.\"\n\nAt that point, the system should update the relevant character relationships, plot threads, and timeline.\n\nThat's basically **version control for story ideas**.\n\nAnd I think that's something current AI writing workflows are missing.\n\nThis might be one of the most important parts.\n\nA writing system shouldn't treat these two statements as equivalent:\n\n\"Maybe Sarah has a sister.\"\n\nand:\n\n\"Sarah has a sister named Emily.\"\n\nThe first is an idea.\n\nThe second is established canon.\n\nThere should probably be a distinction between things that are:\n\nThat gives the AI something incredibly important:\n\n**a distinction between possibilities and facts.**\n\nOne of the most frustrating things about working with AI on a long project is when an idea casually mentioned 30 chapters ago suddenly gets treated as established fact.\n\nA human writer understands:\n\n\"I was just brainstorming.\"\n\nThe model may not.\n\nA structured project could make that distinction explicit.\n\nYou could have a persistent profile for every major character.\n\nNot just:\n\n\"Sarah is a detective.\"\n\nBut something much richer:\n\nAnd importantly, those answers shouldn't necessarily be static.\n\nSarah in Chapter 5 might be very different from Sarah in Chapter 50.\n\nSo instead of telling Claude:\n\n\"Remember Sarah.\"\n\nYou could give it:\n\n\"Here's Sarah's current canonical state as of Chapter 48.\"\n\nThat's much more useful.\n\nThe system isn't trying to make the model permanently remember Sarah.\n\nIt's maintaining **Sarah's current state** and giving the model the relevant version when needed.\n\nThis is where I think AI-assisted fiction could become genuinely interesting.\n\nBefore committing a chapter, the system could run something resembling a **test suite**.\n\nImagine Claude finishing Chapter 48 and the system checking:\n\n**Continuity check**\n\nSarah says she has never been to Paris.\n\nChapter 14 says Sarah lived in Paris for three years.\n\n**Conflict detected.**\n\nJohn knows about the murder.\n\nCurrent plot state says John shouldn't know this yet.\n\n**Knowledge-state conflict detected.**\n\nSarah's age has changed from 29 to 32.\n\n**Character-state conflict detected.**\n\nThe gun mentioned in Chapter 48 was destroyed in Chapter 21.\n\n**Object continuity conflict detected.**\n\nThat's basically a **novel compiler**.\n\nAnd I actually think this could be incredibly useful.\n\nBecause a bigger context window doesn't automatically solve continuity.\n\nYou could give a model 500k tokens and hide one critical fact somewhere in the middle.\n\nThe model may still fail to use it correctly.\n\nThe problem isn't necessarily:\n\n**\"Does the AI have access to the information?\"**\n\nIt might instead be:\n\n**\"Can the AI reliably identify which information matters right now?\"**\n\nThat's a very different problem.\n\nI think these three concepts get mixed together way too often.\n\n**Context** is the information the model currently sees.\n\n**Memory** is information about the project that can be retrieved when necessary.\n\n**Source of truth** is what has actually been established in the novel.\n\nThose aren't the same thing.\n\nThe actual chapter can be the source of truth.\n\nA character profile can be a structured representation of that truth.\n\nA retrieved memory can provide relevant information from the project.\n\nAnd the active context can contain only what the model needs **right now**.\n\nThat separation seems much more scalable than trying to make the context window itself responsible for everything.\n\nThis is another place where the software analogy becomes interesting.\n\nWriters don't just have one version of a story.\n\nThey have:\n\nImagine being able to ask Claude:\n\n\"Show me all the abandoned plot ideas involving John's brother.\"\n\nOr:\n\n\"Why did we decide that Sarah couldn't know about the letter yet?\"\n\n\"Compare the current ending with the ending from Draft 2.\"\n\nThat's much closer to how humans actually work on large creative projects.\n\nAnd it's also much closer to how we already manage complex software projects.\n\nYou don't throw away the old code every time you change something.\n\nYou keep history.\n\nYou keep state.\n\nYou keep a source of truth.\n\nYou create branches when necessary.\n\nWhy shouldn't long-form AI writing work the same way?\n\nThere's another distinction I'd make.\n\nA lot of current AI systems essentially ask:\n\n\"What information looks relevant to this prompt?\"\n\nBut for a large creative project, I think the better question is:\n\n**\"What project state is required to perform this task correctly?\"**\n\nIf I'm asking Claude to write Chapter 48, the system should know that it probably needs the previous chapter, the current character states, active plot threads, relevant continuity information, the current timeline, and the established writing style.\n\nIt probably doesn't need a rejected plot idea from Draft 2.\n\nIt doesn't need a deleted scene.\n\nIt doesn't need an unrelated subplot from a completely different part of the book.\n\nThe goal isn't to retrieve **more** information.\n\nIt's to retrieve the **right** information.\n\nThe same architecture could work for almost any large creative project.\n\nA screenplay.\n\nA game.\n\nA comic universe.\n\nA research project.\n\nA long-running YouTube channel.\n\nA technical documentation project.\n\nEven software development itself.\n\nThe underlying problem is the same:\n\n**The project is larger than the model's useful working context.**\n\nSo instead of constantly increasing the context window, perhaps we should get better at managing the information that enters it.\n\nThis is what I'm starting to wonder.\n\nMaybe the future isn't simply:\n\n**LLM + giant context window**\n\nMaybe it's:\n\n**LLM + project state + retrieval + structured memory + validation**\n\nThe model becomes the reasoning engine.\n\nThe project system becomes the persistent memory.\n\nAnd the application decides what information the model actually needs for each operation.\n\nIn other words, instead of asking the model to **remember the entire book**, we build a system that helps the model **navigate the book**.\n\nI don't think the answer to long-form AI writing is necessarily:\n\n\"Give the model a 5-million-token context window.\"\n\nBecause at some point the book gets bigger.\n\nThen there are sequels.\n\nThen character notes.\n\nThen research.\n\nThen alternate drafts.\n\nThen deleted scenes.\n\nThen worldbuilding.\n\nThen 400 random ideas I had at 3 AM.\n\nEventually you're going to have a giant pile of text and tell the AI:\n\n\"Remember all this.\"\n\nThat's not really a memory system.\n\nThat's a very expensive folder.\n\nI'd rather have something closer to:\n\n**Novel → structured project memory → relevant context → Claude → continuity check → updated story state**\n\nThe novel remains the source of truth.\n\nThe memory system keeps track of what matters.\n\nThe context is assembled for the task at hand.\n\nClaude does the reasoning and writing.\n\nThen the resulting changes get fed back into the project state.\n\n**Git + RAG + structured memory + an LLM + a novel.**\n\nAnd honestly, that is starting to feel less like a workaround and more like the architecture we'd actually want.\n\nI'm curious if anyone is already doing something along these lines with Claude Projects, Claude Code, Obsidian, RAG, custom MCP servers, or something completely different.\n\nBecause if someone has already built **\"Git for novels\"**, I desperately want to know about it.", "url": "https://wpnews.pro/news/what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re", "canonical_source": "https://dev.to/suryacreatx/what-if-the-problem-with-ai-writing-novels-isnt-context-length-but-how-were-using-memory-2lci", "published_at": "2026-10-08 16:39:15+00:00", "updated_at": "2026-10-08 16:49:40.363933+00:00", "lang": "en", "topics": ["large-language-models", "generative-ai", "ai-agents", "ai-tools"], "entities": ["Claude"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re", "markdown": "https://wpnews.pro/news/what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re.md", "text": "https://wpnews.pro/news/what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re.txt", "jsonld": "https://wpnews.pro/news/what-if-the-problem-with-ai-writing-novels-isn-t-context-length-but-how-we-re.jsonld"}}