Why your AI assistant hallucinates Roblox APIs — and how a project knowledge vault fixes it

A developer outlined a project knowledge vault approach for stopping AI assistants like Claude and ChatGPT from hallucinating Roblox Luau APIs, arguing that chat windows lack persistent project memory. The method stores an index, stable notes, a tested code library, and a per-game status.md outside the chat, with a fixed loop of reading relevant notes before a task and logging lessons, error fixes, and decisions afterward. The developer notes Claude Desktop can connect to Roblox Studio through Roblox's official Studio MCP interface to build, playtest, and read console output, though copy-paste workflows with ChatGPT or Gemini also work.

If you have ever asked Claude or ChatGPT to write Luau for Roblox Studio, you have probably watched it invent an API. A service method that does not exist. A property with the wrong name. A “standard” pattern copied from five-year-old Lua tutorials that Studio will reject the moment you press Play. This is not mainly because the model is “bad at coding.” It is because a chat window is a terrible place to keep a Roblox project. A general assistant has three gaps at once. First, Roblox is not generic Lua. The language is Luau, the APIs live in Studio services, and the official documentation changes. Training data mixes old forum posts, deprecated patterns, and generic Lua. When the model is unsure, it completes the sentence anyway. Second, each new chat starts almost empty. It does not remember that you already decided to store data in one DataService , that a RemoteEvent failed last night with a specific message, or that your game must work on touch, mouse, and gamepad. So it cheerfully opens a second shop, a second currency, or a second save path. Third, nothing in a default chat forces the model to mark uncertainty. A confident paragraph about MarketplaceService:PromptSomething looks the same as a verified one. The result is familiar: you paste the script, Studio errors, you paste the error back, the model apologizes and invents a slightly different API. A typical twin of that failure is architectural: the next session forgets that a DataStore wrapper already exists and generates a second one “just to be safe.” Both bugs come from missing memory, not from a lack of adjectives in the prompt. A vault, in this sense, is not a pile of bookmarks. It is a source of truth outside the chat, in three layers: status.md with open tasks and decisions, an index of modules you already tested. Without layer 3, you repeat yesterday’s bug. Without layer 2, the model hallucinates. Without layer 1, every session has a different standard of “finished.” A practical folder looks like this: an index the model reads first, a copy-paste master prompt for the project instructions, a 01-Wissen folder for stable notes, a code library with a short index, and 05-Projekte/<Game /status.md . That status file can be short: current stage, the core loop in one sentence, open tasks, dated decisions, one next step, and a launch checklist. Obsidian is useful because the links work, but a plain folder is enough. The structure only helps if you use it the same way every time. Before a task: open the index, then only the notes you need — lessons, preferences, the current status.md , the code index. If an error appears, check the error log first. Reuse tested modules. Do not reread the entire vault. After a task, including an aborted one: add one new lesson as a single line, record solved errors in the “message → cause → fix” form, put tested snippets in the library, and update status.md plus a dated changelog line. Extend existing notes; do not duplicate them. Change sourced knowledge only when you have a source. If the assistant cannot write files itself, have it emit a short “learning update” block that you paste in. Two rules keep the system honest. Uncertain APIs, asset IDs, or numbers get looked up or tested in Studio; otherwise the answer says “uncertain” and how to check. And “done” requires a playtest. A short report should separate what is finished from what is still open. The cleanest loop today is Claude Desktop talking to Roblox Studio through Roblox’s official Studio MCP interface: list the open Studio session, build, run a playtest, read the console, take a screenshot, then fix what actually failed. That is optional. ChatGPT and Gemini work well as a knowledge base plus a fixed prompt; you stay in copy-paste mode, with complete scripts, a setup path, and a test checklist. The vault is what stops both setups from drifting. A vault will not invent a good game idea, pass the Device Simulator for you, or guarantee players. Market numbers go stale; if they have no source and date, they do not belong in 01-Wissen . An oversized vault that nobody updates is just slower chaos. A few notes that you actually read beat a 200-file archive. What it does do is reduce invented APIs, duplicate systems, and the feeling that every morning is a new project. I put this structure into a German-language kit called Roblox Second Brain : a copy-paste master prompt, a 292-page knowledge base PDF and Markdown, dated 31 October 2026 , setup from zero for Windows and macOS, a Luau course, genre blueprints, 101 game briefs, a CSV catalog of 2,174 experiences Roblox API snapshot from the same date , a code library, Blender scripts for your own low-poly models, and the Obsidian-friendly vault with the before/after routine above. It is 19 € once. There is no success claim, and it is not an official Roblox or Anthropic product. A free sample — 12 patterns shared by many hit games, a second-by-second first-minute plan, and a 30-day zero-budget launch template — is here: https://whop.com/kelvaro-creator-kits/leseprobe-roblox-second-brain https://whop.com/kelvaro-creator-kits/leseprobe-roblox-second-brain The full kit is here: https://whop.com/kelvaro-creator-kits/roblox-second-brain https://whop.com/kelvaro-creator-kits/roblox-second-brain Disclosure: the linked kit is my own paid product. The sample is free.