# From the Fantasy of Local Models to a Truly Controllable Sports Agent > Source: > Published: 2026-07-08 14:14:27+00:00 Looking back, this project did not begin as a fully designed AI agent project. In fact, I had not even decided from the outset that I wanted to build a sports question-answering system. The original idea was much simpler. At the time, I frequently saw people on X sharing their experiences with locally deployed large language models. Local AI was advertised as having many advantages: data could remain on your own device, privacy requirements could be satisfied, you would not be restricted by the service policies of major model providers, and you would not have to worry about your account, API access, or region suddenly being restricted one day. The models might not be as capable as the best cloud-based models, but as long as they were good enough for everyday use, the idea still seemed highly attractive. I had also been following Mr. Wang from LazyCat Microserver for some time. He promoted the company’s products almost every day, which sparked my interest in the LazyCat AI Pod. Eventually, I ordered one. After receiving the device, I began experimenting with various models that could be deployed locally. Following a round of comparisons, I ultimately chose the DeepSeek R1 distilled version of Qwen 32B. To extract as much performance as possible from the hardware, I repeatedly adjusted the model’s startup parameters based on recommendations from GPT. I pushed VRAM allocation, context length, concurrency, and inference settings as close as possible to the maximum that the hardware could handle at the time. When I first got the model running, I was naturally a little excited. After all, a model with tens of billions of parameters was genuinely running on a device that I owned. It felt as though I finally had an AI that belonged entirely to me. But after using it a few times, my most immediate reaction was: **Fuck, something was wrong.** Its responses were slow, and its knowledge was outdated. More importantly, there was an enormous gap between it and the GPT experience I was used to when it came to understanding questions, maintaining context, organizing answers, and handling complex tasks. It was no longer simply a matter of being “slightly less capable” or “slightly slower.” They were not even operating in the same dimension. That was the first reality I encountered during this project: **Being able to run a model does not mean that the model has practical value.** Local deployment solves questions about where the model runs, who controls the data, and how dependent you are on a vendor. It does not automatically solve problems involving model capability, knowledge freshness, contextual understanding, reasoning quality, or response speed. When the model itself is clearly not capable enough, the idea of “local AI freedom” can easily turn into an expensive, slow machine that produces answers that are not particularly useful. Since I had already bought the device and successfully deployed the model, I decided to make some use of it by building at least a simple chatbot. At first, I did not seriously consider whether it had the conditions required for real-world operation. I simply added several prompt chains around the model, hoping that breaking tasks into multiple prompting stages would compensate for the model’s limited capabilities. But I quickly realized that building a general-purpose chatbot was pointless. It could never be more general than cloud models such as GPT or Gemini, nor could it compete with them in knowledge, reasoning, or response speed. Once I built it, even I found it frustrating to use. A general-purpose chatbot that I did not want to use myself was not worth continuing to develop. So I narrowed the scope and decided to build a sports scores chatbot. Compared with open-domain question answering, sports scores at least offered a clearly defined data scope, fixed rules, queryable interfaces, and relatively well-defined question types. From a product-scope perspective, it seemed more suitable for a limited local model. However, relying solely on an LLM was obviously insufficient. The model did not know the latest scores, and it should not be expected to explain every sports rule from its own memory. Following guidance from GPT, I began collecting rules and documentation for major sports and esports. I organized the material into YAML files, built a vector index with FAISS, and integrated it into the question-answering workflow. This was probably my first real exposure to the idea of retrieval augmentation. My understanding at the time was very straightforward: if the model did not know something, search the knowledge base first, then insert the retrieved results into the prompt. The idea itself was not wrong, but after running the system a few times, the problems quickly became apparent. Each vector search returned a large volume of rules data. That content could consume roughly 80% of Qwen 32B’s available input space. The remaining context might only be enough for the user’s first question. By the second turn, once the conversation history, rules content, and previous answer were all included, the system could easily exceed the context limit. That was when I truly realized: **Context is not a warehouse. It is a highly limited runtime budget.** Retrieval does not mean stuffing every piece of “potentially relevant” information into the model. More retrieved content does not necessarily produce a more accurate answer. Large amounts of low-density information not only consume context, but also dilute the genuinely important details, making it even harder for an already limited model to identify what matters. Although I was already using RAG and vector retrieval, I had only completed the step of “search for it and put it into the prompt.” I had not yet truly understood context engineering. As the project continued, I began adding more sports scores APIs. These APIs were not designed specifically for LLMs. Some were used to search for competitions, while others queried fixtures, standings, lineups, match events, team information, or player details. Many of these APIs had explicit hierarchical dependencies: the output of one endpoint served as an input parameter for another. To help the system understand these interfaces, I gradually created an `api_kb.yaml` file containing API descriptions, input parameters, output fields, enumerations, data relationships, and usage restrictions. Since I had already used FAISS, I naturally applied the same idea to the API knowledge base. I hoped that vector search could identify the APIs relevant to a user’s question, together with their query rules and required execution order. At the time, I assumed that if the API documentation was sufficiently complete and vector search could retrieve semantically similar endpoints, the LLM would be able to combine those endpoints on its own. The actual results were poor. FAISS could find API documentation that appeared relevant, but it could not reliably understand the real business dependencies between interfaces. It might retrieve an API with a similar field name while ignoring the fact that one of its parameters first had to be obtained from another API. It might also mix together interfaces intended for teams, competitions, seasons, or match details. More importantly, the retrieval results themselves were unpredictable. The same question could return different documents at different times. A slight change in intent could cause the retrieved results to drift away from what was actually needed. The final experience was that the system sometimes worked and sometimes failed completely, with no reliable way to predict which outcome would occur. The user experience was terrible. At the time, however, I did not abandon vector retrieval. I simply continued adding rules and prompts around it, adjusting similarity thresholds, and changing the number of retrieved documents, hoping that further optimization would solve the problem. Looking back, I was asking a retrieval system to take responsibility for something it was not good at. Vector retrieval is suitable for finding related knowledge. It is not suitable for independently deciding how a group of APIs with strict parameter dependencies should be executed. **Knowledge relevance and tool executability are two entirely different problems.** During this process, I also integrated the Gemini API. My goal was to classify requests by complexity and route them to different models. Simple tasks could be handled by cheaper, faster models, while more complex tasks would be sent to more capable ones. In theory, this would reduce costs while improving overall response time. Later, I also integrated the LLM APIs provided by GitHub and the Grok API, further expanding the routing system in the hope that different providers could complement one another. However, LLM calls involve many unpredictable situations, particularly when local models are involved. Sometimes a request would remain unanswered for a long time. Sometimes the returned format would not meet the requirements. Sometimes the model would misunderstand the intent. Sometimes the generated JSON could not be parsed. Sometimes the service was already in an unhealthy state, while the upstream workflow continued waiting. I therefore began assigning different timeout limits to different models and added service health checks, exception handling, response-format validation, and fallback mechanisms. When a cheaper model failed, the request would switch to another model. If that model also failed, it would switch again. These features did improve availability, but they also helped me understand something: **Model routing can address cost and some reliability issues, but it cannot fix an incorrect system architecture.** If intent recognition is unstable, API planning is fundamentally flawed, and the input-output boundaries are unclear, adding more models simply passes the same problem from one model to another. The original workflow included intent detection, rule retrieval, API retrieval, API execution, and final answer generation, but it lacked a node that was genuinely responsible for business planning. I therefore decided to add a “decision flow” node. This node needed to select the appropriate scores APIs based on user intent, process API inputs and outputs, understand dependencies between different interfaces, and determine which calls needed to run sequentially and which could run in parallel. The direction itself was correct. The problem was the implementation. While building out the decision flow, Codex chose the easiest and most direct development approach: identifying user intent through keyword matching. If the user mentioned “standings,” the system selected the standings API. If the user mentioned “lineup,” it selected the lineup API. If a particular fixed term appeared, the request entered a corresponding fixed workflow. With a small number of test cases, this approach appeared highly effective. The code was generated quickly, and the functionality could be made to work almost immediately. But as the range of question types expanded, the keyword rules grew rapidly. The same word could express different intents in different contexts. Different languages, aliases, natural expressions, and conversational follow-up questions could never be exhaustively covered through a finite list of keywords. When the user said “What about this match?”, “The second one,” “The one from earlier,” or “This Portugal match,” the system did not need to understand an isolated keyword. It needed to understand the current conversational focus, the candidate list from the previous turn, and the goal the user was continuing to pursue. Once keyword matching entered this kind of scenario, it became increasingly difficult to maintain. To patch these problems, the decision flow gradually accumulated conversational disambiguation, candidate matching, returning to the original question after candidate selection, user-focus redirection, and new-topic versus old-topic detection. The number of features continued to grow, and the code became increasingly bloated. At the time, I was still relying heavily on Codex for vibe coding. As long as the current tests passed, I deliberately ignored code structure and long-term maintainability. Those problems did not disappear. They were merely postponed. Another source of complexity was that the sports scores APIs used by the project were never designed to be called by an LLM. Many endpoints required internal IDs, but ordinary users only referred to team names, league names, or particular matches. They did not know, and should not need to know, values such as `matchId` , `leagueId` , `seasonId` , `teamId` , or `playerId` . Before calling the actual business API, the system therefore usually needed to call a search API first, converting a team, competition, or player mentioned in natural language into an internal object. When the candidates were not unique, the system needed to present them to the user for selection. After the user selected one, the system then had to return to the original question and complete the original task, rather than stopping at “You selected the second item.” API responses could not be passed directly to the model either. Many endpoints returned large numbers of fields containing `null` , internal statuses, irrelevant media URLs, duplicated structures, and data that had no value for the current question. Without preprocessing, the response from a single API could consume a large amount of context. As a result, the project accumulated an increasing number of pre-call and post-call processing steps: parameter completion, contextual binding, dependency-parameter extraction, removal of null-valued fields, list truncation, field projection, and response normalization. I gradually realized that allowing an LLM to call tools was not as simple as asking the model to generate an API name and a set of parameters. The genuinely difficult part was establishing a stable boundary between the world of natural language and the world of internal APIs. As development continued, I realized that I could not support every sport at the same time. Different sports have different competition structures, match states, rules, statistical fields, and APIs. Mixing football, basketball, esports, and other sports together would cause the intent taxonomy, data structures, and testing scope to expand without limit. I therefore modified the user-intent detection node so that only football and basketball questions were allowed to enter the internal sports API workflow. Esports and other sports were rejected directly. On the surface, this reduced the product’s capabilities. In practice, it was the first time the project established a genuine product boundary. A broad system whose correctness cannot be guaranteed is less valuable than a limited system that can be explained, tested, and controlled. The project later defined its priorities as follows: **Accuracy over controllability, controllability over speed, and speed over completeness.** This principle was not designed at the beginning. It was a conclusion reached only after a large number of failures. The project truly began to change when I started reading Google Cloud’s materials on AI agents and learning more about agentic design patterns. Before that, I already had prompt chains, intent detection, API calls, contextual state, candidate selection, and model routing. But these elements were merely accumulated features. They did not form a system with clearly defined responsibilities. After reading those materials, I began to understand agents differently. An agent does not mean allowing a model to decide everything freely. Nor does it mean placing all data, tools, and rules into a prompt at once. A genuinely controllable agent system must separate uncertain judgment from deterministic execution. LLMs are good at understanding natural language, identifying user goals, handling ambiguous expressions, generating plans, and organizing answers. Deterministic code is better suited to checking parameters, validating dependencies, restricting the available toolset, binding internal IDs, executing APIs, and blocking invalid plans. Based on this understanding, I redesigned the entire project. I disabled rule retrieval by default. When a user provides an exact score and asks the system to ground the answer, a passage explaining football rules cannot serve as evidence for that score. Match facts should come from real-time APIs. The rules knowledge base can explain rules, fields, and enumerations, but it cannot be treated as evidence for live match facts. I also gradually abandoned the practice of integrating large numbers of models to keep the workflow running. I switched entirely to Gemini and divided the runtime into two modes: `normal` and `optimal` . Both modes use the same business workflow and safety boundaries. They differ only in the capability and cost of the models used at different stages. In this design, differences between models no longer change system behavior. They only affect execution quality and cost. The final system was not a freely acting autonomous agent. It was a controlled agentic workflow. A user question first passes through goal monitoring, intent understanding, and contextual resolution. A planner then generates a structured plan. The Tool Graph Validator checks whether the requested tools exist, whether their parameters are available, and whether their dependency chains are complete. The Tool Executor calls the internal sports APIs. The Web Evidence Agent handles official sources and general web searches. Reflection determines whether the available data is sufficient and, when necessary, permits only one repair attempt. Finally, the Answer Composer produces a unified response, while grounding checks restrict the permitted sources of factual claims. Within this architecture, the LLM can perform the tasks it is good at, but it does not have the authority to bypass the validator and execute tools arbitrarily. Only then did I finally understand that the “intelligence” of an agent should not come from giving the model unlimited freedom. It should come from this principle: **Within clearly defined boundaries, leave uncertainty to the model and certainty to the program.** Later in the project, I added Web Evidence. When I originally built the rules knowledge base and FAISS retrieval, my understanding of external knowledge was still based on traditional RAG: collect content, build an index, perform vector retrieval, and insert the results into the prompt. After the redesign, however, I did not turn Web Evidence into another permanent vector database. The current workflow only runs `official_source` or `general_search` when the LLM planner determines that the user’s question genuinely requires external evidence. Official sources are primarily used for websites such as FIFA and UEFA. General search is used for news, injuries, transfers, historical context, and questions that cannot be covered by the internal data. Web Evidence is also not allowed to generate the final answer directly. It can only return a structured Evidence Pack containing sources, evidence, conflicts, failures, and timing information. The unified Answer Composer must still generate the final response by combining that evidence with data from the internal APIs. When internal sports APIs conflict with external official sources, the system cannot simply choose one and overwrite the other. The conflict must be explicitly preserved. While reverse-engineering the UEFA and FIFA websites, I also gained a stronger appreciation for the advantages of AI when handling large codebases and web resources. The API requests used by these official websites involved many complex restrictions, and their frontend JavaScript files were extremely large. However, once the relevant JavaScript files were provided to an AI, it could quickly locate the required keys, endpoint URLs, and parameter logic through search. Whether a file contained ten thousand lines, one hundred thousand lines, or even more made little fundamental difference to the AI. This is one of the most powerful aspects of AI-assisted programming. But the same capability also concealed another problem. Once the project’s functionality was largely complete, I began seriously reading the code that Codex had generated for me. The experience was extremely painful. Thousands or even tens of thousands of lines of code had been concentrated into individual files. For an AI, this was not a problem. It did not need to read the code from the first line to the last in the way a human would. It could search directly for the relevant locations through commands, symbols, and keywords. To an AI, the difference between one hundred thousand lines and ten thousand lines might simply be the size of the search space. But that was not true for me. I needed to build my own index of the entire project inside my head. I needed to know where a request entered the system, which components it passed through, where state was modified, who generated a particular parameter, who called a particular method, which path was taken after a failure, and where the final answer was produced. Only after building this index could I genuinely understand the project, rather than merely knowing that “it currently runs.” Building that mental index was far more difficult than asking an AI to search through files. But I deserved that pain. All the design, reading, and organizational work that I had skipped through vibe coding eventually returned in another form. AI had helped me generate the code earlier, but it had not taken responsibility for understanding the code on my behalf. Technical debt does not disappear simply because the code was written by AI. On the contrary, because AI can generate code more quickly, technical debt can accumulate more quickly as well. While reading the code, I noticed another clear characteristic of Codex-generated implementations: they contained an enormous amount of defensive code. At almost every stage, Codex would repeatedly verify whether data existed, whether its type was correct, whether fields matched expectations, whether parameters were empty, and whether the current object had a particular attribute. When viewed in isolation, these checks usually made sense. Codex did not naturally trust that validation had already been performed upstream, nor did it proactively rely on assumptions that had already been established elsewhere in the call context. From its perspective, checking again inside the current function was the safest implementation. This is a highly reliable code-generation strategy, but not necessarily a good long-term engineering strategy. When the same parameter is repeatedly checked in the Router, Planner, Validator, Executor, and Answer stages, the code becomes extremely verbose. The actual business logic is surrounded by layers of protective code, making it difficult for developers to distinguish between checks that define genuine system boundaries and checks that are merely redundant. During the reading and refactoring process, I therefore began asking Codex to reduce repetitive validation based on the established context. Data that has already passed validation through a Pydantic model at a stable boundary does not need every internal function to revalidate every field. Similarly, tool dependencies already guaranteed by the Tool Graph Validator should not be reimplemented as incomplete validation logic inside every execution function. More defensive code is not always better. What matters is knowing where the system needs to defend itself. External input, model output, API responses, persisted data, and component boundaries require strict validation. Inside those boundaries, the code should be able to trust the contracts that have already been established. Otherwise, so-called defensive programming eventually becomes a large volume of noise that no one can confidently determine is safe to remove. After encountering these oversized files, I began directing Codex to split the code apart. However, mechanically dividing one large file into several smaller files was not enough. If smaller files caused the call relationships to become more scattered, they could make the system equally difficult to understand. I therefore required detailed comments at critical locations: which component called the current method, which file the method called next, where its inputs came from, and which component received its outputs. These comments were not written for the AI. AI could search symbols and call relationships directly. I needed this information to build an overall mental model quickly. As the restructuring continued, the project gradually developed clearer directory boundaries: `app/nlu` became responsible for natural-language understanding and contextual resolution. `app/agents` became responsible for planning, execution, reflection, and answer generation. `app/service_runtime` became responsible for Redis, PostgreSQL, and runtime storage. `app/api_runtime` became responsible for the API knowledge base and internal API calls. `app/web_evidence` became responsible for external evidence. `app/services.py` retained only HTTP adaptation and dependency assembly. Stable structures also gradually moved away from anonymous dictionaries and toward explicit Pydantic models. Model outputs, API call results, session context, long-term memory, and final-answer inputs all began to have their own typed boundaries. At that point, I finally understood that modularity is not about making the directory structure look tidy. The value of modularity is that it reduces the amount of information a human must load into their mind at the same time in order to understand the system. A file should allow a developer to answer the following questions: What is it responsible for? What is it not responsible for? Which inputs does it trust? What outputs does it promise to downstream components? This project significantly changed how I understand local models, RAG, agents, and AI-assisted programming. I no longer believe that local deployment automatically means autonomous and controllable AI. Local deployment only gives you control over the runtime environment. Model capability, knowledge quality, and reasoning performance still determine the final user experience. I also no longer believe that RAG simply means placing search results into a prompt. Effective retrieval must consider information density, context budgets, task stages, and factual boundaries. I no longer allow vector search to determine the order of tool calls. Relevant knowledge can be retrieved, but strict API dependencies must be controlled through structured plans and deterministic validators. I no longer believe that adding more models can solve system-level problems. Model fallback can improve availability, but it cannot repair an incorrect division of responsibilities. I also no longer treat “the code runs” as meaning that development is complete. Code must be understandable, verifiable, testable, and modifiable. Otherwise, it is merely a collection of implementations that happen to work at the current moment. Most importantly, I began to understand vibe coding differently. Vibe coding is not without value. It allowed me to explore a large number of ideas in a short period of time, build prototypes quickly, validate APIs, integrate models, and cover a range of functionality that would have taken a single developer much longer to implement manually. The problem is that the speed borrowed through vibe coding must eventually be repaid by the developer. You can temporarily avoid reading the code. You can temporarily ignore the fact that the files are becoming larger and larger. You can allow AI to keep patching failed tests. But once the project becomes real, every structural problem that was ignored will return. AI can help me generate code, search code, reverse-engineer websites, add tests, and perform refactoring. But it cannot replace my responsibility to understand the system. Architectural boundaries, sources of truth, runtime rules, and final trade-offs must still be determined by me. In the end, the project did satisfy my requirements. However, compared with the final achievement of building a sports scores agent, the more valuable outcome was experiencing firsthand how a project could evolve from a local-model experiment, a simple chatbot, FAISS retrieval, an API knowledge base, a keyword-based decision flow, and multi-model fallback into a controlled, testable, and explainable agentic workflow. Many of the early implementations do not look elegant in retrospect. Some of the designs were wrong from the very beginning. But I do not believe the process was entirely wasted. If I had not personally encountered the problem of rules data consuming the entire context window, I would never have genuinely understood context engineering. If I had not experienced vector search selecting the wrong API, I would not have understood the difference between knowledge retrieval and tool planning. If I had not maintained a keyword-based decision flow, I would not have understood why natural-language intent cannot be exhaustively represented through rules. If I had not faced files containing tens of thousands of lines generated by Codex, I would not have truly appreciated human readability and module boundaries. This project was not completed in a single pass according to one correct architecture diagram. It only gradually became what it is today after I repeatedly discovered that “this approach does not work.” And that may be the most honest summary of my entire development experience: **AI can dramatically accelerate trial and error, but it cannot allow developers to skip growth. Every piece of understanding temporarily sacrificed for speed must eventually be recovered through their own effort.**