# I stopped letting LLMs guess financial facts > Source: > Published: 2026-08-16 12:26:59+00:00 LLMs can be surprisingly useful for company research. But I kept running into a strange split: parts of the reasoning were useful, while the financial facts underneath them were much harder to trust. A model could identify an accounting risk in one paragraph, then mix fiscal periods, accounting scopes, or currencies in the next. Missing values might quietly become zeros. A deterministic calculation could be performed probabilistically. A citation could point to a real filing without actually supporting the claim. Those are different failure modes, and treating all of them as one giant prompting problem did not feel like a reliable architecture. So I started building [OpenThesis](https://github.com/zjy1346/OpenThesis), an Apache-2.0 desktop system for evidence-first, AI-assisted company research. The project is not a stock picker or a trading bot. The idea is simpler: use ordinary software for work that should be deterministic, and give the LLM a bounded evidence set for the reasoning work where it can actually help. A common company-research workflow looks roughly like this: ``` company question ↓ LLM ↓ answer ``` That single model call is implicitly responsible for remembering reported values, selecting the right fiscal period, recognizing the accounting scope, finding sources, performing calculations, comparing scenarios, identifying risks, and writing a conclusion. Some of those tasks are probabilistic by nature. Others are not. Qualitative reasoning, connecting evidence, forming scenarios, and challenging an assumption are reasonable uses of a language model. Remembering an exact reported value, deciding whether a value is missing, and calculating a margin or valuation are poor places to accept probabilistic behavior. My design rule became: Deterministic work should stay deterministic. Use LLMs for reasoning, not as the database and calculator underneath the reasoning. OpenThesis starts from official filings rather than from model memory or a general web search. The current ingestion paths cover: The pipeline is intentionally staged: ``` official filings ↓ evidence extraction ↓ validated financial facts ↓ deterministic finance ↓ specialist research agents ↓ synthesis ↓ verification ↓ traceable research thesis ``` During a research run, extracted evidence is assigned IDs. Agents receive a bounded evidence set and are expected to attach those IDs to factual claims. Unknown evidence references and unsupported claims can then be flagged instead of being silently accepted because the prose sounds plausible. The final synthesis goes through another verification step. If the required evidence coverage is not there, the run can be marked partial rather than being presented as a fully supported report. This does not eliminate extraction errors or ambiguous filings. It makes the failure boundary more visible: a reader can inspect what was extracted, what the analysis used, and where support is missing. The same separation applies to financial analysis. OpenThesis builds financial summaries and reverse-DCF calculations in regular code. Missing values are not silently converted to zero. If core coverage is insufficient, the pipeline fails closed before asking an AI model to reason over an invented or incomplete numeric picture. That distinction matters more than sophisticated prompting. Consider revenue growth when one period is missing. A fluent model may still produce a percentage because completing patterns is what it does. A deterministic function can return “not available,” preserve the reason, and prevent the result from contaminating downstream analysis. The model can still discuss what a reported margin change might imply, compare scenarios, or challenge assumptions. It just does not get to manufacture the inputs or improvise the arithmetic. OpenThesis has separate roles for financial analysis, business analysis, accounting risk, growth, skepticism, forecasting, synthesis, and verification. But “more agents” is not the architectural claim. Eight agents with eight inconsistent versions of the facts would create more confidence without more reliability. The useful part is that the roles operate over the same research pack and evidence model. Their boundaries exist because the questions are different: In v1.3.0, specialist work is capped at two parallel agents. The goal is not to maximize agent count. It is to make the reasoning roles inspectable and to keep them attached to a shared set of facts. The research layer is model-agnostic. The current adapters include hosted providers such as OpenAI, DeepSeek, Gemini, Qwen, Kimi, GLM, and OpenRouter, plus Ollama and custom OpenAI-compatible endpoints. That is important for more than cost. It lets the evidence and deterministic-finance layers remain stable while models change. It also makes it possible to run with a local model through Ollama, or to inspect the deterministic pipeline with the offline synthetic demo before configuring any model. API keys are session-only in the current desktop app. They are not written to SQLite, settings, generated reports, or logs. When a hosted model is selected, research context is sent to that provider, so the privacy boundary still depends on the provider you choose. A generated report is difficult to evaluate if the run configuration disappears. OpenThesis records the provider and model, parameters, research configuration, evidence and data snapshot, and report language with the run. This is not perfect reproducibility—the behavior of a hosted model can change—but it preserves enough context to explain which data and configuration produced a thesis and to compare runs more honestly. The latest formal release is **v1.3.0**. There is a downloadable Windows x64 portable build, and the source combines Python for the research engine, React and TypeScript for the interface, and Rust/Tauri for the desktop shell. The offline synthetic demo is the lowest-friction way to inspect the pipeline without an API key or network dependency. A normal research run can ingest filings, create an evidence-backed research pack, run deterministic finance, execute the selected specialist roles, synthesize the result, and verify the final output. **AI-native, evidence-first company research for long-term investors** Research companies—not short-term price movements. OpenThesis is an open-source desktop research system for individual long-term investors. It turns public filings, deterministic financial analysis, and specialized AI agents into a traceable investment thesis. You choose the model OpenThesis provides the workflow, evidence protocol, financial tools, and reproducibility layer. Important OpenThesis does not connect to brokerage accounts, execute trades, provide short-term signals, or promise investment returns. Note **v1.2.0 is the current Windows test release.** In addition to US SEC research OpenThesis now researches China A-shares on SSE, SZSE, and BSE, plus Hong Kong listings on HKEX Main Board and GEM. It keeps issuer identity separate from each listed security, reads official financial-report PDFs, accepts explicitly dated manual market data, and treats banks, insurers, and securities firms as Financials Beta. OpenThesis is young, and there are real limitations: Those constraints are part of why I am sharing the project now. I would especially value feedback on the boundary between extraction and evidence, the deterministic checks that should be added next, and filing-analysis failure cases that current verification approaches tend to miss. OpenThesis is open source under Apache-2.0. If separating evidence from LLM reasoning is useful to you, [take a look at the repository](https://github.com/zjy1346/OpenThesis). Feedback, issues, contributions, and stars are all appreciated.