I built an autonomous accounting tool to let AI do my taxes

A developer released Autonomous Accounting, an open-source tool that uses a user-selected vision LLM to convert receipts, invoices and bank statements into a reconciled, categorized ledger plus a downloadable audit binder. The tool runs entirely on one machine, requires Python 3.11+, Node 20+ and PostgreSQL 15+, and supports any OpenAI-compatible vision endpoint including vLLM, llama.cpp, Ollama's /v1 and LM Studio; it hosts no model of its own. Reconciliation uses four matching passes, uncertain items are flagged for human review rather than guessed, and documents are sent only to the endpoint the user configures via LLM_BASE_URL.

Watch the demo see-it-work · Quickstart quickstart · How it works how-it-works · Your data https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/data-schema.md · Known issues go-deeper Autonomous Accounting turns a pile of receipts, invoices and bank statements into a reconciled, categorized ledger — plus an audit binder with every source document attached. A vision LLM you choose reads the documents, a four-pass matcher pairs them with your bank lines, and anything it is not sure of is flagged for you instead of guessed. Everything runs on one machine you own: no account to create, no vendor to pay, no copy of your books anywhere but your own disk. Two minutes twenty: one invented company's January, from eight documents and a bank CSV to a downloadable audit binder. Watch the full demo 2:20, captioned, no sound https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/media/demo.mp4 <sub Everything shown is synthetic. python scripts/gen demo data.py writes the same month on your machine — see Try the demo month try-the-demo-month .</sub | A review queue, not a black box. Exact pairs approve themselves; date gaps and cross-currency pairs wait for you. | A ledger with CRA/GIFI categories. Monthly and annual, with a GST/HST/PST summary and QuickBooks- and Xero-style CSVs. | An audit binder in one ZIP. The XLSX workbook, a self-contained HTML report, and the document behind every line. | - Extraction. Receipts, invoices and PDF bank statements are rasterised and read by a vision LLM into structured JSON vendor, date, amount, tax, currency , with totals cross-checked against line items. Uploads queue and answer 202 ; a background worker drains the queue, so dropping half a year of documents at once cannot time out, and a restart re-queues whatever was mid-flight. - Reconciliation. Four passes over the unmatched set: strict same-currency, FX-relaxed inside a configurable band, an aggressive pass for splits and installments, and an LLM pass that links across accounts. - Categorization. A deterministic regex pre-pass config/category rules.yaml locks what it recognises, vendor clustering normalises descriptors, and only the remainder reaches the model, under a per-run call budget. - Reports. Monthly and annual ledgers, a GST/HST/PST tax summary, a PDF report, QuickBooks- and Xero-style CSVs, a shareable read-only link, the ZIP audit binder — and an optional "reasonableness" pass that compares your expense ratios against published Canadian industry data. - It flags; it does not guess. A wrong number in a tax filing is worse than a blank. Bank fees, interest and internal transfers are excluded rather than force-matched. - It never sums across currencies. A USD invoice paid in CAD is an FX match for review, never an "exact" one. - Your documents go only where you point them. Set LLM BASE URL to a server on your own machine or LAN and nothing leaves. An empty provider key means off ; nothing silently falls back to a paid provider, and GET /health shows where an upload would go. If the endpoint is unreachable, jobs park in waiting for model and resume by themselves. - No telemetry, no analytics, no webfonts. Loading a page fetches nothing from anyone else. The log files on your disk are the only record. - Row-level security on every table , keyed on the signed-in user and enforced from the catalogue by a test. The long form — backups, encryption at rest, what reaches a model — is in Honesty about data https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/data-handling.md . You need Python 3.11+ , Node 20+ , PostgreSQL 15+ on loopback, and any OpenAI-compatible, vision-capable endpoint — vLLM, llama.cpp, Ollama's /v1 , LM Studio. This project hosts no model of its own. python -m venv .venv && . .venv/bin/activate Windows: .venv\Scripts\activate pip install -r requirements.txt cp .env.example .env then fill in the three values it names createdb autonomous accounting the order below is not interchangeable psql -d autonomous accounting -f db/local auth schema.sql psql -d autonomous accounting -f db/schema pg.sql python -m db.migrate psql -d autonomous accounting -f db/local grants.sql npm --prefix web ci && npm --prefix web run build python -m uvicorn server.app:app --port 8080 then open http://localhost:8080 Registration is closed by default. Set AUTH ALLOW SIGNUP=1 , restart, sign up, then set it back to 0 . GET /health reports database, storage and LLM reachability. The three .env values, and how to generate the secrets .env values, and how to generate the secrets Generate each secret exactly as .env.example says: python python -c "import secrets;print secrets.token urlsafe 48 " AUTH JWT SECRET python -c "import secrets;print secrets.token urlsafe 48 " STORAGE URL SECRET and set DATABASE URL to the database you are about to create. Every other variable, with its default, is in .env.example . AUTH AUTOCONFIRM=1 the default means a new account works immediately without any mail transport. Why the schema files run in that order local auth schema.sql must run first: it creates the auth schema, auth.uid and the authenticated role, all of which schema pg.sql needs. schema pg.sql is the whole schema — every table, index, trigger and row-level-security policy — as a single baseline; db/migrations/ ships empty and python -m db.migrate therefore has nothing to do on a fresh install, but run it anyway so anything added after the baseline is picked up. local grants.sql must run last: it grants on the tables the steps above created, and without it every query fails with permission denied . Each step is idempotent, so re-running the sequence on an existing database changes nothing. Nothing in db/ needs a server feature newer than PostgreSQL 13; 15 is just the oldest release still maintained upstream. Developing the frontend with the Vite dev server FastAPI serves the built bundle from the same origin. For frontend work, run the Vite dev server instead, which proxies /api to the backend: cd web npm ci npm run dev http://localhost:5174 Peer-benchmark data optional, one command The reasonableness engine needs a benchmark table, which is not committed: python scripts/ingest ised benchmarks.py --year 2024 --out config/ised benchmarks.json That downloads the Financial Performance Data CSVs published by Innovation, Science and Economic Development Canada on the Government of Canada Open Government Portal, normalises them into per-industry cohort cells, validates them fail-loud, and writes a large JSON file that .gitignore excludes. The data is licensed under the Open Government Licence – Canada ; the generated file and every report built from it carry the attribution the licence requires. No Open Government Licence data file is distributed with this repository — you build the table yourself. See NOTICE . Every run validates structure and plausibility. --anchor anchor.json adds an optional check that one named cell still carries the figures you expect, so a refresh that quietly changes what a cell means aborts instead of overwriting your table; with no anchor supplied, no cell-specific figures are demanded. --seed-only writes a small table of invented example figures instead, with no download — enough for the engine and its tests to run, and labelled in its own meta as not being industry data. Skip the step entirely and the feature stays off; nothing else is affected. The files under tests/fixtures/ are per-parser unit fixtures; uploaded through the UI they match nothing, because no receipt among them belongs to any bank line. To see the whole pipeline work, generate a coherent synthetic month instead: python scripts/gen demo data.py writes data/demo/, which is gitignored That writes one invented company's January 2026 — a 12-row CAD statement in the built-in CSV layout, seven PDFs and one PNG — plus a README.txt saying what each file is there to show. Then, in the app: 1. Set OWN COMPANY PATTERNS=example corp in .env and restart, so the invoice issued by the demo company is read as income and can match the incoming wire. 2. Sign up, choose Manitoba as the province the demo documents charge GST and RST , and upload demo bank cad.csv as the statement. 3. Upload all eight documents at once. They queue; each takes roughly a minute on a 27B-class local vision model, less on a hosted one. 4. Press Match Receipts . Expect exact same-day pairs to be auto-approved without a model call, a six-day date gap and a USD invoice paid in CAD to land in Review , the monthly fee, interest and inter-account transfer to be marked as needing no receipt, two bank lines left asking for a document, and a cash receipt left unmatched. 5. Approve the pending pairs in Review , then download the audit binder from Reports → January 2026 and open proof of transaction/ and index.html . What that run looks like on one model is written up under Verification gaps in docs/backlog.md , next to the defects it found. - Not a hosted service. Nothing to sign up for. You run it or it does not run. - Not something you buy. The software takes no payment. Every signed-in account has full access to every feature. - Not supported. No warranty, no SLA, no security response commitment, no promise that an upgrade will not want a manual migration. See the licence. - Not a filing service. It produces a ledger and reports for a Canadian small corporation, and models GST/HST/PST only. It files nothing with the CRA and it is not accounting advice. Check the numbers before you use them. | Layer | What runs | |---|---| | API + web app | FastAPI on uvicorn, port 8080 by default; serves /api/ and the built SPA from one origin | | Database | PostgreSQL with row-level security keyed on the signed-in user | | File storage | A directory on local disk, served only through short-lived HMAC-signed URLs | | Frontend | React 19 + Vite + TypeScript, Tailwind, TanStack Query, Zustand | | Extraction | Any OpenAI-compatible, vision-capable endpoint you point LLM BASE URL at. Gemini and OpenAI are optional fallbacks that stay inert while their keys are empty | | Auth | Local email + password server/local auth.py : HS256 access tokens, rotating refresh tokens, PBKDF2-HMAC-SHA256 hashes. Sign in with Google is optional | | Bring your own data https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/data-schema.md | The six YAML tables under config/ you are expected to edit, every record shape and status vocabulary, the built-in bank parsers BMO, Wise, PayPal, Amazon; everything else goes through the LLM statement parser , and the minimum CSV a hand-made statement needs | | Honesty about data https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/data-handling.md | Where everything sits, what is and is not encrypted, backups, what reaches a model | | Known issues https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/backlog.md | Open engineering items. Read Security hardening before exposing an instance beyond loopback: the defaults — one owner, loopback only, registration closed — are what this was reviewed for | | Testing https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/docs/testing.md | ./scripts/gates.sh runs ruff, pytest, typecheck and build — there is no CI. How pytest isolates its database, and what the synthetic fixtures do not prove | | Working on it with a coding agent https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/.claude/README.md | CLAUDE.md and AGENTS.md are the entry points; .claude/ is an optional Claude Code harness that denies agent reads of .env , data/ and logs/ . Nothing in the application depends on it | One thing a stranger will meet early: Gmail's OAuth redirect has to match what you registered. The code builds <BASE URL /api/onboarding/gmail/callback , and BASE URL defaults to http://localhost:8080 . If you serve the app anywhere else, set BASE URL or GMAIL REDIRECT URI before connecting Gmail, and register the same value with Google. Gmail, PayPal and Wise gather integrations stay hidden until you configure credentials. See CONTRIBUTING.md https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/CONTRIBUTING.md : synthetic fixtures only, ruff and tsc clean, one topic per pull request, sign off your commits git commit -s . The most useful contribution is a statement parser or a category rule for a layout the engine does not read yet — built from invented data, never real books. If this saved you a bookkeeping afternoon, a star helps the next small-business owner find it. Autonomous Accounting — self-hosted bookkeeping with LLM document extraction and bank reconciliation. Copyright C 2026 Autonomous Accounting contributors This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or at your option any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/ https://www.gnu.org/licenses/ . Full text in LICENSE https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/LICENSE . Because this is the AGPL, running a modified version as a network service obliges you to offer its source to its users. Third-party attributions — including PyMuPDF, itself AGPL — are in NOTICE https://github.com/CodeGameDev29/KFAutonomousAccounting/blob/main/NOTICE .