SynnoDB โ€“ Synthesizing Database engines for your workloads SynnoDB, a drop-in replacement for DuckDB that transparently accelerates SQL with auto-generated bespoke C++ engines, is now available on PyPI. The project packages the Bespoke-OLAP research into a production-ready router that falls back to DuckDB for everything else and cross-checks correctness. SynnoDB uses an LLM agent to synthesize workload-specific query engines and supports stages like storage planning, base implementation, optimization, multi-threading, and correctness checking. A drop-in replacement for DuckDB that transparently accelerates your SQL with auto-generated bespoke C++ engines - falling back to DuckDB for everything else, cross-checked for correctness. ๐ŸŒ Website https://synnodb.com ยท ๐Ÿ“„ Paper https://arxiv.org/pdf/2603.02001 ยท ๐Ÿ“ฆ PyPI https://pypi.org/project/synnodb/ ยท ๐Ÿ““ Demo Notebook /SynnoDB/SynnoDB/blob/main/tutorials/gen tpch demo.ipynb Required Notice: Copyright 2026 SynnoDB SynnoDB grew out of the research project Bespoke-OLAP https://github.com/DataManagementLab/BespokeOLAP paper https://arxiv.org/pdf/2603.02001 : an LLM agent that synthesizes workload-specific, one-size-fits-one C++ query engines. SynnoDB packages that idea as a production-ready DuckDB drop-in. Install from PyPI : pip install synnodb the demo DuckDB drop-in router pip install "synnodb factory " + the Bespoke-Agent factory that generates engines New here? tutorials/gen tpch demo.ipynb /SynnoDB/SynnoDB/blob/main/tutorials/gen tpch demo.ipynb runs the whole loop end to end - generate TPC-H data, build an engine, and drop it in against DuckDB. See Installation installation for the system libraries the generated engines compile against. SYNNO DATA DIR must point at the data root parquet, caches, logs ; set it in the environment or .env . python from synnodb import SynnoDB db = SynnoDB.in memory workload="tpch", model="anthropic/claude-sonnet-4-6" plan = db.createStoragePlan queries="1" - StoragePlan print plan.text the storage plan.txt document print plan.path, plan.run id on disk + wandb provenance impl = db.createBaseImpl storage plan=plan.text pass the plan content W&B-free print impl.files "db loader.cpp" - BaseImplementation generated C++ opt = db.runOptimLoop base impl=impl - OptimizedImplementation Each stage returns a domain object StoragePlan , BaseImplementation , OptimizedImplementation , MultiThreadedImplementation , CorrectnessReport that carries the produced artifact and chains into the next stage. SynnoDB ... takes enums or strings db storage="ssd" , alternative constructors in memory / on ssd / for tpch / for ceb / from env , and with ... for per-call overrides. Every stage is a method on SynnoDB ; there are no per-stage scripts. Each call runs one stage to completion and returns an artifact that chains into the next. Stages chain in-process pass the artifact or across runs via the W&B run id wandb id= , requires wandb entity / wandb project on the producing run : python from synnodb import SynnoDB db = SynnoDB.on ssd workload="tpch", queries="1-22", model="anthropic/claude-sonnet-4-6", notify=True, wandb entity="my-entity", presence of entity/project enables W&B plan = db.createStoragePlan - StoragePlan impl = db.createBaseImpl storage plan wandb id="8xn0t04p" or storage plan=plan opt = db.runOptimLoop base impl wandb id="q45vm9fz" or base impl=impl mt = db.addMultiThreading optimized wandb id="0br4bjqb" or optimized=opt rep = db.checkSfCorrectness source wandb id="0br4bjqb", target sf=50 The run output dir defaults to a local ./output ; set workspace= or SYNNO WORKSPACE . Any RunConfig setting the typed config does not model can be forced through the extra config={...} escape hatch. The built-in stages are ordinary ConversationPlan s; you can assemble and run your own conversation from the same primitives via run synthesis , the single entry point every stage goes through: python from synnodb import AssertCorrect, Benchmark, Compact, ConversationPlan, ConvContext, PerQueryLoop, PrepareFeatures, PromptStage, SynnoDB, db = SynnoDB.in memory workload="tpch", queries="1,4,6" def my stages ctx: ConvContext : return AssertCorrect , PromptStage descriptor="inspect hot loops", get prompt=lambda exec settings, rt: f"Profile {ctx.filenames.query impl path} and summarize the hot loops." , measure performance after stage=False, auto revert on regression=False, , Compact , PerQueryLoop lambda qid, ctx: PromptStage descriptor=f"tune {qid}", runtime and tracing data arrive exactly as in the built-in stages get prompt with tracing=lambda exec settings, rt, trace: f"Query {qid} currently runs in {rt:.0f} ms.\n" f"Trace:\n{trace}\nOptimize it." , max turns=125, defaults: measure after stage, auto-revert on regression , , Benchmark , plan = ConversationPlan name="myTuningPass", run identity: naming, logging, caching prepare=PrepareFeatures tracing=True , what the workspace must provide stages=my stages, result = db.run synthesis plan, start=base impl start: artifact | snapshot hash | None prepare states what the workspace must have scaffold, tracing instrumentation, MT helpers, ... as independent feature flags; the features actually applied are recorded in a git-tracked .synnodb prepare.json inside every snapshot, so chained runs know what they start from. stages receives a ConvContext queries, workspace filenames, run tool, lazy helpers like ctx.reference plans source="umbra" and returns a flat list of stage items. PerQueryLoop runs one conversation branch per query, feeding each stage the freshly measured runtime and trace data.- The returned artifact carries the final snapshot hash and the prepare record, so it chains into db.checkSfCorrectness result, target sf=100 or further custom plans with no extra ceremony. SynnoDB is published on PyPI , so a single pip command pulls in every Python dependency - no need to manage them yourself: pip install synnodb the DuckDB drop-in router / runtime pip install "synnodb factory " + the LLM factory that generates engines That is everything needed to import synnodb . Two things live outside the wheel: The generated engines are compiled C++ against Apache Arrow / Parquet, so a toolchain and the dev headers must be present. cloc https://github.com/AlDanial/cloc is optional - the factory uses it to report generated-code size. On Debian/Ubuntu: sudo apt install -y build-essential cloc C++ compiler + optional cloc Apache Arrow + Parquet development libraries wget https://packages.apache.org/artifactory/arrow/$ lsb release --id --short | tr 'A-Z' 'a-z' /apache-arrow-apt-source-latest-$ lsb release --codename --short .deb sudo apt install -y -V ./apache-arrow-apt-source-latest-$ lsb release --codename --short .deb sudo apt update sudo apt install -y libarrow-dev libparquet-dev parquet-tools Linux x86-64, Python 3.13+. Create a .env in your working directory with the model credentials and optional run tracking : ANTHROPIC API KEY=... for the default anthropic/... models OPENROUTER API KEY=... for openrouter/... models LLM API BASE=http://host:PORT/v1 a self-hosted, OpenAI-compatible endpoint WANDB ENTITY=... WANDB PROJECT=... optional Weights & Biases run tracking Point SYNNO DATA DIR at the data root that holds the parquet, caches, and published engines. The CLI and API require it - export it, put it in .env , or pass data dir=... to SynnoDB ... . The demo notebook /SynnoDB/SynnoDB/blob/main/tutorials/gen tpch demo.ipynb is self-contained: it defaults to a project-local .synno data/ when the variable is unset and generates its own TPC-H parquet, so there is nothing else to configure or download to run it. Building from a source checkout for contributors uses uv https://github.com/astral-sh/uv instead of pip - it manages the virtualenv and the optional-dependency extras: curl -LsSf https://astral.sh/uv/install.sh | sh install uv git clone https://github.com/JWehrstein/SynnoDB.git cd SynnoDB uv sync --extra factory --extra dev editable install: factory + test deps The extras map to pyproject.toml : factory the engine-generation stack plus the standalone dashboard's wandb , dev pytest , notebook Jupyter kernel + nbformat , and benchmark the ClickHouse comparison . Install only what you need, e.g. uv sync for the runtime alone or uv sync --extra factory to generate engines. Still install the system libraries 1-system-libraries above. Run the test suite with .venv/bin/python -m pytest . watch -n1 -d ./misc/get db procs.sh To share snapshots across machines, set up a bare git repository and start a git daemon: git init --bare synno cache.git touch synno cache.git/git-daemon-export-ok git daemon \ --base-path=./ \ --export-all \ --enable=receive-pack \ --reuseaddr \ --verbose The cache URL is git://