{"slug": "building-an-open-source-turbopuffer-alternative-in-rust-kosha", "title": "Building an open source (turbopuffer alternative)in Rust – Kosha", "summary": "Kosha, an open-source storage-disaggregated search engine built in Rust as an alternative to Elasticsearch/OpenSearch, has completed Phase 1 with BM25 lexical and kNN/ANN vector search implemented across all seven crates. The engine uses S3 as the source of truth with local NVMe SSD as a transparent cache, and is available via Docker image 'ravidecoverai/kosha', PyPI package 'kosha-client', and a CLI. Features include hybrid retrieval, filters, aggregations, and S3-compatible storage, with Phase 2 planned for RRF fusion and rerank.", "body_md": "A storage-disaggregated search engine: **S3 is the source of truth, local NVMe\nSSD is a transparent cache, and compute nodes are disposable.** Kosha is being\nbuilt to replace Elasticsearch/OpenSearch and is intended to be reusable as a general-purpose, schema-driven search service.\n\nSee [DESIGN.md](/decover-tech/kosha/blob/main/DESIGN.md) for the full architecture.\n\nStatus: Phase 1 complete — BM25 lexical + kNN/ANN search implemented.All seven crates have functional BM25 indexing, query, filtering, aggregation, and HNSW vector search. RRF fusion and rerank are Phase 2 (DESIGN.md §3.1).\n\n| Artifact | Where | Get it |\n|---|---|---|\n| Server image |\n`ravidecoverai/kosha` |\n\n`docker pull ravidecoverai/kosha:latest`\n\n[PyPI —](https://pypi.org/project/kosha-client/)`kosha-client`\n\n`pip install kosha-client`\n\n`cargo install --path crates/kosha-cli`\n\n| Feature | Notes |\n|---|---|\n| BM25 lexical search | Tunable `k1` / `b` per request (defaults 1.2 / 0.75) |\n`operator: and` / `or` |\nAND (default) requires every query term; OR scores the union |\n| Block-max WAND | Skip-list postings, rarest-first probe ordering, leapfrog AND join |\n`match_phrase` |\nPositional postings, configurable `slop` |\n`wildcard` |\nPer-field pattern match, case-insensitive by default |\n| kNN / ANN vector search | Cluster-and-posting on-disk vector index with centroid probing and triangle-inequality lower bounds; global probe budget; flat/HNSW fallback for older segments |\n| Hybrid retrieval | A `knn` clause and `query_text` in one request; vector and BM25 hits merged per segment |\n| Filters | `term` , `terms` , `range` (`gte` /`gt` /`lte` /`lt` ), `bool` (`must` / `must_not` / `should` / `minimum_should_match` ), `match_all` |\n| kNN-scoped filters | `knn.filter` restricts vector candidates; the top-level `filter` still governs the merged result set |\n| Sorting | Multi-field `sort` over stored fields, plus `_id` |\n| Pagination | `from` / `max_results` , and an OpenSearch-style `search_after` cursor |\n| Highlighting | Per-field, with configurable `pre_tags` / `post_tags` |\n| Aggregations | `terms` , `cardinality` , `composite` |\n| Total-hit accounting | Capped counting with an `eq` / `gte` relation (`track_total_hits` -style), overridable per query via `exact_total_hits` / `total_hits_cap` |\n| Degradation signal | `knn_degraded_segments` reports segments whose vector search failed, so a 200 with silently missing neighbors is visible to the caller |\n\n- Bulk document indexing, with field types\n`Text`\n\n,`Keyword`\n\n,`Integer`\n\n,`Float`\n\n,`Date`\n\n,`Boolean`\n\n,`Vector`\n\n. - Upsert by document id — segments holding a prior version of an id are rewritten.\n- Delete by query (tombstone-based) and a document\n`exists`\n\ncheck. - Write-ahead log for buffered documents, replayed on restart.\n- Auto-flush at a configurable document threshold (\n`KOSHA_FLUSH_THRESHOLD`\n\n), plus an explicit`flush`\n\n. - Size-tiered compaction with a cap on merged-segment size (5 GiB default), triggered by the admin endpoint or the compaction CronJob.\n\n- S3 as the source of truth; any S3-compatible endpoint works (MinIO locally, path-style supported).\n- Local NVMe read-through cache, size-bounded with LRU eviction.\n- Lazy segment loading — the doc store, filter columns, and postings are read on demand, with ranged GETs for doc-store pages instead of whole-blob fetches.\n- In-memory parsed-segment cache governed by a live-bytes ledger and an admission gate, with a per-request hydration byte budget and bounded hydration concurrency.\n- Posting-blob presence cache, postings cache, and vector-postings cache.\n- Whole-response result cache for\n`POST /search`\n\n(bypassable per query with`no_cache`\n\n). - Cross-replica hydration leases in Postgres: one pod fetches a cold segment from S3, its peers stream the bytes from it over\n`GET /internal/segment/...`\n\nrather than stampeding S3. - Bloom filters over terms and filter fields, so a segment that cannot match is skipped without being read.\n- Namespace warmup on boot, gated behind\n`/readyz`\n\nso traffic is not routed to a cold pod.\n\n- Namespace registry and manifest store, in-memory by default or Postgres-backed (\n`postgres`\n\nfeature +`DATABASE_URL`\n\n), with compare-and-swap manifest publishes. - Multi-tenancy: every API key maps to a tenant prefix that scopes all namespace access.\n- Read/write split: query pods forward every mutating route to\n`KOSHA_INGEST_HOST`\n\nand serve reads locally. - Backpressure controls: max concurrent searches, search queue depth and timeout, hydration concurrency, admission timeout.\n`GET /healthz`\n\n(liveness),`GET /readyz`\n\n(readiness),`GET /v1/stats`\n\nand per-namespace stats.- Admin endpoints: create API key, rebuild filter blooms, backfill offset tables, compact a namespace, import a namespace.\n- Schema migrations via\n`kosha-server migrate`\n\n.\n\n- HTTP/JSON\n`/v1`\n\nAPI —`documents`\n\n,`search`\n\n,`flush`\n\n,`delete`\n\n,`exists`\n\n,`stats`\n\n— with the Phase 1 unversioned routes still served for backward compatibility. `proto/kosha/v1/kosha.proto`\n\nis the canonical API contract and the source for generated stubs and the OpenAPI spec.`kosha`\n\nCLI (built from`crates/kosha-cli`\n\n, not yet on crates.io): health, index, search, flush, delete, stats, admin commands, named profiles,`--json`\n\noutput, and a`kosha curl`\n\nescape hatch.- OpenSearch-compatible Python client,\n:`kosha-client`\n\non PyPI`search`\n\n,`index`\n\n,`bulk`\n\n,`count`\n\n,`update`\n\n,`delete_by_query`\n\n,`update_by_query`\n\n,`scroll`\n\n, plus`indices`\n\nand`tasks`\n\nnamespaces, translating the ES query DSL to Kosha's native shape.\n\n- RRF fusion and reranking (Phase 2, DESIGN.md §3.1).\n- Configurable analyzers — tokenization is fixed (whitespace split, ASCII-punctuation trim, lowercase). The\n`analyzer`\n\nfield exists in the schema proto but is not honored. - gRPC — the service is defined in the proto, but HTTP/JSON is the only implemented transport.\n`kosha-vector-spfresh`\n\n(SPFresh/SPANN with LIRE rebalancing) is a standalone, benchmarked prototype; it is not wired into the segment format or the query path.\n\n```\ncrates/\n  kosha-core      shared types, data model, filter/query DSL  — Epic 2\n  kosha-segment   segment format: inverted idx, doc store,    — Epic 2\n                 filter columns, vector store, HNSW graph\n  kosha-write     document buffer + flush-to-segment          — Epic 3\n  kosha-cache     NVMe SSD read-through cache (§9)            — Epic 4\n  kosha-query     BM25 scorer, kNN/ANN search, aggregations,  — Epic 5\n                 wildcard, match phrase, filtering\n  kosha-control   in-memory namespace + manifest store        — Epic 6\n  kosha-server    HTTP API (healthz, index, search, stats,    — Epic 8\n                 delete, flush)\n  kosha-cli       Remote `kosha` CLI (profiles, search,       — Epic 12\n                 index, curl escape hatch)\nclients/\n  python/kosha_client  OpenSearch-compatible Python client    — Epic 11\n                       (spec → codegen → thin client)\n  python/README.md     how the client is structured\nproto/\n  buf.yaml             Buf module config\n  kosha/v1/kosha.proto  Canonical API contract (source of truth) — Epic 1\ngen/                   Generated stubs + OpenAPI spec (git-ignored)\ntools/codegen/         Code generation scripts and docs\ndocs/                  Development and integration guides\nDESIGN.md              Architecture document (v1 draft)\n# 1. Start Kosha locally\ndocker compose up --build\n\n# 2. Install the CLI (or use cargo run -p kosha-cli -- …)\ncargo install --path crates/kosha-cli\n\n# 3. In another terminal, explore the index\nexport KOSHA_HOST=http://localhost:8080\nexport KOSHA_API_KEY=sk-kosha-dev\n\nkosha health\nkosha index -n quickstart-demo --file crates/kosha-cli/examples/docs.jsonl\nkosha flush -n quickstart-demo\nkosha search -n quickstart-demo \"breach\"\nkosha stats -n quickstart-demo\n```\n\nSee [crates/kosha-cli/README.md](/decover-tech/kosha/blob/main/crates/kosha-cli/README.md) for profiles\n(`~/.kosha/config.toml`\n\n), `--json`\n\noutput, and the `kosha curl`\n\nescape hatch.\n\nThe Python client ([ kosha-client on PyPI](https://pypi.org/project/kosha-client/))\nstill works the same way for application code:\n\n```\npip install kosha-client\npython\nfrom kosha_client import KoshaClient\n\nclient = KoshaClient(\n    hosts=\"https://app.kosha.io\",\n    api_key=\"sk-acme-corp-xxx\",\n)\nclient.ping()  # True\n```\n\nBoth the CLI and the Python client respect `KOSHA_HOST`\n\nand `KOSHA_API_KEY`\n\n.\n`scripts/quickstart.py`\n\nremains as a scripted alternative to the CLI flow.\n\nPrerequisites: Rust stable (via [rustup](https://rustup.rs) — the repo pins the\nchannel via `rust-toolchain.toml`\n\n), Docker, and optionally `pre-commit`\n\n.\n\n```\ncargo build                                # build the workspace\ncargo test                                 # run unit tests\ncargo fmt --all -- --check                 # formatting (CI gate)\ncargo clippy --all-targets -- -D warnings  # linting (CI gate)\npre-commit install                         # optional: run gates on commit\ndocker compose up -d minio createbuckets\n```\n\nMinIO serves an S3-compatible API on `localhost:9000`\n\n(web console on `:9001`\n\n,\ncredentials `kosha`\n\n/ `kosha-dev-secret`\n\n) with the `dsearch-dev`\n\nbucket\nauto-created.\n\nPre-built images are published to Docker Hub at\n[ ravidecoverai/kosha](https://hub.docker.com/r/ravidecoverai/kosha) —\n\n`:main`\n\non every merge to `main`\n\n, and `:latest`\n\nplus the semver tags (`:0.1.0`\n\n, `:0.1`\n\n)\nfor every `v*`\n\ntag. Tagged releases are multi-arch (amd64 + arm64); `:main`\n\nis\namd64 only.\n\n```\ndocker pull ravidecoverai/kosha:latest\ndocker run --rm -p 8080:8080 ravidecoverai/kosha:latest\ncurl localhost:8080/healthz   # -> ok\n```\n\nOr build locally:\n\n```\ndocker build -t kosha:latest .\ndocker run --rm -p 8080:8080 kosha:latest\n```\n\nOr bring up the whole local stack (MinIO + server): `docker compose up --build`\n\n.\n\nThe backend repo's Tilt setup builds and deploys `kosha`\n\ninto the local k8s\ncluster as `kosha-service`\n\n(`:8080`\n\nHTTP, `:50051`\n\ngRPC). See\n[docs/local-development.md](/decover-tech/kosha/blob/main/docs/local-development.md) — including how the\nOpenSearch → Kosha swap will work once the read/write path lands.\n\nSee [CONTRIBUTING.md](/decover-tech/kosha/blob/main/CONTRIBUTING.md).\n\nApache-2.0 (see [LICENSE](/decover-tech/kosha/blob/main/LICENSE)). The final open-source license choice is\nconfirmed before the v0.1 release (implementation plan step 126).", "url": "https://wpnews.pro/news/building-an-open-source-turbopuffer-alternative-in-rust-kosha", "canonical_source": "https://github.com/decover-tech/kosha", "published_at": "2026-08-31 07:28:43+00:00", "updated_at": "2026-08-31 07:52:47.936805+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-tools", "ai-research"], "entities": ["Kosha", "Elasticsearch", "OpenSearch", "S3", "NVMe SSD", "HNSW", "BM25", "PyPI"], "alternates": {"html": "https://wpnews.pro/news/building-an-open-source-turbopuffer-alternative-in-rust-kosha", "markdown": "https://wpnews.pro/news/building-an-open-source-turbopuffer-alternative-in-rust-kosha.md", "text": "https://wpnews.pro/news/building-an-open-source-turbopuffer-alternative-in-rust-kosha.txt", "jsonld": "https://wpnews.pro/news/building-an-open-source-turbopuffer-alternative-in-rust-kosha.jsonld"}}