Itzma, an embedded vector database backed by object storage Itzma released version 0.1 of its embedded vector database, which stores vectors as plain objects in an S3-compatible bucket and runs as a library inside the caller's process rather than as a server, with Rust and Python APIs. The library relies on the object store's put-if-absent and conditional update operations for correctness, and supports namespaces with fixed dimension and metric (L2, cosine, dot product), dense search via HNSW (the default) or SQ8 with clustered partitions, sparse vectors of (term id, weight) pairs, typed attribute filtering, batched upserts and deletes with optional idempotency keys, and compaction, garbage collection and manifest recovery. Itzma is available as the `itzma-core` crate on crates.io and as `pip install itzma` wheels for macOS and Linux on Python 3.10 and later, with S3, local disk and in-memory stores supported through `object_store` 0.14. Itzma is vector retrieval on top of object storage, written in Rust, with Python bindings. Vectors are stored as plain objects in a bucket. There is no server between the caller and the data: any process that can reach the bucket can write, and any process can query. Correctness rests on two things the object store already does, put-if-absent and conditional update. It is a library that runs in your process. Open a namespace, and S3 or a local directory, or memory holds the data. - Namespaces with a fixed dimension and metric L2, cosine, dot product . - Dense vector search, exact or through an index: HNSW the default or SQ8 with clustered partitions. - Sparse vectors of term id, weight pairs beside the dense one. Sparse queries are exact, and the inverted index is optional. - Typed attributes declared at create. Filter on them alone, or use them to narrow a vector query. - Upserts and deletes go in batches, and many writers can send them at once. Idempotency keys are optional. - S3 and anything else object store supports , local disk and in-memory stores. - Compaction, garbage collection and manifest recovery. - A RAM and NVMe read cache and a cache of decoded structures. - Rust and Python APIs; NumPy and Arrow input in Python. - A read-only health report and a byte budget on every read. Rust: add itzma-core from crates.io. Python: pip install itzma wheels for macOS and Linux, Python 3.10 and later; see py-itzma https://github.com/sergiommarcial/itzma/blob/main/py-itzma/README.md . This example uses the in-memory store. dependencies itzma-core = "0.1" object store = "0.14" tokio = { version = "1", features = "macros", "rt-multi-thread" } serde json = "1" use std::sync::Arc; use itzma core::{AttrSchema, AttrType, Document, Metric, Namespace, Schema}; use object store::memory::InMemory; tokio::main async fn main - itzma core::Result< { let store = Arc::new InMemory::new ; // Dimension, metric and the type of every attribute are fixed for the life // of the namespace. let schema = Schema::new 3, Metric::Cosine .with attrs AttrSchema::new .field "title", AttrType::Text ; let ns = Namespace::create store.clone , "articles", schema .await?; // One call is one atomic write. It returns the sequence number. let seq = ns .upsert vec Document { id: "intro".into , vector: vec 0.9, 0.1, 0.0 , attrs: Some serde json::json { "title": "Introduction" } , sparse: None, }, Document { id: "faq".into , vector: vec 0.0, 1.0, 0.2 , attrs: None, sparse: None, }, .await?; assert eq seq, 1 ; // Closest first. Lower distance means closer, for every metric. let hits = ns.query & 1.0, 0.0, 0.0 , 2 .await?; assert eq hits 0 .id, "intro" ; ns.delete vec "faq".into .await?; // Another handle, possibly in another process, sees the same data. let other = Namespace::open store, "articles" .await?; assert eq other.query & 0.0, 1.0, 0.0 , 5 .await?.len , 1 ; Ok } The same in Python build it with make py-dev , see py-itzma/README.md https://github.com/sergiommarcial/itzma/blob/main/py-itzma/README.md : python import itzma from itzma import attr store = itzma.Store "memory://" or "file:///var/lib/itzma", "s3://bucket/prefix" schema = itzma.Schema dim=3, metric=itzma.Metric.COSINE, attrs={"year": itzma.AttrType.NUMBER} ns = itzma.Namespace.create store, "articles", schema ns.upsert {"id": "intro", "vector": 0.9, 0.1, 0.0 , "attrs": {"year": 2024}}, {"id": "faq", "vector": 0.0, 1.0, 0.2 , "attrs": {"year": 2021}}, hits = ns.query 1.0, 0.0, 0.0 , top k=2, filter=attr "year" = 2022 print hits.ids, hits.distances Writes also take NumPy arrays ids= , vectors= , attrs= and Arrow, Polars or DuckDB tables. Namespace. A collection of documents that share a schema. On storage it is a key prefix,