# Rayforce > Source: > Published: 2026-06-06 19:01:32+00:00 Columnar analytics and graph traversal in one fused pipeline. Rayforce is a pure C17 zero-dependency embeddable engine where columnar analytics and graph traversals share a single operation DAG, pass through a multi-pass optimizer, and execute as fused morsel-driven bytecode. No malloc. ``` make # debug build (ASan + UBSan) make release # optimized build make test # run full test suite ./rayforce # start the Rayfall REPL ``` Rayforce ships with **Rayfall** — a Lisp-like query language with a rich set of builtins. The REPL prompt is `‣` : ``` ‣ (set t (table [Symbol Side Qty] (list [AAPL GOOG MSFT AAPL GOOG] [Buy Sell Buy Sell Buy] [100 200 150 300 250]))) ‣ (select {from:t by: Symbol Qty: (sum Qty)}) +--------+----------------------------+ | Symbol | Qty | | sym | i64 | +--------+----------------------------+ | AAPL | 400 | | GOOG | 450 | | MSFT | 150 | +-------------------------------------+ | 3 rows (3 shown) 2 columns (2 shown)| +-------------------------------------+ ‣ (pivot t 'Symbol 'Side 'Qty sum) +--------+-----+----------------------+ | Symbol | Buy | Sell | | sym | i64 | i64 | +--------+-----+----------------------+ | AAPL | 100 | 300 | | GOOG | 250 | 200 | | MSFT | 150 | 0 | +-------------------------------------+ | 3 rows (3 shown) 3 columns (3 shown)| +-------------------------------------+ ``` Headers: [ include/rayforce.h](/RayforceDB/rayforce/blob/master/include/rayforce.h) (types, memory, atoms, vectors, tables, symbols), `src/ops/ops.h` (DAG construction, opcodes, optimizer, executor, graph algorithms), `src/mem/heap.h` (allocator lifecycle). ``` #include #include "mem/heap.h" #include "ops/ops.h" int main(void) { ray_heap_init(); ray_sym_init(); /* Build a table */ int64_t regions[] = {0, 0, 1, 1, 2, 2}; int64_t amounts[] = {100, 200, 150, 300, 175, 225}; ray_t* reg = ray_vec_from_raw(RAY_I64, regions, 6); ray_t* amt = ray_vec_from_raw(RAY_I64, amounts, 6); ray_t* tbl = ray_table_new(2); tbl = ray_table_add_col(tbl, ray_sym_intern("region", 6), reg); tbl = ray_table_add_col(tbl, ray_sym_intern("amount", 6), amt); ray_release(reg); ray_release(amt); /* Group by region, sum amounts */ ray_graph_t* g = ray_graph_new(tbl); ray_op_t* keys[] = { ray_scan(g, "region") }; uint16_t agg_ops[] = { OP_SUM }; ray_op_t* agg_ins[] = { ray_scan(g, "amount") }; ray_op_t* grp = ray_group(g, keys, 1, agg_ops, agg_ins, 1); ray_t* result = ray_execute(g, grp); if (result && !RAY_IS_ERR(result)) ray_release(result); ray_graph_free(g); ray_release(tbl); ray_sym_destroy(); ray_heap_destroy(); } ``` **Build** — Construct a lazy DAG: scans, filters, joins, aggregations, window functions, graph traversals. Nothing executes yet. **Optimize** — Multi-pass rewriting: type inference → constant folding → SIP → factorize → predicate pushdown → filter reorder → projection pushdown → partition pruning → fusion → DCE. **Execute** — Fused morsel-driven bytecode processes 1024-element chunks that stay L1-resident. Radix-partitioned hash joins size partitions to fit L2. Thread pool dispatches morsels in parallel. **Execution engine** - Lazy operation DAG — nothing runs until `ray_execute` - Multi-pass optimizer with sideways information passing - Fused morsel-driven bytecode — element-wise ops merged into single-pass chunks - Radix-partitioned hash joins sized for L2 cache - Thread pool with parallel morsel dispatch **Graph engine** - Double-indexed CSR storage (forward + reverse), mmap support - BFS, DFS, Dijkstra, A*, PageRank, Louvain, Betweenness, LFTJ, and more - Factorized execution avoids materializing cross-products - SIP propagates selection bitmaps backward through expand chains **Rayfall language** - Arithmetic, string, aggregation, joins, higher-order, I/O builtins - Lambdas compile lazily to bytecode, run in computed-goto VM `select` /`update` /`pivot` bridge to the DAG optimizer at runtime **Memory** - Buddy allocator with slab cache — O(1) for ~90% of allocations - Thread-local arenas, lock-free allocation, COW ref counting - No system allocator — `ray_alloc` /`ray_free` for everything **Vector search** - Multi-metric HNSW index (cosine / L2 / inner-product) with save/load - Rayfall builtins: `cos-dist` /`l2-dist` /`inner-prod` /`norm` /`knn` and the HNSW lifecycle`hnsw-build` /`ann` /`hnsw-save` /`hnsw-load` /`hnsw-free` /`hnsw-info` - Filter-aware ANN via `select ... where ... nearest (ann handle query) take k` - Iterative streaming scan: the `where` predicate is pushed into HNSW's beam loop so rejected candidates don't consume result slots **Storage** - Columnar files with mmap, splayed tables, date-partitioned tables - CSV reader with parallel mmap parse, type inference, null handling ``` include/rayforce.h Single public header src/mem/ Buddy allocator, slab cache, arena, COW src/core/ Type system, platform abstraction, runtime src/vec/ Vector, list, string, selection bitmap ops src/table/ Table, symbol intern table src/store/ Column files, CSR, splayed/parted tables, HNSW src/ops/ DAG, optimizer, fused executor, LFTJ src/io/ CSV reader/writer (parallel mmap) src/lang/ Rayfall parser, evaluator, bytecode VM src/app/ REPL, terminal, pretty-printer test/ Test suites examples/rfl/ Rayfall example scripts examples/ C API examples website/ Documentation site (GitHub Pages) ``` Full docs: [rayforcedb.github.io/rayforce](https://rayforcedb.github.io/rayforce/) [Quick Start](https://rayforcedb.github.io/rayforce/docs/quick-start.html)— build, REPL, first query[Rayfall Language](https://rayforcedb.github.io/rayforce/docs/rayfall-syntax.html)— syntax and builtins[Data Types](https://rayforcedb.github.io/rayforce/docs/data-types.html)— types and collections[Queries](https://rayforcedb.github.io/rayforce/docs/queries-select.html)— select, joins, pivot, window[C API](https://rayforcedb.github.io/rayforce/docs/c-api-core.html)— full API reference[Graph Engine](https://rayforcedb.github.io/rayforce/docs/graph-algorithms.html)— algorithms[Architecture](https://rayforcedb.github.io/rayforce/docs/architecture-pipeline.html)— DAG, optimizer, memory Rayforce has Python bindings at ** rayforce-py** — contributions welcome. Contributions are welcome. You can help by: - Reporting bugs and requesting features via [GitHub Issues](https://github.com/RayforceDB/rayforce/issues) - Submitting pull requests - Creating example scripts and use cases - Improving documentation Rayforce is jointly developed with and sponsored by ** Lynx**. This partnership has been instrumental in making Rayforce a mature, production-ready engine. Lynx's active involvement in development and their commitment to innovative open-source technologies in the financial sector has enabled Rayforce to reach its full potential.