Make LLM-powered apps Smarter and Cheaper, Reliable, Auditable changing 1 LOC Consortium, an open-source framework for running reliable and auditable LLM workflows, has launched in v0.1. It enables ensemble and fusion methods like majority vote and LLM-as-a-judge through an OpenAI-compatible API, targeting high-consequence applications in legal, healthcare, and finance. The project aims to reduce costs and improve reliability by combining multiple models while maintaining existing application interfaces. Consortium runs LLM workflows in a reliable, durable, and measured way. It is built for applications that need more than a single model call: durable DAG execution, nested workflows, retries, replay data, audit trails, budgets, cost/token accounting, benchmark tooling, and operator visibility. Ensemble and fusion methods are first-class in the architecture. Consortium ships with multiple reasoning workflows that call several models, compare or combine their outputs, and expose the result through an OpenAI-compatible /v1 API. Existing applications can keep their OpenAI SDK/client shape while Consortium handles the workflow complexity behind a model route. Status: v0.1. The core workflow runtime is usable, but this is still early software. Read the security notes before exposing it outside a trusted environment. | Ensemble UI | Workflow Builder | |---|---| | Job Timeline | Workflow Trace | |---|---| | Workflow Catalog | OpenAI-Compatible API Routes | |---|---| Reliability for high-consequence decisions. Legal, healthcare, compliance, finance, and other costly workflows should not depend blindly on one non-deterministic model response. Ensembles can add redundancy, independent checks, visible disagreement, and stronger audit trails. They do not replace domain review, validation, or regulatory controls. Economies of ensemble. Fusing several cheaper, diverse models can be competitive with a stronger single model on some tasks while lowering model spend. The tradeoff is usually latency and orchestration complexity. Useful theory and empirical precedent. Condorcet's Jury Theorem motivates independent voters; LLM-as-a-judge, scoring, peer review, and self-consistency methods show practical ways to use multiple model outputs. Consortium makes these patterns runnable and measurable, but you should benchmark them on your own workload. Different tasks fit different reasoning primitives. Majority vote fits extractable answers such as MCQA, yes/no, classification, and routing. Self-consistency is useful for math, code, and reasoning paths from one strong model. Judge and scoring workflows fit open-ended answers where reasoning quality matters. Synthesis fits reports and summaries that should merge complementary perspectives. Adversarial defense and camp-debate workflows fit ambiguous cases where disagreement itself is useful signal. Keep your application unchanged. Point an OpenAI-compatible client at Consortium and route a model name to a direct model or a workflow. Use ensemble topologies out of the box. Shipped workflows include majority vote, self-consistency, LLM judge, rubric scoring, peer-matrix review, synthesis, adversarial defense, and camp-based debate. Run workflows durably. Jobs execute as persisted DAGs with nested child workflows, retries, histories, replay data, WebSocket progress, budgets, limits, and cost/token accounting. Measure what works. Benchmark harnesses, admin analysis views, and workflow attribution make cost, latency, failure modes, and accuracy visible. Optimize semi-autonomously. Experimental benchloop tooling and GEPA-style prompt mutation can iterate on workflow prompts against benchmarks. Operate visually when useful. The React workflow builder and admin UI let operators inspect, fork, run, and tune workflows without editing JSON by hand. - Visual workflow builder and preset workflows. - Durable job execution with nested child workflows, retries, replay data, event history, and WebSocket progress. - OpenRouter-backed model calls with cost/token accounting. - OpenAI-compatible Chat Completions and Responses endpoints under /v1 . - Local admin/operator UI for jobs, workflows, benchmarks, optimization, and API keys. - Optional Novomo agent-runtime nodes agent run , novo run / Superagent . - Experimental benchloop benchmark-tuning workflow. The main user-facing methods are L1 reasoning- workflows. L0 aggregation- workflows are reusable internals, L2 composite- workflows combine primitives, and L3 benchmark- workflows wrap primitives for evaluation. Most primitives also ship with -cheap variants. | Primitive | Best for | Tradeoff | |---|---|---| reasoning-informed-captain-synthesis | Unified answers, reports, and summaries that should merge complementary model outputs | Generates new text, so it is not a pure winner selection | reasoning-majority-pick | MCQA, yes/no, classification, routing, and other extractable answers | Zero-cost aggregation on clear majorities; weak fit for open-ended answers | reasoning-self-consistency-majority-pick | Checking whether one strong model reaches the same answer through varied samples | Uses sampling diversity, not model-architecture diversity | reasoning-judge-pick | Fast open-ended winner selection across multiple model answers | Single evaluator is cheap but can be a point of failure | reasoning-judge-score-pick | Rubric-based evaluation where per-response scores matter | More calls than a judge; dynamic rubrics add one setup call | reasoning-peer-score-pick | High-scrutiny evaluation with multiple evaluator perspectives | Strongest evaluator diversity, highest cost/latency | reasoning-camp-split-judge-pick | Discrete answers where disagreement camps and minority reasoning matter | No live back-and-forth; camps are judged in one pass | reasoning-adversarial-defense-judge-pick | Ambiguous or high-stakes questions where answers should survive challenge | Adds a defense/challenge round before judging | reasoning-multi-round-majority-pick | Deliberation where agents can revise after seeing peer reasoning | Higher latency than single-round majority vote | See docs/reasoning-architecture.md /AlhasanIQ/consortium/blob/master/docs/reasoning-architecture.md for call counts, layer boundaries, and tuning invariants. - Go 1.25+ - Bun 1.3.7 for frontend builds .bun-version - Make - POSIX shell tools for local dev scripts sh , bash , lsof , curl , tail , kill - OpenRouter API key for real LLM calls Windows users should use WSL for the full local development workflow. Release binaries should run natively once published for the target platform. Fresh machine setup examples: macOS brew install go bun make Debian/Ubuntu/WSL sudo apt-get update sudo apt-get install -y git curl unzip make bash lsof ca-certificates Install Go 1.25+ from https://go.dev/doc/install curl -fsSL https://bun.sh/install | bash -s "bun-v1.3.7" export PATH="$HOME/.bun/bin:$PATH" git clone https://github.com/AlhasanIQ/consortium.git cd consortium cp .env.example .env Edit .env and set OPENROUTER API KEY=