{"slug": "mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node", "title": "Mohdel 1.0: a self-hosted LLM gateway and SDK for Node", "summary": "A developer released Mohdel 1.0.0, an MIT-licensed self-hosted LLM gateway and SDK for Node that unifies access to Anthropic, OpenAI, Gemini, Mistral, Groq, xAI, Cerebras, Fireworks, DeepSeek, Qwen Cloud, Xiaomi, OpenRouter, Novita, and local inference through a single call. The project, which runs the inference layer of a document analysis product with hundreds of thousands of users, offers in-process and cross-process deployment paths, a user-owned cost catalog, and documented NDJSON wire protocol with reference clients in Lua, Gleam, Rust, and OCaml.", "body_md": "I released [mohdel](https://github.com/clbrge/mohdel) 1.0.0 this week. It is a self-hosted LLM gateway and SDK for Node, MIT licensed, and it runs the inference layer of a document analysis product with hundreds of thousands of users.\n\nThe closest thing to it is LiteLLM, which lives in Python. If your stack is JavaScript, your options have been a Python sidecar to deploy and monitor, a SaaS router sitting in your request path, or N provider SDKs with N shapes and no accounting. Mohdel is the Node-native version of that layer.\n\nHere is what it does, section by section.\n\nAnthropic, OpenAI, Gemini, Mistral, Groq, xAI, Cerebras, Fireworks, DeepSeek, Qwen Cloud, Xiaomi, OpenRouter, Novita, plus local inference. One call:\n\n``` python\nimport mohdel from 'mohdel'\n\nconst mo = await mohdel()\nconst result = await mo.use('anthropic/claude-sonnet-4-6').answer('Hello')\n\nconsole.log(result.output, result.cost)\n```\n\nSwitching provider is one string. The result shape does not move:\n\n```\n{\n  status: 'completed',      // 'completed' | 'tool_use' | 'incomplete'\n  output: 'Generated text',\n  inputTokens: 42,\n  outputTokens: 128,        // visible output, excludes thinking\n  thinkingTokens: 0,\n  cost: 0.002046,           // USD, from your catalog\n  timestamps: { start, first, end }\n}\n```\n\nTool calls, streaming, vision and speech to text go through the same call. Adapter differences stay inside mohdel: `max_tokens` versus `max_output_tokens`, top-level `system` versus `instructions`, five different usage shapes.\n\nLocal models are the `local/` provider, pointed at any OpenAI-compatible server: Ollama, vLLM, llama.cpp server, LM Studio. There is no default endpoint and no per-call override, so a call to a local model can never fall through to a cloud provider by accident.\n\n**In-process.** `import mohdel from 'mohdel'` and call it. No subprocess, nothing to run. Right for scripts, CLI tools, tests and single-process services, which is most projects.\n\n**Cross-process.** A Rust supervisor, `thin-gate`, owns a pool of session subprocesses. Your app talks to it over a unix socket; the provider SDKs run somewhere else entirely. An adapter that hangs, leaks or panics takes down one session, and the supervisor respawns it while the caller gets a recoverable error.\n\nThe wire between them is documented NDJSON, so a caller does not have to be JavaScript. There are reference clients in Lua, Gleam, Rust and OCaml in the repo, all validated against the same conformance fixtures.\n\nSwitching between the two paths is configuration, not code.\n\n`result.cost` is a USD number, computed from a catalog that belongs to you:\n\n```\n{\n  \"anthropic/claude-sonnet-4-6\": {\n    \"model\": \"claude-sonnet-4-6\",\n    \"creator\": \"anthropic\",\n    \"provider\": \"anthropic\",\n    \"inputPrice\": 3,\n    \"outputPrice\": 15,\n    \"cacheWritePrice\": 3.75,\n    \"cacheReadPrice\": 0.3,\n    \"contextTokenLimit\": 1000000,\n    \"outputTokenLimit\": 128000,\n    \"tags\": [\"chat\", \"tool-loop\", \"vision\"]\n  }\n}\n```\n\nYours, so it carries the rate you negotiated and the tags your routing code selects on, rather than a central price map you inherit and hope is current.\n\nThe catch is that no provider publishes prices in machine-readable form: no endpoint, just a marketing page with a table and a footnote about cached reads. So mohdel writes a brief and hands the transcription to the coding agent already running in your terminal:\n\n```\nmo model instructions anthropic > mohdel-brief.md\nclaude \"read mohdel-brief.md, then add claude-haiku-5 to my mohdel catalog\"\n\nmo model check --entry mohdel-candidate.json   # schema, types, required fields\nmo model apply mohdel-candidate.json           # prints the diff, waits for you\n```\n\nThe agent never writes the catalog. It writes a candidate file, `check` validates it, and `apply` prints a full diff before anything lands. Entries record the URL the numbers came from and the date they were read, and the brief tells the agent to leave a field out rather than guess it.\n\nOpenRouter needs none of this: it publishes per-token prices in its own model list, so setup offers to add every free model in one keystroke.\n\nSet `OTEL_EXPORTER_OTLP_ENDPOINT` and you get spans, trace-linked logs and OTLP metrics. The call span follows the GenAI semantic conventions (`gen_ai.request.model`, `gen_ai.usage.input_tokens`) and adds mohdel's own `mohdel.cost` and `mohdel.time_to_first_token_ms`. Pass a `traceparent` on the call and it parents under your span.\n\nRunning the gate, you also get sessions alive and respawned, calls by provider and status, a call-duration histogram, and cooldown, quota and policy rejections. Per-model and per-provider rate limits live in the catalog, and a provider that keeps failing goes into cooldown so calls fast-fail instead of hitting the wire.\n\nThe source is JavaScript with JSDoc. Declarations are generated from it and ship with the package, so a TypeScript consumer typechecks against the real contract: `CallEnvelope`, `Event`, `AnswerResult` and a single `MohdelError`. No `@types` package, nothing to keep in sync.\n\nThe NDJSON protocol between client, gate and session has been frozen since 0.90 and is enforced by fixtures that both the JS and the Rust side round-trip. 1.0.0 puts that under SemVer along with the library API.\n\nSelf-hosted means the keys stay in your infrastructure and calls go straight to the provider. Nothing routes through a third party and nothing marks up your tokens.\n\nThe gateway itself is built to hold nothing worth taking: no network listener (the supervisor binds unix sockets at mode 0600), no credential store (the key rides on each call and goes to the SDK client), and nothing that executes, meaning no eval, no child_process, no automatic tool loop. The session subprocess starts from an empty environment and gets back only the variables the runtime reads, so a key for a provider it will never call is not in its address space.\n\nTool execution stays in your process, where you can see it.\n\nNot an orchestrator: no chains, no agents, no memory, no retrieval. Wrap it with whatever you like. Not a retry or fallback engine; errors are classified with `retryable`, `severity` and `type`, and the caller decides. Not a response cache. Not a token budgeter.\n\nIt does not expose an OpenAI-compatible endpoint, so a cross-process caller uses the JS client or implements the documented NDJSON protocol. The library and CLI run anywhere Node 22 does; the prebuilt supervisor binary is Linux x64 only so far.\n\n```\nnpm install -g mohdel\nmo                           # pick a provider, paste a key\nmo ask openai/gpt-5.6-luna \"why is the sky blue\"\n```\n\n`npm install mohdel` for the library. MIT licensed, and the issues are open: [https://github.com/clbrge/mohdel](https://github.com/clbrge/mohdel)", "url": "https://wpnews.pro/news/mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node", "canonical_source": "https://dev.to/clbrge/mohdel-10-a-self-hosted-llm-gateway-and-sdk-for-node-3n03", "published_at": "2026-09-11 18:07:59+00:00", "updated_at": "2026-09-11 18:43:39.835879+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools", "large-language-models", "ai-tools", "mlops"], "entities": ["Mohdel", "Node.js", "LiteLLM", "Anthropic", "OpenAI", "Gemini", "Ollama", "OpenRouter"], "alternates": {"html": "https://wpnews.pro/news/mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node", "markdown": "https://wpnews.pro/news/mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node.md", "text": "https://wpnews.pro/news/mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node.txt", "jsonld": "https://wpnews.pro/news/mohdel-1-0-a-self-hosted-llm-gateway-and-sdk-for-node.jsonld"}}