{"slug": "daptin-plus-an-llm-router", "title": "Daptin Plus an LLM Router", "summary": "Daptin published a walkthrough for running a self-hosted LLM router on its platform, letting operators connect multiple LLM providers and expose chosen models to users through the same accounts and groups used in their applications. The guide covers configuring an OpenRouter credential and exposing GPT as a model named `assistant`, adding Gemini as a fallback, streaming, image input, and per-account plans with request limits, token allowances and spending budgets, all managed through Daptin's resource APIs. The walkthrough runs Daptin from the `daptin/daptin:latest` Docker image on port 6336 with a SQLite database, and notes Postgres is preferred for production.", "body_md": "## Repository navigation\n\n# Walkthrough LLM Router\n\n[1 revision](https://github.com/daptin/daptin/wiki/Walkthrough-LLM-Router/_history)\n\nDaptin handles users, permissions, data, and APIs. Now you can use it to run your own LLM router too. Connect different LLM providers, expose the models you want, and give your users access through the same accounts and groups they use in your application. Each account can have a plan with request limits, a token allowance, and a spending budget. Providers, models, routes, plans, and usage records are all available through Daptin's resource APIs, so you can inspect and change the setup as you go.\n\nWe'll build this up one piece at a time:\n\n1. \n**Get Daptin running** : Start a Docker container with persistent storage and check that the server is ready.\n2. \n**Set up authentication** : Create the first account, sign in, claim administrator access, and get the token we'll use for configuration.\n3. \n**Connect our first model** : Configure an OpenRouter credential, create a provider, and expose GPT through a model named`assistant` . This introduces the three resources behind routing: providers, models, and deployments.\n4. \n**Create a regular user** : Create an user account and a group, grant that group permission to invoke`assistant` , and check that guests remain denied. We'll also revoke access and try the request again.\n5. \n**Add a fallback** : Connect Gemini to the same public model name, then disable the primary deployment to see the fallback handle a request.\n6. \n**Try streaming** : Perform the same request with and without streaming so you can see how the responses differ.\n7. \n**Save a conversation** : Conversation continuation: retain a response and continue from its ID, with Daptin loading the earlier conversation for us.\n8. \n**Send an image to LLM** : Enable image input and make a request containing both text and an image URL.\n9. \n**Configure budget for user** : Create a plan, assign its membership to the account, configure model prices, and inspect the recorded usage. Also test exhausted a limit and check that Daptin rejects the next request.\n10. \n**Set operational limits** : Configure capacity limits and per-IP HTTP rate protection, and explain how these differ from a user's plan.\n\nWe will use curl in this walkthru (but all of it can be configured using the built in web dashboard as well) with actual responses and checks along the way. You can follow each configuration change top to bottom on your local daptin instance.\n\nResponse examples show the fields relevant for that step. Expand **Full response** to inspect the complete response. JWTs are replaced with placeholders throughout; copy your own token from your sign-in response.\n\nPull the daptin docker image\n\n```\ndocker pull daptin/daptin:latest\n```\n\nCreate a docker volume for persisting the sqlite db that we will use for the local setup (prefer postgres for a production setup)\n\n```\ndocker volume create daptin-router-data\n\ndocker run -d \\\n --name daptin-router \\\n --restart unless-stopped \\\n -p 127.0.0.1:6336:8080 \\\n -v daptin-router-data:/var/lib/daptin \\\n -e DAPTIN_DB_TYPE=sqlite3 \\\n -e DAPTIN_DB_CONNECTION_STRING=/var/lib/daptin/daptin.db \\\n -e DAPTIN_DISABLE_SMTP=true \\\n daptin/daptin:latest\n```\n\nDocker logs should give you something like this\n\n```\nINFO[2026-11-13 06:51:48] Our admin is [No one]\nINFO[2026-11-13 06:51:48] Listening at: [:8080]\n```\n\n## Full startup log excerpt\n\n```\n. (truncated for brevity)\n.\n.\nINFO[2026-11-13 06:51:48] [/live/contact/:referenceId/content/yjs] YJS websocket endpoint for contact[content]\nINFO[2026-11-13 06:51:48] Listening websocket server at ... /live\nWARN[2026-11-13 06:51:48] Failed to execute query 789: SELECT `u`.` email` FROM `user_account_user_account_id_has_usergroup_usergroup_id` AS `uu` LEFT JOIN `user_account` AS `u` ON (`uu`.` user_account_id` = `u`.` id`) WHERE (` uu`.` usergroup_id` = ?) ORDER BY `uu`.` created_at` ASC LIMIT ? == [2 1]\nWARN[2026-11-13 06:51:48] Failed to scan user group id from the result 830: sql: no rows in result set\nINFO[2026-11-13 06:51:48] Our admin is [No one]\nINFO[2026-11-13 06:51:48] [ProcessId=1] Listening at port: :8080\nINFO[2026-11-13 06:51:48] [198] No config value set for [enable_https]: sql: no rows in result set\nINFO[2026-11-13 06:51:48] [198] No config value set for [http.read_header_timeout_seconds]: sql: no rows in result set\nINFO[2026-11-13 06:51:48] [198] No config value set for [http.idle_timeout_seconds]: sql: no rows in result set\nINFO[2026-11-13 06:51:48] HTTPS is disabled\nINFO[2026-11-13 06:51:48] Listening at: [:8080]\n```\n\nYou can call the /ready endpoint also for checking over api if the server is up and ready to handle requests.\n\n```\ncurl http://localhost:6336/ready\n```\n\nReady\n\nSignup and setup the administrator account first:\n\n```\ncurl http://localhost:6336/action/user_account/signup \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"attributes\": {\n \"name\": \"Administrator\",\n \"email\": \"admin@example.com\",\n \"password\": \"YOUR_PASSWORD\",\n \"passwordConfirm\": \"YOUR_PASSWORD\"\n }\n }'\n```\n\nThe response to signup will be something like this\n\n```\n[\n {\n \"ResponseType\": \"client.notify\",\n \"Attributes\": {\n \"message\": \"Sign-up successful. Redirecting to sign in\",\n \"type\": \"success\"\n }\n }\n]\n```\n\n## Full response\n\n```\n[\n {\n \"ResponseType\": \"client.notify\",\n \"Attributes\": {\n \"__type\": \"client.notify\",\n \"message\": \"Sign-up successful. Redirecting to sign in\",\n \"title\": \"Success\",\n \"type\": \"success\"\n }\n },\n {\n \"ResponseType\": \"client.redirect\",\n \"Attributes\": {\n \"__type\": \"client.redirect\",\n \"delay\": 2000,\n \"location\": \"/auth/signin\",\n \"window\": \"self\"\n }\n }\n]\ncurl http://localhost:6336/action/user_account/signin \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"attributes\": {\n \"email\": \"admin@example.com\",\n \"password\": \"YOUR_PASSWORD\"\n }\n }'\n[\n {\n \"ResponseType\": \"client.store.set\",\n \"Attributes\": {\n \"key\": \"token\",\n \"value\": \"YOUR_ADMIN_JWT\"\n }\n }\n]\n```\n\n## Full response\n\n```\n[\n {\n \"ResponseType\": \"client.store.set\",\n \"Attributes\": {\n \"key\": \"token\",\n \"value\": \"YOUR_ADMIN_JWT\"\n }\n },\n {\n \"ResponseType\": \"client.cookie.set\",\n \"Attributes\": {\n \"key\": \"token\",\n \"value\": \"YOUR_ADMIN_JWT; SameSite=Strict\"\n }\n },\n {\n \"ResponseType\": \"client.notify\",\n \"Attributes\": {\n \"message\": \"Logged in\",\n \"title\": \"Success\",\n \"type\": \"success\"\n }\n },\n {\n \"ResponseType\": \"client.redirect\",\n \"Attributes\": {\n \"delay\": 2000,\n \"location\": \"/\",\n \"window\": \"self\"\n }\n }\n]\n```\n\nIn the response, find the `client.store.set` item whose `Attributes.key` is `token`. Copy its `Attributes.value`.\n\nUse that value wherever the following commands say `ADMIN_TOKEN`.\n\n```\nADMIN_TOKEN=YOUR_ADMIN_JWT\n```\n\nOnly the first user can claim admin, and only 1 user can claim admin, After 1 administrator is claimed more administrators can be added via the dashboard directly.\n\n```\ncurl http://localhost:6336/action/world/become_an_administrator \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{}'\n```\n\nA successful response will look like this\n\n```\n[\n {\n \"ResponseType\": \"Restart\",\n \"Attributes\": null\n }\n]\n```\n\n## Full response\n\n```\n[\n {\n \"ResponseType\": \"client.redirect\",\n \"Attributes\": {\n \"delay\": 7000,\n \"location\": \"/\",\n \"window\": \"self\"\n }\n },\n {\n \"ResponseType\": \"Restart\",\n \"Attributes\": null\n }\n]\n```\n\nA failed response will be as follows (if admin already exists)\n\n```\n[\n {\n \"ResponseType\": \"client.notify\",\n \"Attributes\": {\n \"message\": \"Unauthorized\",\n \"type\": \"error\"\n }\n }\n]\n```\n\n## Full response\n\n```\n[\n {\n \"ResponseType\": \"client.notify\",\n \"Attributes\": {\n \"message\": \"Unauthorized\",\n \"title\": \"failed\",\n \"type\": \"error\"\n }\n }\n]\n```\n\nWe will use OpenRouter for this guide. The router configuration uses four Daptin resources:\n\n| Resource | What it represents | Example in this guide | \n|---|---|---|\n| `credential` | The API key used to authenticate with the upstream provider | `openrouter-key` | \n| `llm_provider` | A provider account and the credential it uses | `openrouter` | \n| `llm_model` | The name clients request, its allowed operations, and its access policy | `assistant` | \n| `llm_deployment` | A route from that public model to a specific upstream model and provider account | `assistant-primary` →`openai/gpt-4o-mini` | \n\nA model can have several deployments. Clients keep requesting `assistant`, while\nDaptin selects a route from the deployments attached to it.\n\n`llm_model` defines **what your user requests**. `llm_deployment` defines **where Daptin sends that request**.\n\n| | `llm_model` | `llm_deployment` | \n|---|---|---|\n| Purpose | Public model and its access policy | Concrete upstream route | \n| Name example | `assistant` | `assistant-primary` | \n| Model name | Name clients use: `assistant` | Provider's name: `openai/gpt-4o-mini` | \n| Configuration | Allowed operations, capabilities, defaults, execute permissions | Provider account, priority, weight, timeouts, concurrency, pricing | \n| Relationships | Can have multiple deployments | Links one public model to one provider account | \n\nThe setup we will build looks like this. We will add the primary route in this section and the fallback route in section 5:\n\n```\nApplication requests: \"assistant\" ← llm_model\n │\n ├── assistant-primary\n │ OpenRouter → openai/gpt-4o-mini\n │ Priority 0\n │\n └── assistant-fallback\n OpenRouter → google/gemini-2.5-flash\n Priority 1\n ↑\n llm_deployment rows\n```\n\nYour application keeps sending `\"model\": \"assistant\"`. You can change upstream models, replace provider credentials, or add fallback routes by changing deployments.\n\nPermissions belong to the public model: **who may invoke `assistant`**. Provider and Deployment configuration controls execution: **which upstream handles it and under what operational limits**.\n\n```\ncurl http://localhost:6336/api/credential \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"credential\",\n \"attributes\": {\n \"name\": \"openrouter-key\",\n \"content\": \"{\\\"api_key\\\":\\\"sk-or-v1-YOUR_OPENROUTER_KEY\\\"}\"\n }\n }\n }'\n```\n\nThe escaped quotes in `content` are required because this field contains a JSON string.\n\nThe response to a successful creation will be like this\n\n```\n{\n \"data\": {\n \"type\": \"credential\",\n \"id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"attributes\": {\n \"name\": \"openrouter-key\",\n \"content\": \"ENCRYPTED_VALUE\"\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"credential\",\n \"id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"attributes\": {\n \"__type\": \"credential\",\n \"content\": \"kJdzKDH16Brnfp1mPZU5DQqK9ZCQqNjYvPHt063axp9JEx5C5ukZP7XFc-7SrCjFvM_0P1cSXXGxHlooXWnwRUPr_NxoN-yyokM-6bE23F9VdYbyn2665cRnt39dyvvnhdVDdhTLJg==\",\n \"created_at\": \"2026-11-13T07:12:01.909312051Z\",\n \"name\": \"openrouter-key\",\n \"permission\": 2097151,\n \"reference_id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"updated_at\": \"2026-11-13T07:12:01.909313259Z\",\n \"user_account_id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\",\n \"version\": 1\n },\n \"relationships\": {\n \"llm_provider_id\": {\n \"data\": null\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nNote that since the \"content\" field of the credential entity is marked as \"encrypted\" column type, its value will never be returned or CRUD api response. The base64 value above is the encrypted payload. You can use encrypted columns in your own custom entities as well in the same way.\n\nCopy `data.id` from the response. That is your `$CREDENTIAL_ID`.\n\n```\nCREDENTIAL_ID=01a1d3bf-c173-79bb-ab9c-3583d7cae432\ncurl http://localhost:6336/api/llm_provider \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_provider\",\n \"attributes\": {\n \"name\": \"openrouter\",\n \"provider_type\": \"openrouter\",\n \"provider_parameters\": {},\n \"credential_id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"enable\": true\n }\n }\n }'\n```\n\nThe reponse will be something like this\n\n```\n{\n \"data\": {\n \"type\": \"llm_provider\",\n \"id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"attributes\": {\n \"name\": \"openrouter\",\n \"provider_type\": \"openrouter\",\n \"credential_id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"provider_parameters\": {},\n \"enable\": 1\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"llm_provider\",\n \"id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"attributes\": {\n \"__type\": \"llm_provider\",\n \"allow_insecure\": 0,\n \"allow_private_network\": 0,\n \"base_url\": null,\n \"created_at\": \"2026-11-13T07:18:19.497550961Z\",\n \"credential_id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\",\n \"enable\": 1,\n \"name\": \"openrouter\",\n \"permission\": 2097151,\n \"provider_parameters\": {},\n \"provider_type\": \"openrouter\",\n \"reference_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"updated_at\": \"2026-11-13T07:18:19.497551503Z\",\n \"user_account_id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\",\n \"version\": 1\n },\n \"relationships\": {\n \"credential_id\": {\n \"data\": {\n \"type\": \"credential\",\n \"id\": \"01a1d3bf-c173-79bb-ab9c-3583d7cae432\"\n }\n },\n \"llm_deployment_id\": {\n \"data\": []\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nCopy `data.id`. That is your `PROVIDER_ID`.\n\n```\ncurl http://localhost:6336/api/llm_model \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_model\",\n \"attributes\": {\n \"name\": \"assistant\",\n \"operations\": \"[\\\"chat\\\",\\\"responses\\\"]\",\n \"routing_strategy\": \"priority_weighted\",\n \"enable\": true\n }\n }\n }'\n```\n\nWhich gives us the response\n\n```\n{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\n \"name\": \"assistant\",\n \"operations\": [\n \"chat\",\n \"responses\"\n ],\n \"permission\": 2097151\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\n \"__type\": \"llm_model\",\n \"capabilities\": {},\n \"created_at\": \"2026-11-13T07:22:23.107978838Z\",\n \"default_parameters\": {},\n \"enable\": 1,\n \"fallback_models\": [],\n \"name\": \"assistant\",\n \"operations\": [\n \"chat\",\n \"responses\"\n ],\n \"permission\": 2097151,\n \"reference_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"routing_strategy\": \"priority_weighted\",\n \"unsupported_parameter_policy\": \"reject\",\n \"updated_at\": \"2026-11-13T07:22:23.107979755Z\",\n \"user_account_id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\",\n \"version\": 1\n },\n \"relationships\": {\n \"llm_deployment_id\": {\n \"data\": []\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nCopy `data.id`. That is your `MODEL_ID`.\n\nThe permission value you are seeing in the respoonse above includes GUEST_EXECUTE so anyone will be able to use the model, lets lock it down to admin only for now and we will reopen it to users/guests later\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_model/01a1d3c9-3c03-7ac3-a864-7b7cf5705b14 \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\"permission\": 0}\n }\n }'\ncurl http://localhost:6336/api/llm_deployment \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"attributes\": {\n \"name\": \"assistant-primary\",\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\", \n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"upstream_model\": \"openai/gpt-4o-mini\",\n \"operations\": \"[\\\"chat\\\",\\\"responses\\\"]\",\n \"priority\": 0,\n \"weight\": 1,\n \"enable\": true\n }\n }\n }'\n```\n\nA successful request will give the following response\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3ce-2c3c-7bfe-bb8e-6be92b322368\",\n \"attributes\": {\n \"name\": \"assistant-primary\",\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"upstream_model\": \"openai/gpt-4o-mini\",\n \"priority\": 0,\n \"weight\": 1\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3ce-2c3c-7bfe-bb8e-6be92b322368\",\n \"attributes\": {\n \"__type\": \"llm_deployment\",\n \"connect_timeout_ms\": 10000,\n \"created_at\": \"2026-11-13T07:27:46.749376377Z\",\n \"enable\": 1,\n \"health_check\": {},\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"max_concurrency\": -1,\n \"name\": \"assistant-primary\",\n \"operations\": [\n \"chat\",\n \"responses\"\n ],\n \"parameters\": {},\n \"permission\": 2097151,\n \"pricing\": {},\n \"priority\": 0,\n \"reference_id\": \"01a1d3ce-2c3c-7bfe-bb8e-6be92b322368\",\n \"request_timeout_ms\": 90000,\n \"rpm\": -1,\n \"tpm\": -1,\n \"updated_at\": \"2026-11-13T07:27:46.749376793Z\",\n \"upstream_model\": \"openai/gpt-4o-mini\",\n \"user_account_id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\",\n \"version\": 1,\n \"weight\": 1\n },\n \"relationships\": {\n \"llm_model_id\": {\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\"\n }\n },\n \"llm_provider_id\": {\n \"data\": {\n \"type\": \"llm_provider\",\n \"id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nWait a few seconds for the catalog to reload (or you can restart the container/process).\n\nAt this point we have done all the configuration and we can start calling the llm apis, we will start with the model list which should list our public facing names (the one we created in llm_deployment)\n\n```\ncurl http://localhost:6336/v1/models \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\nResponse:\n\n```\n{\n \"data\": [\n {\n \"id\": \"assistant\",\n \"object\": \"model\"\n }\n ],\n \"object\": \"list\"\n}\n```\n\n## Full response\n\n```\n{\n \"data\": [\n {\n \"created\": 0,\n \"id\": \"assistant\",\n \"llmgateway\": {\n \"capabilities\": {},\n \"operations\": [\n \"chat\",\n \"responses\"\n ]\n },\n \"object\": \"model\",\n \"owned_by\": \"llmgateway\"\n }\n ],\n \"object\": \"list\"\n}\ncurl http://localhost:6336/v1/chat/completions \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n \"max_tokens\": 128\n }'\n```\n\nThis will giv permission denied error since there is no auth token and we had set the permission for the llm_model to 0 earlier.\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H 'Content-Type: application/json' \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n \"max_tokens\": 128\n }'\n```\n\nResponse:\n\n```\n{\n \"model\": \"assistant\",\n \"choices\": [\n {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Hello! How can I assist you today?\"\n },\n \"finish_reason\": \"stop\"\n }\n ],\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 9,\n \"total_tokens\": 18\n }\n}\n```\n\n## Full response\n\n```\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! How can I assist you today?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1794555847,\n \"id\": \"gen-1794555847-wSj38rh3abVghxCJDUsD\",\n \"model\": \"assistant\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 9,\n \"total_tokens\": 18\n }\n}\n```\n\nWe have a very basic setup for the LLM router now. Admin can use the /v1/ apis to list models or invoke /completions Next will do some advanced stuff, including adding a fallback model, changing permission to allow users/guests and also budgeting/token limiting.\n\nThese execution grants have different meanings:\n\n| Permission | Meaning | \n|---|---|\n| Model row `0` | No row-level execution grant; administrators and separately granted groups may still execute | \n| Model row `32` | Allows guest invocation | \n| Model row `4096` | Allows the record's owner—not every signed-in user | \n| Model row `2097152` | Allows any authenticated account | \n| Model's group relationship `524288` | `GroupExecute` : grants execution to members of that group | \n\nPermission values are bitsets. PATCHing a permission value replaces that record's current bitset. Apply model permissions through a separate PATCH after creation.\n\n**For the group-specific setup below, keep the model row's permission at `0`.**\n\nAs administrator, create an application account if you do not have one (or you can enable the signup action to let guests signup):\n\n```\ncurl http://localhost:6336/api/user_account \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"user_account\",\n \"attributes\": {\n \"name\": \"Router application\",\n \"email\": \"app@example.com\",\n \"password\": \"YOUR_USER_PASSWORD\"\n }\n }\n }'\n```\n\nResponse\n\n```\n{\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n}\n```\n\nAnd then we create a new usergroup\n\n```\ncurl http://localhost:6336/api/usergroup \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"usergroup\",\n \"attributes\": {\"name\": \"assistant-users\"}\n }\n }'\n```\n\nResponse\n\n```\n{\n \"data\": {\n \"type\": \"usergroup\",\n \"id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"\n }\n}\n```\n\nCopy the group response's `data.id` as `GROUP_ID`.\n\nAdd the newly created user account to this new usergroup\n\n```\ncurl -X PATCH http://localhost:6336/api/user_account/01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"relationships\": {\n \"usergroup_id\": {\n \"data\": [{\"type\": \"usergroup\", \"id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"}]\n }\n }\n }\n }'\n```\n\nExpect HTTP 200. This adds membership and preserves existing memberships.\n\nResponse\n\n```\n{\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n}\n```\n\nWe can verify the actual group IDs with a separate read:\n\n```\ncurl http://localhost:6336/api/user_account/01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc/relationships/usergroup_id \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\nResponse\n\n```\n{\n \"data\": [\n {\n \"type\": \"usergroup\",\n \"id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"\n },\n {\n \"type\": \"usergroup\",\n \"id\": \"01a1d45c-4675-7f29-985e-f7a678355c66\"\n }\n ]\n}\n```\n\n## Full response\n\n```\n{\n \"links\": {\n \"current_page\": 1,\n \"from\": 0,\n \"last_page\": 1,\n \"per_page\": 10,\n \"to\": 10,\n \"total\": 2\n },\n \"data\": [\n {\n \"type\": \"usergroup\",\n \"id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"\n },\n {\n \"type\": \"usergroup\",\n \"id\": \"01a1d45c-4675-7f29-985e-f7a678355c66\"\n }\n ]\n}\n```\n\nUser is part of the usergroup. We will do the same with the llm_model entity we created earlier. Add it to the same usergroup\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_model/01a1d3c9-3c03-7ac3-a864-7b7cf5705b14 \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"relationships\": {\n \"usergroup_id\": {\n \"data\": [{\"type\": \"usergroup\", \"id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"}]\n }\n }\n }\n }'\n```\n\nResponse\n\n```\n{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\"\n }\n}\n```\n\nThe new additions to the usergroup are created with the default permissions, we will update the group permission of the llm_model and allow GROUP_EXECUTE so that members of that group can execute llm calls on this model\n\n```\ncurl -X PATCH http://localhost:6336/api/usergroup/01a1d46c-2c35-7502-a00b-3244b16944a2/relationships/llm_model_id \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": [{\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\"permission\": 524288}\n }]\n }'\n```\n\nThis PATCH returns nothing in response body. This permission update replaces the relation's entire bitmask with `524288`.\nIt does not change the model row's own permission, which should remain `0`.\n\nVerify the permission persisted\n\n```\ncurl http://localhost:6336/api/llm_model/01a1d3c9-3c03-7ac3-a864-7b7cf5705b14/usergroup_id \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\nIn the response to this one of the usergroup will be the one we just modified\n\n```\n{\n \"data\": [\n {\n \"type\": \"usergroup\",\n \"id\": \"01a1d482-1632-77ae-b3d3-e8cdd85b8b9d\",\n \"attributes\": {\n \"name\": \"assistant-users\",\n \"permission\": 524288,\n \"relation_reference_id\": \"01a1d46c-2c35-7502-a00b-3244b16944a2\"\n }\n }\n ]\n}\n```\n\nNow we should be able to login as the new user and use the llm model through that account\n\nSign in as the application account and copy the token as before:\n\n```\ncurl http://localhost:6336/action/user_account/signin \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"attributes\": {\n \"email\": \"app@example.com\",\n \"password\": \"YOUR_USER_PASSWORD\"\n }\n }'\n```\n\nResponse\n\n```\n[\n {\n \"ResponseType\": \"client.store.set\",\n \"Attributes\": {\n \"key\": \"token\",\n \"value\": \"YOUR_APPLICATION_JWT\"\n }\n }\n]\n```\n\nCopy `Attributes.value` from the `client.store.set` item into a separate shell\nvariable for the application account:\n\n```\nUSER_DAPTIN_TOKEN='YOUR_APPLICATION_JWT'\n```\n\nUse `$USER_DAPTIN_TOKEN` for model calls from this point onward. Keep\n`$ADMIN_TOKEN` for configuration and inspecting administrative resources.\n\nTry a request as the application account:\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n \"max_tokens\": 128,\n \"stream\": false\n }'\n{\n \"model\": \"assistant\",\n \"choices\": [\n {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Hello! How can I assist you today?\"\n },\n \"finish_reason\": \"stop\"\n }\n ],\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 9,\n \"total_tokens\": 18\n }\n}\n```\n\n## Full response\n\n```\n{\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"logprobs\": null,\n \"message\": {\n \"content\": \"Hello! How can I assist you today?\",\n \"role\": \"assistant\"\n }\n }\n ],\n \"created\": 1794567109,\n \"id\": \"gen-1794567109-Mnjrxj9MqH12b4rkmM75\",\n \"model\": \"assistant\",\n \"object\": \"chat.completion\",\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 9,\n \"total_tokens\": 18\n }\n}\n```\n\nNow all users that belong to this new usergroup will be able to execute the /v1 llm apis for this model. We can set back the permission to restrict it again\n\nSet the existing model–group relation's permission to `0`:\n\n```\ncurl -X PATCH http://localhost:6336/api/usergroup/01a1d46c-2c35-7502-a00b-3244b16944a2/relationships/llm_model_id \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": [{\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\"permission\": 0}\n }]\n }'\n```\n\nTo continue the guide, restore the grant by repeating the group-side permission PATCH above with `\"permission\": 524288`.\nSetting the relation's permission to `0` leaves the link in place but removes its group execution grant.\nIt does not revoke owner, guest, authenticated, other group, or administrator access supplied independently.\n\n```\ncurl -X DELETE http://localhost:6336/api/user_account//relationships/usergroup_id \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": [{\"type\": \"usergroup\", \"id\": \"\"}]\n }'\n```\n\n(same structure works if you want to remove the model from the usergroup)\n\nUse the same `MODEL_ID` and `PROVIDER_ID`:\n\nBoth deployments use the same OpenRouter account. The fallback gives us a second upstream model, but both routes share that account's credit and OpenRouter's availability.\n\n```\ncurl http://localhost:6336/api/llm_deployment \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"attributes\": {\n \"name\": \"assistant-fallback\",\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"upstream_model\": \"google/gemini-2.5-flash\",\n \"operations\": \"[\\\"chat\\\",\\\"responses\\\"]\",\n \"priority\": 1,\n \"weight\": 1,\n \"enable\": true\n }\n }\n }'\n```\n\nWhich gives us the response of successful creation\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3e1-4229-7adb-b7b6-417803e3f465\",\n \"attributes\": {\n \"name\": \"assistant-fallback\",\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"upstream_model\": \"google/gemini-2.5-flash\",\n \"priority\": 1,\n \"weight\": 1\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3e1-4229-7adb-b7b6-417803e3f465\",\n \"attributes\": {\n \"__type\": \"llm_deployment\",\n \"connect_timeout_ms\": 10000,\n \"created_at\": \"2026-11-13T07:48:37.548225595Z\",\n \"enable\": 1,\n \"health_check\": {},\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\",\n \"max_concurrency\": -1,\n \"name\": \"assistant-fallback\",\n \"operations\": [\n \"chat\",\n \"responses\"\n ],\n \"parameters\": {},\n \"permission\": 2097151,\n \"pricing\": {},\n \"priority\": 1,\n \"reference_id\": \"01a1d3e1-4229-7adb-b7b6-417803e3f465\",\n \"request_timeout_ms\": 90000,\n \"rpm\": -1,\n \"tpm\": -1,\n \"updated_at\": \"2026-11-13T07:48:37.548226262Z\",\n \"upstream_model\": \"google/gemini-2.5-flash\",\n \"user_account_id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\",\n \"version\": 1,\n \"weight\": 1\n },\n \"relationships\": {\n \"llm_model_id\": {\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\"\n }\n },\n \"llm_provider_id\": {\n \"data\": {\n \"type\": \"llm_provider\",\n \"id\": \"01a1d3c5-8468-77e9-b962-25eb573dbd58\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d3b5-0c88-7a44-8571-1c8211790a0e\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nRequests still use `\"model\": \"assistant\"`. Daptin prefers priority `0` and can use priority `1` when the primary is unavailable or encounters a retryable failure.\n\nWith GPT at priority `0` and Gemini at priority `1`:\n\n| Situation | Result | \n|---|---|\n| GPT is eligible and succeeds | GPT answers; Gemini is not called | \n| GPT is disabled, unhealthy, or excluded by a deployment protection limit | Daptin can select Gemini | \n| GPT encounters a retryable upstream failure | Daptin can attempt Gemini within the remaining attempt/time budget | \n| Daptin rejects model permission, validation, or quota admission | The request is rejected; fallback does not bypass the decision | \n| Upstream returns a non-retryable error such as 400, 401, 403, or 404 | Daptin returns the error rather than automatically trying Gemini | \n| A stream has already emitted its first client-visible event | Daptin stays with that deployment; it cannot switch midway | \n\nRetryable upstream statuses include **408, 409, 429, and 5xx**. Connection failures and timeouts may also be retryable.\n\nGiving both deployments priority `0` would make them weighted alternatives. Priority `0` followed by `1` expresses primary and standby.\n\nDisable the primary deployment:\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_deployment/01a1d3ce-2c3c-7bfe-bb8e-6be92b322368 \\\n-H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3ce-2c3c-7bfe-bb8e-6be92b322368\",\n \"attributes\": {\n \"enable\": false\n }\n }\n }'\n```\n\nWait a few seconds for the catalog reload, then repeat your normal request:\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n \"max_tokens\": 128,\n \"stream\": false\n }'\n```\n\nGemini should now handle it. The response still reports the public model name `assistant`.\n\nInspect usage as administrator:\n\n```\ncurl -g 'http://localhost:6336/api/api_usage?page[size]=10&sort=-created_at' \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\nLook at the relevant row's `attributes.metadata.attempts`. Compare `DeploymentID`\nwith `FALLBACK_DEPLOYMENT_ID`, copied from the fallback creation response.\n\nRestore the primary by repeating the PATCH with `\"enable\": true`.\n\nFirst, send a normal request with the application token and `stream: false`:\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\"role\":\"user\",\"content\":\"Write a short greeting.\"}],\n \"max_tokens\": 128,\n \"stream\": false\n }'\n```\n\nExpect HTTP 200 and one JSON response. The completed answer is in\n`choices[0].message.content`.\n\nNow send the same prompt with `stream: true` and add `-N` to curl:\n\n```\ncurl -N http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\"role\":\"user\",\"content\":\"Write a short greeting.\"}],\n \"max_tokens\": 128,\n \"stream\": true\n }'\n```\n\n`-N` disables curl's output buffering, so text appears as it arrives. The response\nis a stream of server-sent events: each `data:` line carries a chunk, and text\narrives in `choices[].delta.content`.\n\nHere is a short example showing only the relevant fields. The actual text and chunk boundaries will vary:\n\n```\ndata: {\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"Hello!\"},\"finish_reason\":null}]}\n\ndata: {\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n```\n\nThe last line, `data: [DONE]`, marks the end of a successful Chat Completions\nstream.\n\n## Captured stream excerpt\n\nThe following events were captured from the beginning of a longer response:\n\n```\ndata: {\"choices\":[{\"delta\":{\"content\":\"Hello\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"!\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" I\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" hope\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" this\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" message\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\" finds\",\"role\":\"assistant\"},\"finish_reason\":null,\"index\":0}],\"created\":1794557487,\"id\":\"gen-1794557487-CQwEErjHpcRPWReYlgvZ\",\"model\":\"assistant\",\"object\":\"chat.completion.chunk\",\"usage\":null}\n```\n\nStreaming controls how the answer is delivered. It does not save a conversation. For Chat Completions, send previous messages yourself on subsequent requests. The next section uses Responses to save and load that history through Daptin.\n\nUse Responses when you want Daptin-managed history. Both requests below use the application token, so the saved turns belong to the application account:\n\n```\ncurl http://localhost:6336/v1/responses \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"input\": \"Remember that my favorite color is blue.\",\n \"store\": true,\n \"max_output_tokens\": 128\n }'\n{\n \"id\": \"01a1d4a0-b058-7848-a37a-83ce8c1ee112\",\n \"model\": \"assistant\",\n \"status\": \"completed\",\n \"store\": true,\n \"output\": [\n {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"output_text\",\n \"text\": \"Got it! Your favorite color is blue. If you have any questions or need assistance related to it, feel free to ask!\"\n }\n ]\n }\n ],\n \"usage\": {\n \"input_tokens\": 15,\n \"output_tokens\": 26,\n \"total_tokens\": 41\n }\n}\n```\n\n## Full response\n\n```\n{\n \"id\": \"01a1d4a0-b058-7848-a37a-83ce8c1ee112\",\n \"model\": \"assistant\",\n \"object\": \"response\",\n \"output\": [\n {\n \"content\": [\n {\n \"annotations\": [],\n \"logprobs\": [],\n \"text\": \"Got it! Your favorite color is blue. If you have any questions or need assistance related to it, feel free to ask!\",\n \"type\": \"output_text\"\n }\n ],\n \"id\": \"msg_tmp_ofyyg6iexe\",\n \"role\": \"assistant\",\n \"status\": \"completed\",\n \"type\": \"message\"\n }\n ],\n \"status\": \"completed\",\n \"store\": true,\n \"usage\": {\n \"input_tokens\": 15,\n \"output_tokens\": 26,\n \"total_tokens\": 41\n }\n}\n```\n\nCopy the top-level `id` returned by your request as `RESPONSE_ID`. Replace the\nplaceholder below with that ID, then continue using the same application token:\n\n```\ncurl http://localhost:6336/v1/responses \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"previous_response_id\": \"RESPONSE_ID\",\n \"input\": \"What is my favorite color?\",\n \"store\": true,\n \"max_output_tokens\": 128\n }'\n```\n\nUse the newest completed response ID for the next turn. Do not also resend the previous conversation.\n\n```\n{\n \"id\": \"01a1d4a1-143f-7ed6-a9af-04ddb7f80e5d\",\n \"model\": \"assistant\",\n \"status\": \"completed\",\n \"store\": true,\n \"previous_response_id\": \"01a1d4a0-b058-7848-a37a-83ce8c1ee112\",\n \"output\": [\n {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"output_text\",\n \"text\": \"Your favorite color is blue!\"\n }\n ]\n }\n ],\n \"usage\": {\n \"input_tokens\": 55,\n \"output_tokens\": 6,\n \"total_tokens\": 61\n }\n}\n```\n\n## Full response\n\n```\n{\n \"id\": \"01a1d4a1-143f-7ed6-a9af-04ddb7f80e5d\",\n \"model\": \"assistant\",\n \"object\": \"response\",\n \"output\": [\n {\n \"content\": [\n {\n \"annotations\": [],\n \"logprobs\": [],\n \"text\": \"Your favorite color is blue!\",\n \"type\": \"output_text\"\n }\n ],\n \"id\": \"msg_tmp_4vreiwjwkcb\",\n \"role\": \"assistant\",\n \"status\": \"completed\",\n \"type\": \"message\"\n }\n ],\n \"previous_response_id\": \"01a1d4a0-b058-7848-a37a-83ce8c1ee112\",\n \"status\": \"completed\",\n \"store\": true,\n \"usage\": {\n \"input_tokens\": 55,\n \"output_tokens\": 6,\n \"total_tokens\": 61\n }\n}\n```\n\nStored responses require a signed-in account (it does not store for guests). Each continuation independently checks access to the saved history and execute permission on the requested model.\nUse `store: true` on every turn whose ID you want to continue later; `store: false` does not produce a reusable stored turn.\nOnly completed retained responses can be continued.\nTop-level `instructions` and other request controls belong to the current turn, so repeat them when needed.\n\nThe initial `assistant` configuration enables text operations. Media requires additional configuration:\n\n| Feature | Operation | Additional model capability | \n|---|---|---|\n| Image input | `chat` or`responses` | `vision` | \n| File input | `responses` | `files` | \n| Inline audio input | `chat` | `audio` | \n| Speech-to-text | `audio_transcription` | Dedicated compatible upstream | \n| Text-to-speech | `audio_speech` | Dedicated compatible upstream | \n| Image generation | `image_generation` | Dedicated compatible upstream | \n\nDeclaring a capability permits a request; it does not add that ability to the upstream model. Check the primary and backup routes.\n\nAs administrator, enable image input on the public model:\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_model/01a1d3c9-3c03-7ac3-a864-7b7cf5705b14 \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"attributes\": {\n \"capabilities\": \"{\\\"vision\\\":true}\"\n }\n }\n }'\n```\n\nThis overwrites the capability field, so include any other capabilities you already use.\n\nBefore running the request, replace `PUBLIC_IMAGE_URL` with a public HTTPS URL\nto a JPEG or PNG image. Use a URL that returns the image itself and can be\naccessed without signing in. A local file path or a URL served only on your\nmachine will not work for the upstream provider.\n\nSend the request with the application token:\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Describe this image.\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"PUBLIC_IMAGE_URL\"\n }\n }\n ]\n }],\n \"max_tokens\": 256\n }'\n```\n\nExpect HTTP 200 and a completion describing your image. The wording will vary with the image and the upstream model. Both the primary and fallback routes need to support image input for this request to work through either route.\n\n(I will keep audio/video examples in a separate blog post)\n\nIf you have been following till this point you have a base llm router, with user accounts and usergroups to allow/deny access to the llm models which we route back to different llm providers based on the priority of our llm deployment entities. Notice how everything is basically a bunch of entities defined in the database and effectly making it declarative implementation. All we did was create some db rows and set values of their columns.\n\nNext we will enable metering and cost accounting at a user level. Metering enables recording of request/response payloads. LLM models are metered by default and that is why we were able to query the records earlier by calling /api/api_usage endpoint. But we have not created any api_plan to define how many tokens/seconds is the user allowed. Lets define a api_plan and link it to a user account by the api_member entity.\n\n| Control | Applies to | Configure with | Limit failure | \n|---|---|---|---|\n| Model permissions | Who may invoke `assistant` | Model row and model–group grants | 403 | \n| Account plan | How much the active account may consume across metered operations | `api_plan` and owned`api_member` | 402 for a hard limit | \n| Deployment protection | Capacity of one upstream route, shared by its callers | Deployment RPM, TPM, concurrency | Can select another eligible route; exhausted capacity can produce 429 or an availability error | \n| Global HTTP limiter | Requests from one client IP to one path | Backend `limit.rate` | 429 | \n\nWe create a plan with followinglimits: 60 metered requests per UTC minute, one million total tokens per UTC calendar month, and ten million configured cost micro-units per month. If you price in USD, the cost limit represents $10.\n\n```\ncurl http://localhost:6336/api/api_plan \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"api_plan\",\n \"attributes\": {\n \"name\": \"router-basic\",\n \"price_monthly_cents\": 0,\n \"limits\": [\n {\"metric\":\"requests\",\"window\":\"minute\",\"maximum\":60,\"mode\":\"hard\"},\n {\"metric\":\"total_tokens\",\"window\":\"month\",\"maximum\":1000000,\"mode\":\"hard\"},\n {\"metric\":\"cost_micros\",\"window\":\"month\",\"maximum\":10000000,\"mode\":\"hard\"}\n ],\n \"metadata\": {}\n }\n }\n }'\n```\n\nResponse:\n\n```\n{\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"attributes\": {\n \"name\": \"router-basic\",\n \"price_monthly_cents\": 0,\n \"limits\": [\n {\n \"maximum\": 60,\n \"metric\": \"requests\",\n \"mode\": \"hard\",\n \"window\": \"minute\"\n },\n {\n \"maximum\": 1000000,\n \"metric\": \"total_tokens\",\n \"mode\": \"hard\",\n \"window\": \"month\"\n },\n {\n \"maximum\": 10000000,\n \"metric\": \"cost_micros\",\n \"mode\": \"hard\",\n \"window\": \"month\"\n }\n ]\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"attributes\": {\n \"__type\": \"api_plan\",\n \"archived_at\": null,\n \"created_at\": \"2026-11-14T07:06:20.435512137Z\",\n \"limits\": [\n {\n \"maximum\": 60,\n \"metric\": \"requests\",\n \"mode\": \"hard\",\n \"window\": \"minute\"\n },\n {\n \"maximum\": 1000000,\n \"metric\": \"total_tokens\",\n \"mode\": \"hard\",\n \"window\": \"month\"\n },\n {\n \"maximum\": 10000000,\n \"metric\": \"cost_micros\",\n \"mode\": \"hard\",\n \"window\": \"month\"\n }\n ],\n \"metadata\": {},\n \"name\": \"router-basic\",\n \"permission\": 561441,\n \"price_monthly_cents\": 0,\n \"reference_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"updated_at\": \"2026-11-14T07:06:20.435517929Z\",\n \"user_account_id\": \"01a1d45c-8c11-7229-8520-93c53d474184\",\n \"version\": 1\n },\n \"relationships\": {\n \"api_member_id\": {\n \"data\": null\n },\n \"api_quota_id\": {\n \"data\": null\n },\n \"api_usage_id\": {\n \"data\": null\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d45c-8c11-7229-8520-93c53d474184\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nCopy `data.id` as `PLAN_ID`. Each metric/window pair must be unique. `month` is a UTC calendar month;\nfor a subscription's own billing cycle, use `member_period` and set valid `period_start` and `period_end` on its membership.\n\n`price_monthly_cents` is plan data, not an automatic charge. A zero price does not make upstream calls free.\nPaid checkout and verified payment provisioning need a billing workflow; see [Payments and Checkout](https://github.com/daptin/daptin/wiki/Payments-and-Checkout).\n\nNext we need a way to create the api_member entities which link a user_account (via the user_account_id column) to a api_plan. When you create any entity in daptin the default user_account_id is set to the user_account whos auth token is being used. An `api_member` created with the administrator token initially belongs to the administrator.\nCreate it as inactive, then PATCH its owner and activate it in the same update.\nThis avoids applying the new plan to the administrator before ownership is assigned. We will not provision a custom action here to keep things simple for the guide, but ideally you want the signup action to provision the api_member entity row for the user.\n\nCreate the membership as administrator:\n\n```\ncurl http://localhost:6336/api/api_member \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"api_member\",\n \"attributes\": {\n \"api_plan_id\": \"PLAN_ID\",\n \"status\": \"inactive\",\n \"metadata\": {}\n }\n }\n }'\n```\n\nCreates the following row\n\n```\n{\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"attributes\": {\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"user_account_id\": \"01a1d45c-8c11-7229-8520-93c53d474184\",\n \"status\": \"inactive\"\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"attributes\": {\n \"__type\": \"api_member\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"created_at\": \"2026-11-14T07:18:23.543939597Z\",\n \"metadata\": {},\n \"period_end\": null,\n \"period_start\": \"0001-01-01T00:00:00Z\",\n \"permission\": 561441,\n \"reference_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"status\": \"inactive\",\n \"updated_at\": \"2026-11-14T07:18:23.543941597Z\",\n \"user_account_id\": \"01a1d45c-8c11-7229-8520-93c53d474184\",\n \"version\": 1\n },\n \"relationships\": {\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"api_quota_id\": {\n \"data\": null\n },\n \"api_usage_id\": {\n \"data\": null\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d45c-8c11-7229-8520-93c53d474184\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nCopy `data.id` as `MEMBER_ID`. Use the application account's reference ID from\nsection 4 as `USER_ID`. Assign ownership and activate this new membership:\n\n```\ncurl -X PATCH http://localhost:6336/api/api_member/01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9 \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"attributes\": {\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"status\": \"active\"\n }\n }\n }'\n```\n\nExpect HTTP 200. Read back the membership:\n\n```\ncurl http://localhost:6336/api/api_member/MEMBER_ID \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n{\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"attributes\": {\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"status\": \"active\"\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"attributes\": {\n \"__type\": \"api_member\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"created_at\": \"2026-11-14T07:18:23.543939597Z\",\n \"metadata\": {},\n \"period_end\": null,\n \"period_start\": \"0001-01-01T00:00:00Z\",\n \"permission\": 561441,\n \"reference_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"status\": \"active\",\n \"updated_at\": \"2026-11-14T07:20:25.334852959Z\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"version\": 2\n },\n \"relationships\": {\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"api_quota_id\": {\n \"data\": null\n },\n \"api_usage_id\": {\n \"data\": null\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nThe plan includes a monthly cost budget. To enforce it, Daptin needs a price for the tokens used by each upstream model. Configure those prices on the deployments.\n\nFor this example, suppose a model costs:\n\n- $1 per million input tokens.\n- $2 per million output tokens.\n\nThese are example prices. Look up the prices for your upstream models before using this as a real spending budget.\n\nDaptin stores these rates in **micro-dollars per million tokens**, where\n1 dollar = 1,000,000 micro-dollars. The example prices therefore become:\n\n| Pricing field | Example price | Value to configure | \n|---|---|---|\n| `input_tokens` | $1 per million input tokens | `1000000` | \n| `output_tokens` | $2 per million output tokens | `2000000` | \n\nSet the primary deployment's prices:\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_deployment/01a1d3ce-2c3c-7bfe-bb8e-6be92b322368 \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d3ce-2c3c-7bfe-bb8e-6be92b322368\",\n \"attributes\": {\n \"pricing\": {\"input_tokens\":1000000,\"output_tokens\":2000000}\n }\n }\n }'\n```\n\nResponse:\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d45f-f05d-796a-ad14-78024e0c7102\",\n \"attributes\": {\n \"name\": \"assistant-primary\",\n \"pricing\": {\n \"input_tokens\": 1000000,\n \"output_tokens\": 2000000\n }\n }\n }\n}\n```\n\n## Full response\n\n```\n{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"01a1d45f-f05d-796a-ad14-78024e0c7102\",\n \"attributes\": {\n \"__type\": \"llm_deployment\",\n \"connect_timeout_ms\": 10000,\n \"created_at\": \"2026-11-13T10:06:59.677975507Z\",\n \"enable\": 1,\n \"health_check\": {},\n \"llm_model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"llm_provider_id\": \"01a1d45e-8115-70f1-b959-719c7c846cd5\",\n \"max_concurrency\": -1,\n \"name\": \"assistant-primary\",\n \"operations\": [\n \"chat\",\n \"responses\"\n ],\n \"parameters\": {},\n \"permission\": 561441,\n \"pricing\": {\n \"input_tokens\": 1000000,\n \"output_tokens\": 2000000\n },\n \"priority\": 0,\n \"reference_id\": \"01a1d45f-f05d-796a-ad14-78024e0c7102\",\n \"request_timeout_ms\": 90000,\n \"rpm\": -1,\n \"tpm\": -1,\n \"updated_at\": \"2026-11-13T11:11:15.239219125Z\",\n \"upstream_model\": \"openai/gpt-4o-mini\",\n \"user_account_id\": \"01a1d45c-8c11-7229-8520-93c53d474184\",\n \"version\": 3,\n \"weight\": 1\n },\n \"relationships\": {\n \"llm_model_id\": {\n \"data\": {\n \"type\": \"llm_model\",\n \"id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\"\n }\n },\n \"llm_provider_id\": {\n \"data\": {\n \"type\": \"llm_provider\",\n \"id\": \"01a1d45e-8115-70f1-b959-719c7c846cd5\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d45c-8c11-7229-8520-93c53d474184\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n}\n```\n\nRepeat this for all the LLM deployment you have created and linked for usage. Wait a few seconds for the deployment changes to load.\n\nWith the example prices, a request using 1,000 input tokens and 100 output tokens costs $0.0012.\nDaptin records that as `cost_micros: 1200` and counts it toward the plan's `10000000` micro-dollar ($10) monthly limit.\nDaptin calculates this amount using the rates you configure; it does not fetch OpenRouter prices or invoices.\nIf pricing is empty, token usage can still be counted, but the token cost can remain zero.\nSet pricing on both routes so that using the fallback also counts toward the cost budget.\n\nUse the user token so the request is charged against the user's plan:\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\"role\":\"user\",\"content\":\"Write a short greeting.\"}],\n \"max_tokens\": 128,\n \"stream\": false\n }'\n```\n\nExpect a successful completion while the account has enough budget. Keep an\noutput limit such as `max_tokens` in your requests so Daptin can estimate the\ncapacity needed before starting generation.\n\nAs administrator, fetch the latest usage records:\n\n```\ncurl -g 'http://localhost:6336/api/api_usage?page[size]=1&sort=-created_at' \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\n- \n`state` should be`completed` after a successful request.\n- \n`measures` contains the recorded quantities, including`requests` ,`input_tokens` ,`output_tokens` ,`total_tokens` , and`cost_micros` .\n- \n`user_account_id` identifies the account that made the request.\n\n```\n{\n \"data\": [\n {\n \"type\": \"api_usage\",\n \"id\": \"01a1d901-b2e3-7694-8445-abb2c4a1475b\",\n \"attributes\": {\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"state\": \"completed\",\n \"status_code\": 200,\n \"measures\": {\n \"requests\": 1,\n \"input_tokens\": 12,\n \"output_tokens\": 30,\n \"total_tokens\": 42,\n \"cost_micros\": 72\n },\n \"metadata\": {\n \"attempts\": [\n {\n \"DeploymentID\": \"01a1d45f-f05d-796a-ad14-78024e0c7102\",\n \"Outcome\": \"succeeded\",\n \"HTTPStatus\": 200\n }\n ]\n }\n }\n }\n ]\n}\n```\n\n## Full response\n\n```\n{\n \"links\": {\n \"current_page\": 1,\n \"from\": 0,\n \"last_page\": 10,\n \"last_page_url\": \"/api/api_usage?page[number]=10&page[size]=1&sort=-created_at\",\n \"next_page_url\": \"/api/api_usage?page[number]=2&page[size]=1&sort=-created_at\",\n \"per_page\": 1,\n \"to\": 1,\n \"total\": 10\n },\n \"data\": [\n {\n \"type\": \"api_usage\",\n \"id\": \"01a1d901-b2e3-7694-8445-abb2c4a1475b\",\n \"attributes\": {\n \"__type\": \"api_usage\",\n \"action_name\": null,\n \"api_key_id\": null,\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"created_at\": \"2026-11-14T07:42:09.635806257Z\",\n \"endpoint\": \"/v1/chat\",\n \"entity_type\": \"llm_model\",\n \"error_message\": null,\n \"latency_ms\": 0,\n \"measures\": {\n \"bytes\": 1994,\n \"cache_read_tokens\": 0,\n \"cache_write_tokens\": 0,\n \"cost_micros\": 72,\n \"input_tokens\": 12,\n \"output_tokens\": 30,\n \"reasoning_tokens\": 0,\n \"requests\": 1,\n \"total_tokens\": 42\n },\n \"metadata\": {\n \"attempts\": [\n {\n \"DeploymentID\": \"01a1d45f-f05d-796a-ad14-78024e0c7102\",\n \"EndedAt\": \"2026-11-14T07:42:11.440781925Z\",\n \"ErrorCode\": \"\",\n \"FirstByteAt\": \"0001-01-01T00:00:00Z\",\n \"HTTPStatus\": 200,\n \"Number\": 1,\n \"Outcome\": \"succeeded\",\n \"OutputCommitted\": false,\n \"ProviderID\": \"01a1d45e-8115-70f1-b959-719c7c846cd5\",\n \"Retryable\": false,\n \"StartedAt\": \"2026-11-14T07:42:09.649547674Z\",\n \"UpstreamRequest\": \"\",\n \"Usage\": {\n \"CacheReadTokens\": 0,\n \"CacheWriteTokens\": 0,\n \"CostMicros\": 72,\n \"Estimated\": false,\n \"InputTokens\": 12,\n \"Measures\": null,\n \"OutputTokens\": 30,\n \"ReasoningTokens\": 0,\n \"TotalTokens\": 42\n }\n }\n ],\n \"cache_status\": \"\",\n \"capture_limit\": 16777216,\n \"ended_at\": \"2026-11-14T07:42:11.440781925Z\",\n \"error_code\": \"\",\n \"first_byte_at\": \"0001-01-01T00:00:00Z\",\n \"metering_enabled\": true,\n \"model_id\": \"01a1d3c9-3c03-7ac3-a864-7b7cf5705b14\",\n \"operation\": \"chat\",\n \"payload_format\": \"llm.canonical.v1\",\n \"status\": \"succeeded\"\n },\n \"method\": \"POST\",\n \"parent_usage_id\": null,\n \"permission\": 12160,\n \"reference_id\": \"01a1d901-b2e3-7694-8445-abb2c4a1475b\",\n \"request_body\": \"{\\\"ID\\\":\\\"req_d87e2d429a0d04e60795d21dc37e3711\\\",\\\"Operation\\\":\\\"chat\\\",\\\"PublicModel\\\":\\\"assistant\\\",\\\"Stream\\\":false,\\\"MaxOutputTokens\\\":128,\\\"EstimatedUsage\\\":{\\\"InputTokens\\\":145,\\\"OutputTokens\\\":128,\\\"CacheReadTokens\\\":0,\\\"CacheWriteTokens\\\":0,\\\"ReasoningTokens\\\":0,\\\"TotalTokens\\\":273,\\\"CostMicros\\\":0,\\\"Estimated\\\":true,\\\"Measures\\\":null},\\\"Chat\\\":{\\\"Messages\\\":[{\\\"Role\\\":\\\"user\\\",\\\"Name\\\":\\\"\\\",\\\"Content\\\":[{\\\"Type\\\":\\\"text\\\",\\\"Text\\\":\\\"Write a short greeting.\\\",\\\"Refusal\\\":\\\"\\\",\\\"Annotations\\\":null,\\\"Logprobs\\\":null,\\\"ImageURL\\\":null,\\\"Audio\\\":null,\\\"File\\\":null}],\\\"ToolCalls\\\":null,\\\"ToolCallID\\\":\\\"\\\"}],\\\"Tools\\\":null,\\\"ToolChoice\\\":null,\\\"ResponseFormat\\\":null,\\\"N\\\":1,\\\"Temperature\\\":null,\\\"TopP\\\":null,\\\"FrequencyPenalty\\\":null,\\\"PresencePenalty\\\":null,\\\"MaxCompletionTokens\\\":128,\\\"Stop\\\":null,\\\"User\\\":\\\"\\\",\\\"Seed\\\":null,\\\"Logprobs\\\":false,\\\"TopLogprobs\\\":0,\\\"ParallelToolCalls\\\":null,\\\"ReasoningEffort\\\":\\\"\\\",\\\"Store\\\":null,\\\"PromptCacheKey\\\":\\\"\\\"},\\\"TextCompletion\\\":null,\\\"Responses\\\":null,\\\"Embeddings\\\":null,\\\"ImageGeneration\\\":null,\\\"ImageEdit\\\":null,\\\"ImageVariation\\\":null,\\\"Moderation\\\":null,\\\"Rerank\\\":null,\\\"AudioSpeech\\\":null,\\\"Transcription\\\":null,\\\"Search\\\":null,\\\"OCR\\\":null,\\\"StartedAt\\\":\\\"2026-11-14T07:42:09.625990632Z\\\"}\",\n \"request_body_encoding\": \"utf8\",\n \"request_bytes\": 1111,\n \"request_id\": \"req_d87e2d429a0d04e60795d21dc37e3711\",\n \"request_type\": \"llm_chat\",\n \"reservation_buckets\": {\n \"1:1:cost_micros:month:1793491200\": {\n \"amount\": 401,\n \"bucket_key\": \"1:1:cost_micros:month:1793491200\",\n \"metric\": \"cost_micros\"\n },\n \"1:1:requests:minute:1794642120\": {\n \"amount\": 1,\n \"bucket_key\": \"1:1:requests:minute:1794642120\",\n \"metric\": \"requests\"\n },\n \"1:1:total_tokens:month:1793491200\": {\n \"amount\": 546,\n \"bucket_key\": \"1:1:total_tokens:month:1793491200\",\n \"metric\": \"total_tokens\"\n }\n },\n \"reservation_expires_at\": \"2026-11-14T07:47:09.635328882Z\",\n \"reservation_token\": \"01a1d901-b2df-7ee7-902a-736a150651f0\",\n \"reserved_measures\": {\n \"cache_read_tokens\": 0,\n \"cache_write_tokens\": 0,\n \"cost_micros\": 401,\n \"input_tokens\": 290,\n \"output_tokens\": 256,\n \"reasoning_tokens\": 0,\n \"requests\": 1,\n \"total_tokens\": 546\n },\n \"response_body\": \"{\\\"RequestID\\\":\\\"req_d87e2d429a0d04e60795d21dc37e3711\\\",\\\"Model\\\":\\\"assistant\\\",\\\"Chat\\\":{\\\"ID\\\":\\\"gen-1794642130-y5AWuU6XZPJ07Pgsttwy\\\",\\\"Created\\\":1794642130,\\\"Choices\\\":[{\\\"Index\\\":0,\\\"Message\\\":{\\\"Role\\\":\\\"assistant\\\",\\\"Name\\\":\\\"\\\",\\\"Content\\\":[{\\\"Type\\\":\\\"text\\\",\\\"Text\\\":\\\"Hello! I hope you're having a wonderful day. If there's anything you'd like to chat about or need help with, feel free to let me know!\\\",\\\"Refusal\\\":\\\"\\\",\\\"Annotations\\\":null,\\\"Logprobs\\\":null,\\\"ImageURL\\\":null,\\\"Audio\\\":null,\\\"File\\\":null}],\\\"ToolCalls\\\":null,\\\"ToolCallID\\\":\\\"\\\"},\\\"FinishReason\\\":\\\"stop\\\",\\\"Logprobs\\\":null}]},\\\"TextCompletion\\\":null,\\\"Responses\\\":null,\\\"Embeddings\\\":null,\\\"Images\\\":null,\\\"Moderation\\\":null,\\\"Rerank\\\":null,\\\"AudioSpeech\\\":null,\\\"Transcription\\\":null,\\\"Search\\\":null,\\\"OCR\\\":null,\\\"Usage\\\":{\\\"InputTokens\\\":12,\\\"OutputTokens\\\":30,\\\"CacheReadTokens\\\":0,\\\"CacheWriteTokens\\\":0,\\\"ReasoningTokens\\\":0,\\\"TotalTokens\\\":42,\\\"CostMicros\\\":72,\\\"Estimated\\\":false,\\\"Measures\\\":null}}\",\n \"response_body_encoding\": \"utf8\",\n \"response_bytes\": 883,\n \"state\": \"completed\",\n \"status_code\": 200,\n \"terminal_at\": \"2026-11-14T07:42:11.4654253Z\",\n \"updated_at\": \"2026-11-14T07:42:11.465579883Z\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"version\": 2\n },\n \"relationships\": {\n \"api_member_id\": {\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\"\n }\n },\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"parent_usage_id\": {\n \"data\": null\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n ]\n}\n```\n\nNext, read the quota totals:\n\n```\ncurl -g 'http://localhost:6336/api/api_quota?page[size]=100' \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\"\n```\n\nResponse\n\n```\n{\n \"data\": [\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e2-7c65-b5eb-b21e07a014b8\",\n \"attributes\": {\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"metric\": \"total_tokens\",\n \"maximum\": 1000000,\n \"consumed\": 42,\n \"reserved\": 0,\n \"window_start\": \"2026-11-01T00:00:00Z\",\n \"window_end\": \"2026-12-01T00:00:00Z\"\n }\n },\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e2-73cf-afc4-4c2e125802e0\",\n \"attributes\": {\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"metric\": \"requests\",\n \"maximum\": 60,\n \"consumed\": 1,\n \"reserved\": 0,\n \"window_start\": \"2026-11-14T07:42:00Z\",\n \"window_end\": \"2026-11-14T07:43:00Z\"\n }\n },\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e0-773c-9c21-11248d657707\",\n \"attributes\": {\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"metric\": \"cost_micros\",\n \"maximum\": 10000000,\n \"consumed\": 72,\n \"reserved\": 0,\n \"window_start\": \"2026-11-01T00:00:00Z\",\n \"window_end\": \"2026-12-01T00:00:00Z\"\n }\n }\n ]\n}\n```\n\n## Full response\n\n```\n{\n \"links\": {\n \"current_page\": 1,\n \"from\": 0,\n \"last_page\": 1,\n \"per_page\": 100,\n \"to\": 100,\n \"total\": 3\n },\n \"data\": [\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e2-7c65-b5eb-b21e07a014b8\",\n \"attributes\": {\n \"__type\": \"api_quota\",\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"bucket_key\": \"1:1:total_tokens:month:1793491200\",\n \"consumed\": 42,\n \"created_at\": \"2026-11-14T07:42:09.635099007Z\",\n \"maximum\": 1000000,\n \"metric\": \"total_tokens\",\n \"permission\": 561441,\n \"reference_id\": \"01a1d901-b2e2-7c65-b5eb-b21e07a014b8\",\n \"reserved\": 0,\n \"updated_at\": \"2026-11-14T07:42:11.464126841Z\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"version\": 2,\n \"window_end\": \"2026-12-01T00:00:00Z\",\n \"window_start\": \"2026-11-01T00:00:00Z\"\n },\n \"relationships\": {\n \"api_member_id\": {\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\"\n }\n },\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n },\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e2-73cf-afc4-4c2e125802e0\",\n \"attributes\": {\n \"__type\": \"api_quota\",\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"bucket_key\": \"1:1:requests:minute:1794642120\",\n \"consumed\": 1,\n \"created_at\": \"2026-11-14T07:42:09.634545632Z\",\n \"maximum\": 60,\n \"metric\": \"requests\",\n \"permission\": 561441,\n \"reference_id\": \"01a1d901-b2e2-73cf-afc4-4c2e125802e0\",\n \"reserved\": 0,\n \"updated_at\": \"2026-11-14T07:42:11.44913905Z\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"version\": 2,\n \"window_end\": \"2026-11-14T07:43:00Z\",\n \"window_start\": \"2026-11-14T07:42:00Z\"\n },\n \"relationships\": {\n \"api_member_id\": {\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\"\n }\n },\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n },\n {\n \"type\": \"api_quota\",\n \"id\": \"01a1d901-b2e0-773c-9c21-11248d657707\",\n \"attributes\": {\n \"__type\": \"api_quota\",\n \"api_member_id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\",\n \"api_plan_id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\",\n \"bucket_key\": \"1:1:cost_micros:month:1793491200\",\n \"consumed\": 72,\n \"created_at\": \"2026-11-14T07:42:09.63303309Z\",\n \"maximum\": 10000000,\n \"metric\": \"cost_micros\",\n \"permission\": 561441,\n \"reference_id\": \"01a1d901-b2e0-773c-9c21-11248d657707\",\n \"reserved\": 0,\n \"updated_at\": \"2026-11-14T07:42:11.443380383Z\",\n \"user_account_id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\",\n \"version\": 2,\n \"window_end\": \"2026-12-01T00:00:00Z\",\n \"window_start\": \"2026-11-01T00:00:00Z\"\n },\n \"relationships\": {\n \"api_member_id\": {\n \"data\": {\n \"type\": \"api_member\",\n \"id\": \"01a1d8eb-f034-7cd4-97fa-f8e6bfbe5cd9\"\n }\n },\n \"api_plan_id\": {\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"01a1d8e0-e793-75f0-ba75-1d103fa8994a\"\n }\n },\n \"user_account_id\": {\n \"data\": {\n \"type\": \"user_account\",\n \"id\": \"01a1d46b-46f6-7b6c-a6ea-3f9fe8cc26fc\"\n }\n },\n \"usergroup_id\": {\n \"data\": []\n }\n }\n }\n ]\n}\n```\n\nEach row tracks one metric for one time window. With this plan, look for requests in the current minute, tokens in the current month, and cost in the current month.\n\n| Field | Meaning | \n|---|---|\n| `metric` | What this row counts, such as `requests` or`total_tokens` | \n| `maximum` | The plan's limit for this metric and window | \n| `consumed` | Usage already counted against the limit | \n| `reserved` | Capacity held for requests that are still running | \n| `window_start` ,`window_end` | The UTC time range covered by this row | \n\nThe capacity available for another request is:\n\n```\nmaximum - consumed - reserved\n```\n\nFor example, a token quota with a maximum of 1,000,000, consumed usage of 200,000, and reservations of 10,000 has 790,000 tokens available for new work. Before calling the provider, Daptin estimates how much a request could use and reserves that amount. When the request finishes, it records the usage and releases the unused part of the reservation. This prevents simultaneous requests from all spending the same remaining budget.\n\nA new request must fit within the available capacity. If it needs a reservation of 20,000 tokens but only 15,000 are available, Daptin returns HTTP 402 without calling the provider—even though the account has not consumed its entire limit. Fallback attempts can increase the reservation needed. A conversation continuation also includes the earlier messages replayed to the model, so its input usage grows with the conversation.\n\nTemporarily change this plan to a zero-request hard limit:\n\n```\ncurl -X PATCH http://localhost:6336/api/api_plan/PLAN_ID \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"api_plan\",\n \"id\": \"PLAN_ID\",\n \"attributes\": {\n \"limits\": [\n {\"metric\":\"requests\",\"window\":\"minute\",\"maximum\":0,\"mode\":\"hard\"}\n ]\n }\n }\n }'\n```\n\nAnd repeat the /completions call with the user token, expect a \"insufficient quota\" response\n\n```\ncurl http://localhost:6336/v1/chat/completions \\\n -H \"Authorization: Bearer $USER_DAPTIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"model\": \"assistant\",\n \"messages\": [{\"role\":\"user\",\"content\":\"Write a short greeting.\"}],\n \"max_tokens\": 128,\n \"stream\": false\n }'\n```\n\nExpected response (HTTP 402):\n\n```\n{\"error\":{\"code\":\"insufficient_quota\",\"message\":\"insufficient quota\",\"request_id\":\"req_1a0a162ad2e6393984bb78afe1db5368\",\"type\":\"insufficient_quota\"}}\n```\n\nAn inactive or missing membership does means no plan-defined maximum applies so all requests are let thru.\n\nYou can also apply upstream limits on LLM Deployment to protect upstream from overloading/ratelimiting. For example:\n\n```\ncurl -X PATCH http://localhost:6336/api/llm_deployment/PRIMARY_DEPLOYMENT_ID \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/vnd.api+json' \\\n -d '{\n \"data\": {\n \"type\": \"llm_deployment\",\n \"id\": \"PRIMARY_DEPLOYMENT_ID\",\n \"attributes\": {\"rpm\":60,\"tpm\":100000,\"max_concurrency\":4}\n }\n }'\n```\n\nThe global limiter runs before authentication and is separate from account quotas. Its buckets are client IP plus request path, with one-second UTC windows. The default is 500 requests per second per IP/path. This example sets five per second for each inference path:\n\n```\ncurl http://localhost:6336/_config/backend/limit.rate \\\n -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n -H 'Content-Type: application/json' \\\n -d '{\n \"version\": \"1\",\n \"limits\": {\n \"/v1/chat/completions\": 5,\n \"/v1/responses\": 5\n }\n }'\n```\n\nChange in rate limits require restart\n\nFor more detail, see [API Metering](https://github.com/daptin/daptin/blob/master/wiki/API-Metering.md)\nand [Rate Limiting](https://github.com/daptin/daptin/blob/master/wiki/Rate-Limiting.md).", "url": "https://wpnews.pro/news/daptin-plus-an-llm-router", "canonical_source": "https://github.com/daptin/daptin/wiki/Walkthrough-LLM-Router", "published_at": "2026-11-15 16:39:12+00:00", "updated_at": "2026-11-15 16:53:55.507124+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools", "ai-agents"], "entities": ["Daptin", "OpenRouter", "GPT", "Gemini", "Docker", "SQLite", "Postgres"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/daptin-plus-an-llm-router", "markdown": "https://wpnews.pro/news/daptin-plus-an-llm-router.md", "text": "https://wpnews.pro/news/daptin-plus-an-llm-router.txt", "jsonld": "https://wpnews.pro/news/daptin-plus-an-llm-router.jsonld"}}