{"slug": "a-proposed-browser-standard-for-provider-agnostic-ai-inference", "title": "A proposed browser standard for provider-agnostic AI inference", "summary": "Developer Sam Samskies has proposed the Inference Provider API (IPA), an experimental draft browser standard that lets web applications request AI inference from a user-approved browser extension without accessing API keys, routing to providers such as OpenAI, Anthropic, OpenRouter, Ollama, or compatible servers. The accompanying Inference Bridge Chrome extension implements the standard, and demo apps showcase chat, social, translation, and Nostr feed use cases.", "body_md": "A proposed browser standard for provider-agnostic AI inference.\n\n**Status:** Experimental Draft\n\n**Spec:** [SPEC.md](/SamSamskies/inference-provider-api/blob/main/SPEC.md)\n\n[Inference Bridge](https://chromewebstore.google.com/detail/inference-bridge/ekjldffogogadhfhgkibgkfdhhikfamd)— official Chrome extension that injects`window.inference`\n\nand routes to OpenAI, Anthropic, OpenRouter, Ollama, or experimental OpenAI-compatible servers ([source](https://github.com/SamSamskies/inference-bridge))\n\nInstall from the [Chrome Web Store](https://chromewebstore.google.com/detail/inference-bridge/ekjldffogogadhfhgkibgkfdhhikfamd), or for development clone the repository and load it unpacked from `chrome://extensions`\n\n(Developer mode → Load unpacked → select the repo root).\n\n[Examples index](https://samsamskies.github.io/inference-provider-api/)— gallery of demo apps ([source](/SamSamskies/inference-provider-api/blob/main/examples/index.html))[Chat demo](https://samsamskies.github.io/inference-provider-api/chat/)— minimal chat UI that uses the API ([source](/SamSamskies/inference-provider-api/blob/main/examples/chat))[Social demo](https://samsamskies.github.io/inference-provider-api/social/)— post + replies with a Grok-like Ask AI panel ([source](/SamSamskies/inference-provider-api/blob/main/examples/social))[Translate demo](https://samsamskies.github.io/inference-provider-api/translate/)— short haiku translated with`ipa-tools`\n\n`complete`\n\n([source](/SamSamskies/inference-provider-api/blob/main/examples/translate))[Nostr feed demo](https://samsamskies.github.io/inference-provider-api/nostr/)— a feed of Nostr notes filtered with natural language using Inference Bridge experimental tool calling ([source](/SamSamskies/inference-provider-api/blob/main/examples/nostr))\n\nThe specification defines the standard. Inference Bridge implements that standard and may also include experimental features that are not part of the API contract yet. Applications should target the Inference Provider API (`request`\n\nand `getFeatures`\n\n), not extension-specific namespaces.\n\nToday, every AI-powered web application has to reinvent the same infrastructure:\n\n- Ask users for API keys\n- Integrate every inference provider separately\n- Proxy requests through their own backend\n- Build custom permission systems\n\nThe **Inference Provider API (IPA)** proposes a standard browser interface that allows web applications to request inference from a user-approved browser extension without ever accessing API keys.\n\nInspired by [NIP-07](https://github.com/nostr-protocol/nips/blob/master/07.md), IPA separates **applications** from **providers**, giving users complete control over where inference is performed.\n\n- Users own their API keys.\n- Applications request inference, not providers.\n- Users choose providers.\n- Users choose models.\n- Applications should be provider agnostic.\n- Local and remote inference are first-class citizens.\n- Permission is explicit.\n- API keys never leave the browser extension.\n\n``` js\nfor await (const chunk of window.inference.request({\n  method: \"chat\",\n  messages: [\n    {\n      role: \"user\",\n      content: `Is this true?:\\n\\nNostr is dead.`\n    }\n  ]\n})) {\n  if (chunk.type === \"accepted\") {\n    // permission resolved; provider call may begin\n  } else if (chunk.type === \"reasoning_delta\") {\n    // optional: model reasoning / chain-of-thought\n  } else if (chunk.type === \"delta\") {\n    // append chunk.content to the reply UI\n  } else if (chunk.type === \"done\") {\n    // final message / usage; message.reasoning when reasoning was streamed\n  }\n}\n```\n\n`request`\n\nis required. `getFeatures`\n\nreports optional capabilities such as tool calling and request `options`\n\n(for example `reasoningEffort`\n\n, `temperature`\n\n); implementations that omit it advertise none. If the app only needs the final message, drain to `done`\n\n(inline sketch — or use [ ipa-tools](/SamSamskies/inference-provider-api/blob/main/packages/ipa-tools)’s\n\n`complete`\n\n):\n\n``` js\nasync function complete(request) {\n  let done;\n  for await (const chunk of window.inference.request(request)) {\n    if (chunk.type === \"done\") done = chunk;\n  }\n  return done;\n}\n\nconst { model, message, usage } = await complete({\n  method: \"chat\",\n  messages: [{ role: \"user\", content: \"Is this true?:\\n\\nNostr is dead.\" }],\n});\n```\n\nThe helper is application code, not part of `window.inference`\n\n. It throws `InferenceError`\n\nthe same way iterating `request`\n\ndoes.\n\nFeature-detect optional capabilities before sending tools. Missing `getFeatures`\n\nmeans none:\n\n``` js\nconst features = window.inference.getFeatures?.() ?? {};\n\nif (features.toolCalling) {\n  const tools = [\n    {\n      type: \"function\",\n      function: {\n        name: \"get_weather\",\n        description: \"Get the current weather for a city\",\n        parameters: {\n          type: \"object\",\n          properties: { city: { type: \"string\" } },\n          required: [\"city\"],\n        },\n      },\n    },\n  ];\n\n  for await (const chunk of window.inference.request({\n    method: \"chat\",\n    messages: [{ role: \"user\", content: \"What's the weather in Austin?\" }],\n    tools,\n  })) {\n    if (chunk.type === \"done\" && chunk.message.toolCalls?.length) {\n      // page executes the function, appends role: \"tool\" results, calls request again\n    }\n  }\n}\n```\n\nRequest `options`\n\ncan be sent without feature detection — unsupported keys are ignored:\n\n``` js\n// Prefer less thinking / lower temperature for translation\nfor await (const chunk of window.inference.request({\n  method: \"chat\",\n  messages: [{ role: \"user\", content: \"Translate to Spanish: Hello\" }],\n  options: {\n    reasoningEffort: \"none\",\n    temperature: 0.2,\n  },\n})) {\n  // ...\n}\n```\n\nAny multi-round tool loop is application code. Implementations that do not\nadvertise `toolCalling`\n\nreject `tools`\n\nwith `invalid_request`\n\n. Unsupported\n`options`\n\nkeys are ignored (not rejected) so apps may send them for forward\ncompatibility. For a ready-made loop (plus types and `complete`\n\n), see the\nnon-normative [ ipa-tools](/SamSamskies/inference-provider-api/blob/main/packages/ipa-tools) package\n(\n\n`npm install ipa-tools`\n\n).Sending `tools`\n\non IPA `request`\n\nwithout a `toolCalling`\n\nadvertisement is\n`invalid_request`\n\n. Prefer `getFeatures().toolCalling`\n\nbefore enabling tools;\nsee [ ipa-tools](/SamSamskies/inference-provider-api/blob/main/packages/ipa-tools#when-toolcalling-is-not-advertised).\nThe extension prompts the user for permission:\n\n```\nAllow inference?\nprimal.net\n\nProvider\n[ Ollama ▼ ]\n\nModel\n[ Gemma 4 ▼ ]\n\nRequest preview\nuser: Is this true?:\n\nNostr is dead.\n\n[ ] Remember for this site\nAllow once, or deny only this request.\n\n[Allow]  [Deny]\n```\n\nRequest preview is optional extension UX for this draft, not part of the API\ncontract. When the request includes `tools`\n\n, the permission UI must list the\nfunction names; a persistent chat grant does not silently cover a later tools\nrequest. A follow-up that only appends `role: \"tool\"`\n\nresults may not re-prompt\nand may not appear in any preview, so applications should disclose what data\nthose tools will send to the provider **before** Allow (in the first `messages`\n\n,\nthe tool description, or the page UI). See [SPEC.md — Tool calling](/SamSamskies/inference-provider-api/blob/main/SPEC.md#tool-calling).\n\nThe user chooses the provider and model. With “Remember for this site” checked, Allow persists access for that origin together with the chosen provider and model; Deny permanently blocks it. Changing the extension’s global default does not alter existing origin grants.\n\nText chat is required. Tool calling is optional: implementations that support it\nreturn `{ toolCalling: true }`\n\nfrom `getFeatures`\n\nand accept `tools`\n\non\n`request`\n\n. The page defines and executes function tools; the extension only\nrelays schemas, `toolCalls`\n\n, and results. Optional `options`\n\n(for example\n`options.reasoningEffort`\n\n: `\"auto\" | \"none\" | \"low\" | \"medium\" | \"high\"`\n\n,\n`options.temperature`\n\n: number in `[0, 2]`\n\n) lets apps prefer generation settings\nwhen the matching `getFeatures().options`\n\nkey is true — not a permission change;\nuser override or clamp controls are optional extension UX. See\n[SPEC.md](/SamSamskies/inference-provider-api/blob/main/SPEC.md).\n\n- Standard browser API\n- Provider agnostic\n- Bring Your Own Key (BYOK)\n- Local-first compatible\n- Per-origin permissions\n- Streaming support\n- Zero backend required\n- Optional capability discovery (\n`getFeatures`\n\n) - Optional function tools, executed by the page\n- Optional request\n`options`\n\n(for example`reasoningEffort`\n\n,`temperature`\n\n)\n\n- Replacing provider SDKs\n- Billing\n- Authentication\n- Defining inference protocols\n- Choosing the \"best\" model\n\nAn IPA-compatible browser extension could route requests to any provider, including:\n\n- OpenAI\n- Anthropic\n- Google Gemini\n- xAI\n- OpenRouter\n- ppq.ai\n- Routstr\n- Ollama\n- LM Studio\n- Local inference servers\n\nApplications should not need to know which provider the user has selected.\n\nLocal servers often reject requests that carry a `chrome-extension://`\n\n`Origin`\n\nheader (commonly HTTP 403). IPA extensions that support local inference should\nstrip or rewrite that header on their own requests to loopback endpoints so\nusers are not asked to set `OLLAMA_ORIGINS=chrome-extension://*`\n\nor similar\nallowlists. Widening the local server's origin allowlist remains a fallback, not\nthe preferred path.\n\n**Chrome MV3 reference:** [Inference Bridge](https://github.com/SamSamskies/inference-bridge)\ndoes this with `declarativeNetRequestWithHostAccess`\n\nand dynamic rules in\n[ src/ollama-origin-bypass.js](https://github.com/SamSamskies/inference-bridge/blob/main/src/ollama-origin-bypass.js)\nand\n\n[that remove](https://github.com/SamSamskies/inference-bridge/blob/main/src/loopback-origin-bypass.js)\n\n`src/loopback-origin-bypass.js`\n\n`Origin`\n\n/ `Referer`\n\nfor local Ollama and other loopback\nOpenAI-compatible servers. See [SPEC.md](/SamSamskies/inference-provider-api/blob/main/SPEC.md)Security for the normative guidance.\n\nThat permission lets the extension modify request headers only for hosts already\nlisted in `host_permissions`\n\n—it is not a browser-wide rewrite capability. Still\ntreat it as privileged: a compromised or overly broad extension could alter\nheaders on those hosts. Prefer port-scoped loopback permissions (for example\n`http://localhost:11434/*`\n\n) over `http://localhost/*`\n\n, keep DNR rules limited to\nlocal inference endpoints, and do not use DNR to touch remote provider traffic.\nThis is still preferable to asking every user to set\n`OLLAMA_ORIGINS=chrome-extension://*`\n\n, which trusts every installed extension\ntalking to Ollama.\n\n- A \"Grok\" button on every social post.\n- AI-powered documentation.\n- Browser-based coding tools and other page-executed function tools.\n- Translation.\n- Writing assistance.\n- Local-first AI applications.\n\nSome topics that still need community discussion:\n\n- Is\n`window.inference`\n\nthe right namespace? - Which further capability constraints, if any, do applications need beyond tools and\n`options`\n\n? - Are\n`\"auto\" | \"none\" | \"low\" | \"medium\" | \"high\"`\n\nthe right`options.reasoningEffort`\n\nlevels, or should the field become a provider-mapped budget/token object? - Which further keys belong under\n`options`\n\n(for example`maxTokens`\n\n), and should clamp/override UX stay optional? - Should model selection always remain under user control?\n- Should images, embeddings, and speech use this API or separate APIs?\n- How should extensions surface token usage? Should estimated cost remain optional UX until pricing metadata is defined?\n- Should\n`getFeatures`\n\ngrow beyond booleans (for example nested tool kinds), or stay one key per capability? - Should hosted / provider-executed tools (web search, MCP) be specified, or remain implementation-specific?\n- Should tool calls stream as their own chunk type, or stay on\n`done.message.toolCalls`\n\nonly? - Should structured outputs (e.g. JSON Schema /\n`responseFormat`\n\n) be part of IPA, or left to prompt engineering until providers converge? - How should permission UIs present multi-message requests — e.g. emphasize the last user message and collapse system/context by default?\n- Should applications be encouraged or required to round-trip\n`message.reasoning`\n\non later turns for providers that benefit from it?\n\nThis proposal is intentionally in an early draft stage.\n\nThe goal is to collaboratively design an open browser standard for provider-agnostic inference—not a specific implementation.\n\nContributions of all kinds are welcome, including:\n\n- Design feedback\n- API suggestions\n- Security considerations\n- Alternative approaches\n- Reference implementations\n- Browser extension prototypes\n- Related standards or prior art\n\nIf you have an idea or concern, please open an issue.\n\nThis project is licensed under the MIT License. See [LICENSE](/SamSamskies/inference-provider-api/blob/main/LICENSE) for details.", "url": "https://wpnews.pro/news/a-proposed-browser-standard-for-provider-agnostic-ai-inference", "canonical_source": "https://github.com/SamSamskies/inference-provider-api", "published_at": "2026-08-20 09:00:50+00:00", "updated_at": "2026-08-20 09:44:38.778360+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools", "ai-policy"], "entities": ["Sam Samskies", "Inference Provider API", "Inference Bridge", "OpenAI", "Anthropic", "OpenRouter", "Ollama", "Chrome Web Store"], "alternates": {"html": "https://wpnews.pro/news/a-proposed-browser-standard-for-provider-agnostic-ai-inference", "markdown": "https://wpnews.pro/news/a-proposed-browser-standard-for-provider-agnostic-ai-inference.md", "text": "https://wpnews.pro/news/a-proposed-browser-standard-for-provider-agnostic-ai-inference.txt", "jsonld": "https://wpnews.pro/news/a-proposed-browser-standard-for-provider-agnostic-ai-inference.jsonld"}}