{"slug": "rubyllm-2-0-providers-protocols-and-provider-gems", "title": "RubyLLM 2.0: Providers, Protocols, and Provider Gems", "summary": "RubyLLM 2.0, the upcoming release of the Ruby gem for AI chat, will separate providers from protocols, add four new providers to reach a total of seventeen, and default OpenAI to the Responses API while supporting Chat Completions. The update introduces a generator for creating provider gems and enables one provider to speak multiple protocols, as demonstrated by Vertex AI supporting Gemini, Anthropic, Mistral, and Chat Completions. RubyLLM 2.0 also leverages AWS Bedrock Mantle to route models to the appropriate API, such as Claude using Anthropic's Messages API and other models using Responses or Chat Completions.", "body_md": "RubyLLM 2.0 is almost ready. It isn’t out yet, but it will be soon, and I have been looking forward to showing you what is in it.\n\nThere is a lot in this release. Too much for one enormous announcement, and most of it deserves more than a bullet point. So this is the first in a series of posts about what’s coming in RubyLLM 2.0.\n\nLet’s start with providers and protocols.\n\nIn 2.0, OpenAI uses the Responses API by default. Providers and protocols are separate things. Four new providers bring the total to seventeen. And if the provider you need is still missing, a new generator gives you a complete provider gem to start from.\n\n```\nRubyLLM.chat(model: 'gpt-5.4')                              # OpenAI Responses API\nRubyLLM.chat(model: 'gpt-5.4', protocol: :chat_completions) # same model, old API\nRubyLLM.chat(model: 'claude-opus-4-6', provider: :vertexai) # Vertex AI, Anthropic protocol\n```\n\n## A provider is not a protocol\n\nA provider is the service you connect to: OpenAI, Mistral, Vertex AI, or one of the others. It knows the host, credentials, configuration, and model catalog.\n\nA protocol is the API it speaks: Chat Completions, Responses, Anthropic, Gemini, Bedrock Converse, or Cohere. It knows how to build a request, parse the response, and handle streaming.\n\nMistral, DeepSeek, Perplexity, Ollama, and most self-hosted services all speak some version of OpenAI’s Chat Completions API. In 1.x, I handled that with inheritance. `Mistral < OpenAI`\n\nreused the request code, but it also inherited a pile of OpenAI assumptions and then had to undo the ones that did not apply.\n\nThat was only half the problem. One protocol can be used by many providers, but one provider can also speak many protocols. OpenAI speaks Chat Completions and Responses; Vertex AI speaks Gemini, Claude, Mistral, and open models, all through different API shapes. The old design had no good way to express that.\n\nIn 2.0, providers register the protocols they speak and choose one for each model. Here is the actual routing in the Vertex AI provider:\n\n```\nclass VertexAI < Provider\n  protocol :gemini, VertexAI::Gemini\n  protocol :anthropic, VertexAI::Anthropic\n  protocol :mistral, VertexAI::Mistral\n  protocol :chat_completions, VertexAI::ChatCompletions\n\n  def protocol_for(model, **)\n    case model.id\n    when %r{/} then protocols[:chat_completions]\n    when /\\Aclaude/ then protocols[:anthropic]\n    when VertexAI::Mistral::MODELS then protocols[:mistral]\n    else super\n    end\n  end\nend\n```\n\nSo Claude on Vertex AI speaks Anthropic. `meta/llama-3.3-70b-instruct-maas`\n\nspeaks Chat Completions. Gemini speaks Gemini. The model changes, the wire format changes, and your application does not.\n\n## OpenAI uses Responses by default\n\nOpenAI now has two chat APIs. RubyLLM 2.0 supports both, but defaults to Responses.\n\nThat means reasoning models can use tools and extended thinking together, which Chat Completions (in OpenAI) cannot express. It also gives RubyLLM access to newer OpenAI features without adding special cases to the old API.\n\nIf you still need Chat Completions, choose it for one chat:\n\n```\nchat = RubyLLM.chat(model: 'gpt-5.4', protocol: :chat_completions)\n```\n\nOr keep it as the default for OpenAI:\n\n```\nRubyLLM.configure do |config|\n  config.openai_protocol = :chat_completions\nend\n```\n\nAn explicit `protocol:`\n\nwins over configuration, and configuration wins over the provider default.\n\nThe same protocol code is useful outside OpenAI. xAI’s primary API now looks like Responses, so its RubyLLM provider gets streaming, encrypted reasoning, and server tools from the same implementation. Azure and DeepSeek can opt into Responses too.\n\n## One Bedrock endpoint, three APIs\n\nAWS Bedrock Mantle makes the difference between a provider and a protocol very obvious. You connect to one AWS service, but the API changes with the model.\n\nClaude uses Anthropic’s Messages API. Five models use OpenAI’s Responses API. The other forty-one use Chat Completions. They all live in the same catalog, behind the same authentication, on the same host.\n\nEven the model catalogs disagree. Mantle spells some model IDs differently from Bedrock Converse and includes models that Converse does not know about at all.\n\nWithout protocol routing, switching models can also mean switching request formats. RubyLLM reads both catalogs, records which protocol each model speaks, and routes the request for you. The `RubyLLM.chat`\n\nAPI stays the same. You can even switch models in the middle of a conversation with `.with_model`\n\n, and RubyLLM switches protocols with it.\n\n## Four new providers\n\nRubyLLM 2.0 will ship with seventeen providers, up from thirteen in 1.16:\n\n**Cohere** with native chat, embeddings, reranking, and transcription.**Ollama Cloud** with Ollama’s API against its hosted model catalog. No local server required.**ElevenLabs** for speech and transcription.**Deepgram** for speech and transcription.\n\n## A complete provider in one small file\n\nOnce the protocol handles the wire format, a provider can be very small. Here is the entire Mistral provider in RubyLLM 2.0:\n\n```\nmodule RubyLLM\n  module Providers\n    class Mistral < Provider\n      protocol :chat_completions, ChatCompletions, batches: Mistral::ChatCompletions::Batches\n      protocol :files, Protocols::Mistral::Files\n\n      def api_base\n        @config.mistral_api_base || 'https://api.mistral.ai/v1'\n      end\n\n      def headers\n        {\n          'Authorization' => \"Bearer #{@config.mistral_api_key}\"\n        }\n      end\n\n      class << self\n        def capabilities\n          Mistral::Capabilities\n        end\n\n        def models_dev_alias(...)\n          Mistral::Models.models_dev_alias(...)\n        end\n\n        def configuration_options\n          %i[mistral_api_key mistral_api_base]\n        end\n\n        def configuration_requirements\n          %i[mistral_api_key]\n        end\n      end\n    end\n  end\nend\n```\n\nThat’s the whole file. Most providers are between 35 and 100 lines. The longer ones are doing real provider-specific work, like Google authentication or AWS request signing.\n\n## Build the next provider yourself\n\nThe provider and protocol split pays off outside RubyLLM too. The most common feature request is another provider. In 2.0, you do not have to wait for me to add one.\n\n```\nruby_llm provider-gem Acme --api-base https://api.acme.ai/v1\n```\n\nThat command creates `ruby_llm-providers-acme`\n\n, initializes Git, installs the bundle, and gives you an ignored `.env`\n\n, provider registration, a model-catalog task, live-recording specs, RuboCop, Flay, ArchSpec, and CI across every supported [Ruby](https://www.ruby-lang.org/en/) version. It gets the same kind of care as RubyLLM itself, already wired up.\n\nThe important part is what you do not have to build. If Acme speaks Chat Completions, it reuses RubyLLM’s existing protocol for requests, responses, streaming, tools, and errors. If it speaks another familiar API, pass `--dialect responses`\n\n, `anthropic`\n\n, `gemini`\n\n, `converse`\n\n, or `ollama`\n\n. The provider only needs to supply its host, authentication, model catalog, and any real quirks it has.\n\nThat is what the split makes possible. A new or smaller provider no longer needs another OpenAI-compatible implementation, another Anthropic-compatible implementation, or a growing pile of API-base switches and provider exceptions inside RubyLLM. It gets to reuse the protocol and remain a small adapter.\n\nWhen it is ready, tell your users to add it to their Gemfile:\n\n```\ngem 'ruby_llm-providers-acme', require: 'ruby_llm/providers/acme'\n```\n\nWith their API key configured, that is it. The gem registers itself, brings its model catalog, and works through the same `RubyLLM.chat`\n\nAPI as a built-in provider.\n\nOnce 2.0 is out, go ahead and try it yourself. I’ll save the complete tutorial for another post.\n\n## A model registry applications can rely on\n\nThe model registry that RubyLLM uses for capabilities and pricing is now published at [rubyllm.com/models.json](https://rubyllm.com/models.json).\n\nFor the first time, anyone can download, inspect, or build on the same catalog RubyLLM uses itself.\n\n[models.dev](https://models.dev) is an excellent source, but RubyLLM needs more than a copy of it. Every six hours, RubyLLM rebuilds its registry from models.dev and the providers’ own APIs, reconciles aliases, fills gaps, applies the few provider-specific corrections that remain, validates the result, and refuses suspicious regressions before publishing it.\n\nThis is not just a model directory for the documentation. RubyLLM applications use it every day to validate model names, choose protocols, check capabilities, and turn provider usage into real costs. That last part demands precision: if a price, modality, or capability is wrong, the answer your application gets is wrong too. I will cover the cost ledger in another post, but the registry is what makes it possible.\n\n```\nRubyLLM.models.refresh!\n```\n\n`refresh!`\n\nfetches the latest main registry and persists it. New models, prices, context windows, and capabilities can reach your application without waiting for the next gem release.\n\nProvider gems can ship their own `models.json`\n\ntoo. RubyLLM loads it as a read-only fallback behind the main registry, so installing a provider gem is enough to use its models normally:\n\n```\nRubyLLM.chat(model: 'MiniMax-M3').ask('Hello')\n```\n\nThe global `refresh!`\n\nnever refreshes or rewrites provider gem catalogs. Their authors update them with `rake models`\n\ninside the provider gem.\n\nThe important part is that none of this makes the public API more complicated. `RubyLLM.chat`\n\n, `embed`\n\n, `paint`\n\n, and the [Rails](https://rubyonrails.org) integration still work the same way. Most people will simply get better provider coverage. The new `protocol:`\n\noption is there for the times when you want to choose.\n\nThat is the first piece of RubyLLM 2.0. Next up: the agentic loop, and how 2.0 lets you stop it, resume it, and run it one step at a time.\n\nThe full guide to writing providers and protocols is at [rubyllm.com/next/custom-providers](https://rubyllm.com/next/custom-providers/).", "url": "https://wpnews.pro/news/rubyllm-2-0-providers-protocols-and-provider-gems", "canonical_source": "https://paolino.me/rubyllm-2-0-providers-and-protocols/", "published_at": "2026-08-27 00:00:00+00:00", "updated_at": "2026-08-27 14:49:30.937224+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "artificial-intelligence"], "entities": ["RubyLLM", "OpenAI", "Vertex AI", "Mistral", "DeepSeek", "Perplexity", "Ollama", "AWS Bedrock Mantle"], "alternates": {"html": "https://wpnews.pro/news/rubyllm-2-0-providers-protocols-and-provider-gems", "markdown": "https://wpnews.pro/news/rubyllm-2-0-providers-protocols-and-provider-gems.md", "text": "https://wpnews.pro/news/rubyllm-2-0-providers-protocols-and-provider-gems.txt", "jsonld": "https://wpnews.pro/news/rubyllm-2-0-providers-protocols-and-provider-gems.jsonld"}}