{"slug": "rubyllm-2-1", "title": "RubyLLM 2.1", "summary": "RubyLLM 2.1, released by the RubyLLM project, adds an MCP client, typed judgments via TypeSafe's Jev models, agent evaluations, OpenTelemetry tracing, and mid-conversation provider switching, while cutting per-call work across streaming, requests, and persistence. The release's benchmarks against 2.0.0 show streaming 500 Anthropic text deltas dropping from 9.8 ms to 4.1 ms, a 2 MB event arriving in 16 KB pieces falling from 116 ms to 2.0 ms, and memory for a streamed 40-turn chat with a 256 KB image shrinking from 28 MB to 0.48 MB. Opt-in connection reuse via the :net_http_persistent or :async_http Faraday adapters lets calls after the first skip the TCP and TLS handshake.", "body_md": "# What’s New in 2.1\n\nConnect to MCP servers, ask typed judgments, evaluate your agents, and trace them with OpenTelemetry in RubyLLM 2.1.\n\nAfter reading this guide, you will know:\n\n- What got faster, and how to keep connections open between calls.\n- How to connect your chats and agents to MCP servers.\n- How to show what a slow tool is doing while it runs.\n- How to give a model hundreds of tools without sending them all.\n- How to switch providers in the middle of a conversation.\n- How to send files a provider cannot read, and choose how much detail an image gets.\n- How to give a cache boundary its own lifetime, and what a one-hour write costs.\n- How to share pieces between prompt templates.\n- How to ask typed judgments with TypeSafe’s Jev models.\n- How to evaluate your agents against a dataset.\n- How to trace RubyLLM with OpenTelemetry.\n- How to chat with open-weight models on Hetzner.\n- How to name Azure deployments and keep their pricing.\n- What changed for Perplexity chat.\n- How to build an agent’s configuration from its inputs.\n- How to keep RubyLLM’s tables on a secondary database.\n- Which provider errors you can now rescue by their class.\n- How upgrades work from 2.1 on.\n\nRubyLLM 2.1 does less work on every call, and brings an MCP client, typed judgments, evaluations, OpenTelemetry tracing, and more control over where agents and records get their configuration. For everything that arrived in 2.0, see [What’s New in 2.0](https://rubyllm.com/whats-new-in-2-0/).\n\n## Faster by Default\n\nMost of 2.1’s speed comes without changing your code. Streams parse events with RubyLLM’s own parser, append long text and reasoning in place, deduplicate the citations a provider repeats on every chunk in one pass, and compute the cost once, when the stream ends, instead of on every chunk. Requests pick their protocol once instead of once per message, derive tool names once, and parse a successful response once.\n\nA reply no longer keeps the request it answered, so a long chat holds one copy of its history instead of one per reply. If you read that request, see [Read Requests Before They Are Sent](https://rubyllm.com/upgrading/#read-requests-before-they-are-sent). The model registry loads once when several threads reach it together, and Vertex AI calls share one OAuth token instead of fetching their own. Persisted chats rebuild from the rows they already loaded, with fewer queries in applications that predate `automatic_scope_inversing`, and download a stored file only when a request sends it.\n\nThese measurements compare 2.0.0 with 2.1 on the same machine. Providers answer from canned responses, so the times show RubyLLM’s own work:\n\n| Workload | 2.0 | 2.1 | \n|---|---|---|\n| Stream 500 text deltas from Anthropic | 9.8 ms | 4.1 ms | \n| Stream a 2 MB event that arrives in 16 KB pieces | 116 ms | 2.0 ms | \n| Stream 500 Perplexity chunks that each cite 20 sources | 137 ms | 17 ms | \n| Ask a Bedrock chat with 200 messages of history | 2.9 ms | 0.40 ms | \n| Embed 100 texts and read the 2.4 MB response | 4.5 ms | 2.2 ms | \n| Memory kept by a streamed 40-turn chat with a 256 KB image | 28 MB | 0.48 MB | \n| Eight threads loading the model registry at once | 273 ms | 32 ms | \n| Twenty Vertex AI calls with a service account key | 20 token requests | 1 token request | \n| Rebuild a persisted 100-message chat | 11.4 ms | 6.8 ms | \n| The same chat without `automatic_scope_inversing` | 58 queries | 8 queries | \n| Check `awaiting_approval?` on a chat with ten 1 MB images | 10 downloads | none | \n\nTwo more speedups need you to opt in. Calls now share HTTP connections, so an adapter that keeps connections open lets every call after the first skip the TCP and TLS handshake. Use `:net_http_persistent` from the `faraday-net_http_persistent` gem with threads, as in Puma and Sidekiq, or `:async_http` from `async-http-faraday` inside an Async reactor, as in Falcon:\n\n```\nRubyLLM.configure do |config|\n  config.faraday_adapter = :net_http_persistent\nend\n```\n\nTwenty calls to a local HTTPS server through `:net_http_persistent` took one handshake instead of twenty, and 0.4 ms each instead of 2.1 ms. Across a real network, each handshake you skip also saves its round trips. See [Connection Reuse](https://rubyllm.com/configuration-connection/#connection-reuse).\n\nIn Rails, [run the 2.1 upgrade](https://rubyllm.com/upgrading/#upgrade-the-rails-schema) so chats remember the provider uploads of their stored files. A chat loaded in another process then sends the provider’s copy of a large file instead of downloading the file and uploading it again. Five turns of a chat with a 30 MB PDF, each in a new process, uploaded 150 MB with 2.0 and nothing with 2.1.\n\nTo measure on your own machine, clone RubyLLM and compare any version with your checkout. The [benchmarks](https://github.com/crmne/ruby_llm/tree/main/benchmarks) need no API keys or network access:\n\n```\nbundle exec rake \"benchmark:compare[v2.0.0]\"\n```\n\nThe numbers above are medians from an AMD Ryzen 9 9900X running Ruby 4.0.7. Times vary from machine to machine; counts such as queries, downloads, and uploads do not.\n\n## MCP Client\n\nServices such as Linear, GitHub, Notion, Slack, and Dropbox expose their APIs as Model Context Protocol servers. RubyLLM now includes an MCP client. Describe the server you want to connect to in a class, the way you describe a tool or an agent:\n\n```\nclass Linear < RubyLLM::MCP\n  url \"https://mcp.linear.app/mcp\"\n  inputs :user\n  oauth owner: :user\n\n  only :list_issues, :get_issue, :create_issue\n  requires_approval :create_issue\nend\n\nchat = RubyLLM.chat.with_mcp(Linear.new(user: current_user))\nchat.ask \"What's blocking the release?\"\n```\n\nEvery server tool is also a Ruby method, so you can explore a server from the console:\n\n```\ndocs = RubyLLM.mcp(url: \"https://learn.microsoft.com/api/mcp\")\ndocs.tools\ndocs.microsoft_docs_search(query: \"Azure Blob Storage\").text\n```\n\nYou decide what the model sees. Rename and redescribe tools, fix arguments the model should not choose, pass results through your own method, or build higher-level tools from the server’s primitives with a regular `RubyLLM::Tool`. Resources work as attachments, prompts work with `ask`, and a server’s requests for input pause the chat the way tool approvals do, surviving restarts in Rails.\n\nServers change while you use them. When a server says its tools changed, the next turn of a chat sees the new list. Call `listen` to hear about changes as they happen, including updates to the resources and tasks you care about, and react with `after_change`:\n\n```\nclass Handbook < RubyLLM::MCP\n  url \"https://handbook.example.com/mcp\"\n  after_change { |change| ReindexPolicyJob.perform_later(change.uri) if change.is_a?(RubyLLM::MCP::Resource) }\nend\n\nHandbook.new.listen(resources: [\"handbook://policies\"])\n```\n\nThe client speaks the 2026-07-28 revision of the protocol and falls back for servers that predate it. OAuth follows the MCP authorization spec, and Rails keeps the credentials encrypted. Background jobs can connect as your app itself, with client credentials, a private key, or a workload identity token, and companies can authorize their people through their identity provider. Servers that require DPoP get tokens bound to a key.\n\nDeclare the protocol extensions your app supports with `extension`. RubyLLM implements two. With MCP Apps, tools come with a UI your app renders next to their results, and each tool call keeps the result its UI needs, so the UI renders again after a reload. With Tasks, a server runs a long tool call in the background while the chat pauses, the way it pauses for an approval, so no job waits for it:\n\n```\nclass Reports < RubyLLM::MCP\n  url \"https://reports.example.com/mcp\"\n  extension :apps\n  extension :tasks\nend\n```\n\nTools, results, and resources also expose the `_meta` that extensions write, and `log_level` brings a server’s log messages into your logs. See [MCP Client](https://rubyllm.com/mcp/).\n\n## Tool Progress\n\nA tool that downloads a large file or reads a scanned document can take a while. It can now say what it is doing, and your app can show it before the result arrives:\n\n``` python\nclass ReadReport < RubyLLM::Tool\n  def execute(url:)\n    progress \"Downloading #{File.basename(url)}\"\n    pages = Scanner.pages(url)\n    pages.each_with_index.map do |page, index|\n      progress \"Reading page #{index + 1} of #{pages.size}\", value: index + 1, total: pages.size\n      page.text\n    end.join(\"\\n\")\n  end\nend\n\nchat.with_tools(ReadReport).after_tool_progress do |tool_call, progress|\n  puts \"#{tool_call.name}: #{progress.message}\"\nend\n```\n\nMCP server tools report their progress through the same callback. It works with concurrent tool execution, on agents, and on Rails chat records. See [Reporting Progress](https://rubyllm.com/tool-execution/#reporting-progress).\n\n## Tool Search\n\nA few MCP servers can put hundreds of tool definitions in front of the model on every turn. Defer them, and the model searches for the tools it needs instead:\n\n```\nchat.with_mcp(GitHub, Linear, defer: true)\nchat.ask \"Which open issues mention the flaky login spec?\"\n```\n\n`with_tools`, the agent `tools` and `mcp` macros, and Rails chat records take the same `defer:` option, and tool and MCP classes declare it with `defer`. Anthropic and OpenAI’s Responses API search natively; other providers receive deferred tools as ordinary tools, so the same code runs everywhere. See [Tool Search](https://rubyllm.com/tool-search/).\n\n## Switching Providers Mid-Conversation\n\nA conversation can move to another model or provider at any point, even in the middle of a tool round:\n\n```\nchat = RubyLLM.chat(model: \"claude-sonnet-5\").with_provider_tools(:web_search)\nchat.ask \"What changed in the latest Ruby release?\"\n\nchat.with_model(\"gpt-5.6-luna\")\nchat.ask \"Which of those changes affect Rails apps?\"\n```\n\nThinking signatures, provider tool steps, compaction, and other provider-shaped content now go back only to the model that produced them. Another model receives the same conversation as text and tool calls, so it never sees blocks it cannot read. This matters inside one provider too: Bedrock, Vertex AI, Azure, and OpenRouter serve model families that refuse each other’s reasoning. Gemini continues a tool round another provider began, such as a call that waited for an approval while the user switched models.\n\nA worker that dies mid-turn no longer leaves a conversation providers refuse. Requests leave out the blank reply it saved, and calls from a round the conversation has moved past reach the model as unfinished. See [Understanding the Persistence Flow](https://rubyllm.com/rails/#understanding-the-persistence-flow).\n\n## Files a Provider Cannot Read\n\nSome protocols cannot send every file type, such as a Word document. Convert those files yourself, and RubyLLM sends your replacement while the conversation keeps the original:\n\n```\nchat.convert_unsupported_attachments do |attachment|\n  text = TextExtraction.call(attachment)\n  RubyLLM::Attachment.new(StringIO.new(text), filename: \"extracted.txt\") if text\nend\n\nchat.ask \"Summarize this report.\", with: \"report.docx\"\n```\n\nThe block runs only for files the current protocol cannot send, once per file. Switch to a provider that reads the original, and it gets the original. See [Converting Unsupported Attachments](https://rubyllm.com/attachments/#converting-unsupported-attachments).\n\nSmall print needs more detail than a photo. Set `resolution:` on an attachment to trade tokens for detail, from `:low` to `:ultra_high`, or `:original` for the image as it is:\n\n```\npage = RubyLLM::Attachment.new(\"page-3.png\", resolution: :original)\nchat.ask \"What is the revenue figure?\", with: page\n```\n\nGemini, OpenAI, and Azure translate it to their own settings, other providers ignore it, and Rails keeps it with the stored file. See [Choosing the Media Resolution](https://rubyllm.com/attachments/#choosing-the-media-resolution).\n\n## Cache Lifetimes per Boundary\n\nA stable prefix and a growing conversation often deserve different cache lifetimes. Give a boundary its own `ttl:`, and the rest of the chat keeps the lifetime from `with_caching`:\n\n```\nchat = RubyLLM.chat.with_caching\nchat.with_instructions(policy).cache_until_here(ttl: \"1h\")\nchat.ask \"Summarize today's tickets.\"\n```\n\nAnthropic, OpenRouter, and Bedrock Converse set the lifetime on each boundary, and Rails keeps it with the message. One-hour writes cost twice the input price on Anthropic, Claude on Vertex AI, and Bedrock. `tokens.cache_write_by_ttl` splits the writes by lifetime, and `cost.cache_write` prices each one at its own rate. See [Explicit Cache Boundaries](https://rubyllm.com/prompt-caching/#explicit-cache-boundaries).\n\n## Prompt Partials\n\nPrompt templates render partials the way Action View does, so instructions can share pieces:\n\n```\n<%# app/prompts/work_assistant/instructions.txt.erb %>\n<%= render \"tone\", display_name: display_name %>\n<%= render \"shared/safety\" %>\n```\n\nA bare name looks next to the current prompt, a path looks in the prompt roots, and `local_assigns` reads optional locals. See [Partials](https://rubyllm.com/prompt-rendering/#partials).\n\n## Typed Judgments\n\nDefine questions about your application data and read probabilities, choices, and scores:\n\n```\nclass TicketTriage < RubyLLM::Judge\n  probability :urgent, \"Does this need attention today?\"\n\n  choice :department, \"Which team should handle this?\" do\n    billing \"Payments and refunds\"\n    technical \"Bugs and integrations\"\n    other \"Everything else\"\n  end\nend\n\njudgment = TicketTriage.judge(\"Please refund the duplicate charge today.\")\njudgment.urgent.probability\njudgment.department.choice\n```\n\nJudges use `config.default_judgment_model` unless you override the model. They accept structured input, reusable definitions, and runtime procs. Choice and score answers include full distributions and confidence so your application can choose how to act. See [Judgments](https://rubyllm.com/judgments/).\n\nTypeSafe joins the built-in providers, bringing the total to eighteen. Use its hosted Jev models or a [Jev-compatible local server](https://rubyllm.com/configuration-providers/#jev-compatible-apis) through the same judgment API. OpenAI’s `gpt-6-luna` answers the same questions through OpenAI Decisions, including questions about [images](https://rubyllm.com/judgments/#images) passed with `with:`. To judge locally, run Cloudflare’s Clef through [Ollama](https://rubyllm.com/judgments/#local-decision-models) with `provider: :ollama`.\n\n## Evaluations\n\nFind out how often your agent gets the answer right. Write a class that runs your agent and a dataset of cases with reference answers:\n\n```\n# app/evals/support_evaluation.rb\nclass SupportEvaluation < RubyLLM::Evaluation\n  def perform(input)\n    SupportAgent.new.ask(input)\n  end\nend\n# app/evals/support_evaluation.yml\ncases:\n  - name: unopened_return\n    inputs: Can I return an unopened item after 14 days?\n    expected_output: Yes, unopened items can be returned within 30 days.\nbin/rails \"ruby_llm:eval[SupportEvaluation]\"\n```\n\nRubyLLM checks each answer against the reference with a model, and accepts different wording with the same meaning. Declare your own criteria, grade with an Agent or a Judge, add Ruby assertions on tool calls, and run the same cases as RSpec or Minitest tests. Reports include every verdict with its reason, plus the tokens and cost of the run. See [Evaluations](https://rubyllm.com/evaluations/).\n\n## OpenTelemetry\n\nSend model calls, tool runs, and workflows to your tracing backend as spans that follow the OpenTelemetry GenAI conventions:\n\n```\nRubyLLM::OpenTelemetry.enable\n```\n\nRubyLLM uses the SDK and exporters your application configures, and exports metadata such as models, token counts, and finish reasons, never prompts or responses. See [OpenTelemetry](https://rubyllm.com/opentelemetry/).\n\n## Hetzner\n\nHetzner joins the built-in providers, bringing the total to nineteen. Its experimental Inference API serves open-weight models from Hetzner’s data centers:\n\n```\nRubyLLM.chat(model: \"Qwen3.8-27B\", provider: :hetzner).ask(\"Hello from Hetzner\")\n```\n\nSet `hetzner_api_key` to a token from the Hetzner Console. See [Provider Setup](https://rubyllm.com/configuration-providers/#hetzner).\n\n## Azure Deployments\n\nAzure models now come with pricing and context windows from models.dev, so Azure chats report their cost. When a deployment name differs from the model it deploys, declare it, and the chat takes that model’s pricing, limits, and capabilities:\n\n``` js\nRubyLLM.configure do |config|\n  config.azure_deployments = { \"gpt-4o-global\" => \"gpt-4o\" }\nend\n\nRubyLLM.chat(model: \"gpt-4o-global\", provider: :azure)\n```\n\nGPT-6 deployments run on the Responses API. See [Azure Deployments](https://rubyllm.com/configuration-providers/#azure-deployments).\n\n## Perplexity on the Agent API\n\nPerplexity retires Sonar on September 27, 2026, so Perplexity chat now runs on its Agent API through presets such as `fast` and `high`. Chats that name a Sonar model keep working with the preset Perplexity recommends and a deprecation warning. See [Move Perplexity Chat to Presets](https://rubyllm.com/upgrading/#move-perplexity-chat-to-presets).\n\n## Agent Configuration\n\nAn agent can now build its configuration for each chat from its inputs, so every tenant can bring its own credentials or endpoint. A `context` block without arguments runs when the chat is built:\n\n```\nclass SupportAgent < RubyLLM::Agent\n  model \"gpt-5.6-luna\", provider: :openai\n  inputs :workspace\n\n  context do\n    RubyLLM.context { |config| config.openai_api_key = workspace.openai_api_key }\n  end\nend\n\nSupportAgent.chat(workspace: current_workspace)\n```\n\nIt works with `SupportAgent.chat` and with Rails-backed agents, where the block also sees the chat record. A block that takes the configuration, as before, still runs once when the class is defined. See [Configuration Contexts](https://rubyllm.com/agents/#configuration-contexts).\n\n## A Secondary Database for RubyLLM\n\nRubyLLM’s supporting records can share a secondary database with your chats and messages. Point them at the same connection:\n\n```\nRails.application.config.to_prepare do\n  RubyLLM::ActiveRecord::Record.connection_specification_name =\n    LlmRecord.connection_specification_name\nend\n```\n\nSee [Rails Advanced Configuration](https://rubyllm.com/rails-advanced-config/).\n\n## Usage Beyond Chats\n\nIn Rails, one-shot operations such as `RubyLLM.transcribe` and `RubyLLM.embed` write to the usage ledger too, attributed with `owner: current_user` or `RubyLLM.with_usage_owner(current_user) { ... }`. Video and research jobs write a row when they finish, with the cost xAI and OpenRouter report for videos. See [One-Shot Operations](https://rubyllm.com/cost-and-usage-tracking/#one-shot-operations).\n\nProvider tool use is counted the same way everywhere. `tokens.server_tool_use` reports `web_search_requests` on every provider that runs searches, including OpenAI, Gemini, and Perplexity, and Rails keeps the counts with each usage row so you can price them from your own records. See [Pricing Tool Use](https://rubyllm.com/cost-and-usage-tracking/#pricing-tool-use).\n\n## Errors You Can Rescue\n\nProviders report the same problem in different ways, and RubyLLM now raises the same class for each:\n\n- A rejected API key raises `RubyLLM::UnauthorizedError` on every provider, including Gemini, xAI, and Bedrock, which answer with a 400 or a 403.\n- An Anthropic account out of credit raises `RubyLLM::PaymentRequiredError` .\n- A 400 that says the servers are overloaded raises `RubyLLM::OverloadedError` , so retries and fallbacks treat it as capacity.\n- Token quota 429s raise `RubyLLM::RateLimitError` , and a request too large for the model raises`RubyLLM::ContextLengthExceededError` .\n\nRetries follow a provider’s `retry-after-ms`, and Bedrock retries sign their requests again, so a slow first attempt no longer ends in an expired signature.\n\nWhen a provider rejects a conversation without saying why, `error.request_shape` lists each turn’s parts and their sizes, never their contents, and the problems providers are known to refuse, such as a part with no data. See [Describing a Rejected Request](https://rubyllm.com/error-handling/#describing-a-rejected-request).\n\n## Upgrades, One Release at a Time\n\nFrom 2.1 on, each release ships the upgrade from the release before it. `bin/rails generate ruby_llm:upgrade` in 2.1 adds the MCP credentials table, columns for tool calls waiting on input or a task and for the results MCP Apps render, a table where chats remember the provider uploads of their stored files, a column where usage keeps the provider tool uses of each attempt, a column where messages keep their cache lifetime, and an owner for usage outside a chat. Applications on 1.x upgrade to 2.0 first. See [Upgrading](https://rubyllm.com/upgrading/).\n\n## Try 2.1\n\nRubyLLM 2.1 requires Ruby 3.2 or later. Install it:\n\n```\nbundle add ruby_llm --version \"~> 2.1.0\"\n```\n\nStart with [Getting Started](https://rubyllm.com/getting-started/), or follow [Upgrading](https://rubyllm.com/upgrading/) to update a 2.0 application.", "url": "https://wpnews.pro/news/rubyllm-2-1", "canonical_source": "https://rubyllm.com/whats-new-in-2-1/", "published_at": "2026-10-15 07:39:44+00:00", "updated_at": "2026-10-15 07:47:31.417368+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "developer-tools", "large-language-models", "ai-tools"], "entities": ["RubyLLM", "RubyLLM 2.1", "MCP", "OpenTelemetry", "TypeSafe", "Jev", "Anthropic", "Perplexity"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/rubyllm-2-1", "markdown": "https://wpnews.pro/news/rubyllm-2-1.md", "text": "https://wpnews.pro/news/rubyllm-2-1.txt", "jsonld": "https://wpnews.pro/news/rubyllm-2-1.jsonld"}}