# Solid Queue 1.6.0 Fiber Workers: How Rails Turned Concurrency Primitives into Agent Job Orchestration > Source: > Published: 2026-10-18 20:06:13+00:00 Solid Queue 1.6.0 ships with fiber-based workers, replacing the traditional thread-per-job model with cooperative multitasking. This is not a cosmetic change. Fibers let you run hundreds of concurrent jobs in a single OS thread, cutting memory overhead and context-switching costs. For agent workflows that spawn many lightweight tasks (API calls, LLM token streaming, webhook dispatches), this changes the economics of job orchestration. Rails already had Active Job and Solid Queue for database-backed job queues. The new fiber mode integrates the Async gem, which provides non-blocking I/O primitives. Instead of blocking a thread while waiting for HTTP responses or database queries, a fiber yields control back to the scheduler. The scheduler picks another ready fiber, keeping the CPU busy without spawning new threads. Agent systems often need to coordinate many small tasks: fetching context from vector stores, calling LLM APIs, updating state in a database, sending notifications. Each task involves I/O waits. Traditional thread pools handle this by allocating one thread per job. If you have 100 concurrent jobs, you need 100 threads. Each thread consumes 1-2 MB of stack memory, and the OS scheduler juggles them with preemptive context switches. Fibers flip the model. You can run 1,000 fibers on a handful of threads. Each fiber has a small stack (kilobytes, not megabytes). Context switches happen cooperatively when a fiber explicitly yields, not when the OS preempts it. This reduces memory pressure and eliminates the overhead of kernel-level thread scheduling. The trade-off: fibers only work if your I/O operations are non-blocking. If a fiber calls a blocking system call, it stalls the entire thread. Solid Queue 1.6.0 solves this by integrating Async, which wraps network and file I/O in non-blocking primitives. Solid Queue's fiber mode runs on top of the Async gem. Here's the execution flow: `perform` method. When it hits I/O (HTTP request, database query), it yields control. The supervisor process itself runs in a fiber. This means polling the job queue is non-blocking. If no jobs are ready, the supervisor fiber yields, and other fibers (like health checks or metrics exporters) can run. You enable fiber mode in `config/queue.yml`: ``` production: dispatchers: - polling_interval: 1 batch_size: 50 workers: - queues: default threads: 4 processes: 2 mode: fiber max_fibers: 100 ``` Key parameters: `mode: fiber` activates the fiber scheduler.`max_fibers` caps the number of concurrent fibers per worker process.`threads` still matters: each thread runs its own fiber scheduler. More threads mean more parallel I/O operations. This is where fiber concurrency gets tricky. Active Record's connection pool expects one connection per thread. If you have 4 threads and 100 fibers, you still only have 4 database connections. If 10 fibers try to query the database simultaneously on the same thread, 6 of them will block waiting for a connection. Solid Queue handles this by making database queries fiber-aware. When a fiber requests a connection and none are available, it yields. The scheduler moves to another fiber. When a connection is released, the waiting fiber resumes. This works because Async patches Active Record's connection adapter to use non-blocking I/O. If you use raw SQL or a gem that bypasses Active Record, you lose this benefit. The fiber will block, stalling the thread. Fiber-based concurrency introduces new failure modes: | Failure Mode | Cause | Mitigation | |---|---|---| | **Fiber leak** | Job never yields or completes | Set `max_execution_time` per job class | | **Connection starvation** | Too many fibers, too few DB connections | Increase connection pool size or reduce `max_fibers` | | **Blocking I/O** | Gem uses blocking system calls | Audit dependencies, use Async-compatible libraries | | **Memory bloat** | Fibers accumulate state | Explicitly clear large objects after use | Backpressure is critical. If jobs arrive faster than fibers can process them, the queue grows unbounded. Solid Queue's `max_fibers` setting acts as a concurrency limiter. Once you hit the cap, new jobs wait in the database until a fiber finishes. For agent workflows, this means you need to tune `max_fibers` based on your bottleneck. If your agents spend most of their time waiting for LLM API responses, you can run many fibers. If they do CPU-heavy parsing, fewer fibers per thread makes sense. Agents often maintain state across multiple job executions: conversation history, tool call results, partial outputs. In a thread-per-job model, you can store state in thread-local variables. In fiber mode, thread-locals are shared across all fibers on that thread. You need fiber-local storage. Ruby provides `Fiber[]` for this: ``` python class AgentJob < ApplicationJob def perform(agent_id, task) Fiber[:agent_context] = load_context(agent_id) result = execute_task(task) save_context(agent_id, Fiber[:agent_context]) result ensure Fiber[:agent_context] = nil end end ``` Fiber-local variables are isolated per fiber. When the scheduler switches fibers, each fiber sees its own `Fiber[:agent_context]`. The risk: if you forget to clear fiber-locals, they persist until the fiber is garbage-collected. In a long-running worker process with a fiber pool, this leaks memory. Traditional thread-based observability tools break in fiber mode. Thread IDs no longer map one-to-one with job executions. You need fiber-aware instrumentation. Solid Queue emits Active Support notifications for job lifecycle events: `enqueue.active_job`` perform_start.active_job``perform.active_job` These events include job metadata but not fiber IDs. To trace fiber execution, you need to inject fiber context into your logs: ``` class AgentJob < ApplicationJob around_perform do |job, block| fiber_id = Fiber.current.object_id Rails.logger.tagged("fiber:#{fiber_id}") do block.call end end end ``` This tags all log lines from a fiber with its object ID. You can correlate logs across yields and resumes. For distributed tracing (OpenTelemetry, Datadog), you need to propagate trace context through fiber yields. The Async gem provides hooks for this, but you must configure them explicitly. Fiber mode shines when your jobs are I/O-bound and you need high concurrency. Typical agent workloads that benefit: Fiber mode is a poor fit when: `IO.read` or `Net::HTTP` without Async wrappers. Solid Queue runs as a standalone process, separate from your web server. In production, you typically deploy: For agent systems, you might split workers by job type: This prevents slow jobs from starving fast ones. Use Solid Queue with fiber workers when you need to orchestrate hundreds of lightweight, I/O-bound agent tasks on a single Rails instance. The memory savings and reduced context-switching overhead make it viable to run high-concurrency workloads without scaling horizontally. Avoid fiber mode if your jobs are CPU-intensive, your dependencies use blocking I/O, or you need strict thread isolation. The cooperative multitasking model requires careful auditing of your job code and its dependencies. One blocking call can stall an entire thread's worth of fibers. For agent workflows that mix LLM API calls, database queries, and webhook dispatches, fiber workers are a practical upgrade. You get better resource utilization without rewriting your job logic. Just watch your connection pool size and tune `max_fibers` based on your actual I/O wait times.