# Running AI Agents as Background Jobs with Solid Queue

> Source: <https://nsinenko.com/rails/ai-agents/background-jobs/2026/06/15/solid-queue-ai-agents-background-jobs/>
> Published: 2026-06-15 07:20:00+00:00

# Running AI Agents as Background Jobs with Solid Queue

Run AI agents as background jobs with Solid Queue: queue isolation, concurrency limits, safe retries, resumable runs, and cost control for production.

An AI agent is the worst-behaved background job you will ever run. It is slow, often tens of seconds across a multi-tool loop. It costs real money every time it executes, because every step is tokens on a provider bill. And worst of all, retrying it does not replay. A re-run is a fresh roll of the dice that can take a different path and call different tools than the attempt that failed.

That combination quietly breaks assumptions baked into how most teams configure their queue. The defaults assume jobs are fast, free, and deterministic, so retrying one is harmless. None of that is true for an agent. Solid Queue has the right primitives to handle it - dedicated workers, concurrency limits, controlled retries - but only if you set them up on purpose.

I have written about building the agent itself, with the [Anthropic SDK](/rails/ai-agents/architecture/2026/06/09/building-ai-agents-ruby-anthropic-sdk/) and the [Gemini Interactions API](/rails/ai-agents/architecture/2026/06/26/building-ai-agents-ruby-gemini-interactions-api/). This post is about the other half, the part nobody talks about: once you have a working agent, how do you run it as a background job without it starving your app or torching your budget. It assumes Solid Queue is already running; if not, start with the [practical guide](/rails/background-jobs/performance/2025/10/07/solid-queue-rails-practical-guide/).

## How an agent job differs from a normal job

An agent run violates the three things a queue's defaults assume about a job: that it is fast, that it is free, and that it is safe to retry. Every operational decision below follows from this table, so it is worth fixing in your head before any code:

| Property | Typical background job | AI agent run |
|---|---|---|
| Duration | Milliseconds to a few seconds | Tens of seconds across a tool loop |
| Cost per run | Effectively free | Real money, every step is metered tokens |
| Determinism | Same input, same path | Non-deterministic, a retry can take a new path |
| Safe to blind-retry? | Usually yes | No, replays side effects and spend |
| Right home | Shared `default` queue |
Isolated queue and worker pool |

Read the right column as a list of problems to solve. Isolation, throttling, narrow retries, resumability, and cost tracking each exist to neutralize one row.

## Why it has to be a job

You cannot run a multi-tool agent loop inside a web request. The loop calls the model, runs a tool, calls the model again, and keeps going until it has an answer, which routinely outlives both your Puma worker's timeout and the user's patience. So the controller creates a record, kicks off a job, and returns immediately. The job owns the loop.

``` python
class AgentRunJob < ApplicationJob
  queue_as :agents

  def perform(agent_run_id)
    agent_run = AgentRun.find(agent_run_id)

    result = AgentRunner.new(
      client: LlmClient.build,
      tool_registry: ToolRegistry.new(agent_run.user)
    ).run(
      input: agent_run.input,
      previous_interaction_id: agent_run.previous_interaction_id
    )

    agent_run.update!(status: "completed", output: result[:output])
  end
end
```

The loop internals - tool execution, state, parsing - are their own topic, covered in the agent-building posts. The only line that matters yet is `queue_as :agents`

. That one choice is what makes everything below possible.

## Isolate agent work on its own queue and workers

This is the single highest-leverage move, so do it first. Agent runs are slow, and a slow job on a shared queue blocks every fast job behind it. Picture a 90-second agent run sitting on your `default`

queue while an order-confirmation email waits in line behind it. The email is now 90 seconds late because an unrelated AI task happened to be ahead of it.

Give agent work a dedicated queue and a dedicated worker pool, separate from the workers that run your fast transactional jobs:

```
# config/queue.yml
production:
  workers:
    - queues: [real_time, default, mailers]
      threads: 5
      polling_interval: 0.1
      processes: 2

    - queues: [agents]
      threads: 2
      polling_interval: 1
      processes: 1
```

Now a long agent run can only ever occupy a thread in the agent pool. Your transactional jobs run on their own workers and never queue behind an AI task. The two workloads stop competing for the same threads.

## Throttle with worker count, not just concurrency limits

You have two different throttling needs, and they want two different tools.

| Throttling need | Right tool | Example setting | Why this tool |
|---|---|---|---|
| Global rate-limit and cost ceiling | Worker pool size | 2 threads on the `agents` queue |
Concurrency controls carry per-job overhead when the cap is above 1 |
| Per-account fairness (one run at a time) | `limits_concurrency to: 1` |
keyed on `account_id` |
Cheap at a limit of 1; extra runs wait in `blocked_executions` |

The first is a global cap: respect the provider's rate limit, and do not let a flood of runs hammer the model API or run up the bill all at once. The instinct is to reach for `limits_concurrency`

with a high limit, but Solid Queue's own docs steer you away from that. For throttling where the limit is meaningfully larger than 1, a small worker pool is the right lever, because concurrency controls carry real per-job overhead. You already have that lever from the previous section: two threads on the `agents`

queue means at most two agent runs in flight at once, full stop. Want five? Set five threads. The worker count is your global throttle.

The second need is fairness: one user should not be able to launch fifty concurrent runs and starve everyone else, and for many workflows a single user should only have one run going at a time. That is exactly what `limits_concurrency`

is for, with a limit of 1, keyed per account:

```
class AgentRunJob < ApplicationJob
  queue_as :agents

  limits_concurrency(
    to: 1,
    key: ->(agent_run_id) { "agent_run_account_#{AgentRun.find(agent_run_id).account_id}" },
    duration: 10.minutes
  )

  # ...
end
```

A second run for the same account is held in `blocked_executions`

and promoted to ready when the first finishes. The `duration`

is a failsafe, not a timeout: if a worker dies without releasing the semaphore, the lock expires after `duration`

so the account is not blocked forever. Set it comfortably above your longest expected run. If a run outlives `duration`

, the semaphore is cleaned up while the job keeps going, and a second run can start and overlap, which is the exact thing you were trying to prevent.

If duplicate runs should be dropped rather than queued, like a "refresh this summary" button a user can mash, switch the conflict behavior to discard:

``` php
limits_concurrency to: 1, key: ->(id) { ... }, duration: 10.minutes, on_conflict: :discard
```

Use the worker pool for the global ceiling, and `limits_concurrency`

for per-account fairness. They are not interchangeable.

## Retries cost money and do not replay

Here is the default that will hurt you:

```
retry_on StandardError, attempts: 5
```

On a normal job that is sensible. On an agent it is a liability. If the run already sent an email or charged a card on attempt one and then hit an error, you do not get a clean replay. You get the side effects again, the token spend again, and a retried run that may now take a different path and make different decisions than the one that failed. Retrying a non-deterministic, side-effecting, metered process is neither free nor idempotent unless you make it so.

Two things keep this safe. First, retry narrowly. Only retry the transient, transport-level failures that genuinely benefit from a second attempt, and discard the permanent ones instead of burning five attempts on an error that will never succeed:

```
class AgentRunJob < ApplicationJob
  queue_as :agents

  retry_on Faraday::TimeoutError, wait: :polynomially_longer, attempts: 3
  retry_on ProviderRateLimited,   wait: :polynomially_longer, attempts: 5

  discard_on AgentRun::InvalidInput
  discard_on ActiveRecord::RecordNotFound

  # ...
end
```

Second, make the side effects idempotent and the run resumable. Persist the interaction id and the steps you have already executed, so a retry continues from where it stopped instead of re-charging a card it already charged. The provider's `429`

should trigger a backoff, but the real throttle on how hard you hit the API is the worker count from earlier, not the retry policy. This is the same at-least-once delivery model that makes [recurring and cron jobs in Solid Queue](/rails/background-jobs/scheduling/2026/06/29/solid-queue-recurring-cron-jobs-guide/) demand idempotent bodies; an agent just raises the stakes because a double-run costs money.

## Deploys will interrupt long runs

Long jobs collide with deploys, and agents are long jobs. When Solid Queue shuts a worker down, in-flight jobs get a grace period - `shutdown_timeout`

, around 25 seconds by default - to finish before they are terminated. A two-minute agent run does not finish in 25 seconds, so a deploy in the middle of it kills the run partway through the loop.

You can raise `shutdown_timeout`

to give runs more room, within reason, but the durable fix is the resumability you already built for retries. If a run persists its interaction id and completed steps as it goes, an interrupted run is just another run to continue: re-enqueue it, pass the stored `previous_interaction_id`

, and it picks up rather than starting over. Treat interruption as normal, not exceptional, because on any actively deployed app it is.

## Human-in-the-loop without holding a thread hostage

Good agents pause before doing something irreversible. The write-tool rule - draft the action, let a human confirm, then execute - is covered in the agent-building posts. The question here is how to model that pause as a job, and the wrong answer is tempting: have the job sleep, polling for confirmation.

Do not do that. A sleeping job holds a worker thread for as long as the human takes, which could be minutes or hours, doing nothing but occupying a slot in your small agent pool. And the deploy from the last section will kill it anyway.

The Solid-Queue-native answer is to end the job and start a new one when the human acts. When the agent reaches a step that needs confirmation, persist the proposed action and the interaction id, move the run to a `waiting_for_confirmation`

state, and return. The job is done. No thread is held.

``` python
def perform(agent_run_id)
  agent_run = AgentRun.find(agent_run_id)

  result = AgentRunner.new(...).run(
    input: agent_run.input,
    previous_interaction_id: agent_run.previous_interaction_id
  )

  if result[:needs_confirmation]
    agent_run.update!(
      status: "waiting_for_confirmation",
      previous_interaction_id: result[:interaction_id],
      pending_action: result[:pending_action]
    )
    return
  end

  agent_run.update!(status: "completed", output: result[:output])
end
```

When the user confirms in the UI, the controller enqueues a continuation that resumes from the stored interaction id:

``` python
class ResumeAgentRunJob < ApplicationJob
  queue_as :agents

  def perform(agent_run_id)
    agent_run = AgentRun.find(agent_run_id)
    return unless agent_run.status == "waiting_for_confirmation"

    result = AgentRunner.new(...).run(
      input: "User confirmed the pending action.",
      previous_interaction_id: agent_run.previous_interaction_id
    )

    agent_run.update!(status: "completed", output: result[:output])
  end
end
```

The pause now costs zero worker time and survives any number of deploys, because between confirmation steps there is no running job at all, only a row in the database waiting for a human.

## Persist runs and track cost

There are two reasons to write every run and its token usage to your own tables, beyond whatever the provider stores. The first is debugging: an agent will eventually do something strange in production, and without the step log you are guessing about a probabilistic system. [Mission Control Jobs](/rails/background-jobs/monitoring/2026/01/14/mission-control-jobs-rails-production-monitoring/) gives you the queue-level view of which runs failed and retried, but the per-step trace has to come from your own tables. The second reason is money. Every run is a line item on your model bill, and if you cannot attribute spend to an account or a feature, you cannot see which customer or which workflow is quietly costing you. Capture the usage metadata each run returns and roll it up per account, so cost is a number you can query rather than a surprise at the end of the month.

## When you do not need any of this

If your "agent" is a single cheap model call - classify this ticket, extract these fields, one shot, no tool loop - skip the machinery. It is fast, it is cheap, retrying it is harmless, and a normal job on `default`

is completely fine. The queue isolation, concurrency limits, resumability, and cost tracking all exist to tame the slow-expensive-non-deterministic trio, and a small one-shot call simply does not have it.

One sharp edge to know before you lean hard on concurrency limits: under large spikes of enqueued jobs, Solid Queue can be slow to promote blocked executions back to ready, so do not put tight `limits_concurrency`

on a latency-critical path. For the per-account fairness case it is the right tool; for anything where a few seconds of delay matters, prefer the worker-count approach. If you are weighing the backend itself for this kind of workload, the [Solid Queue vs Sidekiq vs GoodJob comparison](/rails/background-jobs/architecture/2026/02/17/solid-queue-vs-sidekiq-vs-goodjob-rails/) covers where each one earns its keep.

The operational layer is the half nobody demos: the queue isolation that keeps a slow run from starving the app, the right throttle for your provider limits, retry semantics that do not double-charge a customer, runs that survive deploys, and per-account cost controls so the bill never surprises you. The agent itself is the easy half. Need a hand getting agents safe to run in production? I help Rails teams with the queue isolation, throttling, and cost controls that keep agent workloads from starving an app or blowing a budget.