# Show HN: Agent Doctor – let domain experts tune your agent without code changes

> Source: <https://theagentdoctor.com/>
> Published: 2026-11-22 10:17:08+00:00

**Key takeaway:** Adding two lines of Agent Doctor SDK code and one webhook handler auto-captures every LLM call, vector lookup and tool call, letting domain experts override prompts, model and temperature from a dashboard and re-run the pipeline on your own servers without redeploying.

# Diagnose and fix your AI agent. *No code changes. No redeploys.*

Two lines of SDK code capture every LLM call, vector lookup and tool call. Anyone on your team can then **override a prompt, model or temperature** from the dashboard and re-run it on your own servers, as many times as it takes. Ship the winner when it's proven.

## Override any prompt and re-run the whole pipeline on your servers.

A cardiologist with no coding background spots a wrong diagnosis, rewrites the faulty prompt from the dashboard, and re-runs the agent until the answer is right. The developer ships the proven prompt.

**+ POSITIVE EXAMPLE:**

Input: chest pressure, diaphoresis, jaw pain

Output: Unstable angina (ACS). Order troponin, 12-lead ECG.

**- NEGATIVE EXAMPLE:**

Input: chest pressure, diaphoresis, jaw pain

Output: General anxiety. Lifestyle changes. ← wrong"

**unstable angina**. Recommend immediate troponin levels, 12-lead ECG, and cardiology consult."

## Developers build the pipeline and domain experts tune it.

An engineer can wire up the agent, but they can't judge a sound medical diagnosis or a legally airtight clause. Agent Doctor gives the people who can a dashboard to iterate on prompts and parameters, while the developer keeps ownership of the code.

### Builds the pipeline

Writes the agent code, connects to databases, sets up tools, deploys to production.

history = ehr_db.lookup(patient_id)

research = pubmed.search(symptoms)

diagnosis = llm.generate(...)

return treatment_plan(diagnosis)

### Adds two lines & a webhook

Drops in the SDK and one webhook handler for re-runs. Back to building features.

agent_doctor.init(api_key="sk-...")

### Reviews & ships

When the domain expert finds a winning prompt, the dev reviews it and ships it with the next release.

### Sees every step visually

Opens the dashboard, sees the full pipeline trace. Every prompt, every output, every decision the agent made.

### Tweaks prompts & parameters

Spots a bad prompt, rewrites it in plain English. Adjusts the temperature. Hits "Run". No code, no deploys.

+ "You are a board-certified cardiologist.

Prioritize differential diagnosis.

Cite relevant clinical guidelines."

### Iterates until it's right

Compares runs side-by-side, uses Probe to debug, runs consistency checks. All from the dashboard.

**Backend engineer** Built the 6-node triage pipeline

rank_diagnoses → treatment_plan → safety_review

**Cardiologist** Iterates on prompts from the dashboard

Prompt: "Apply AHA guidelines for

cardiac symptom evaluation..."

Temp: 0.7 → 0.2

## Your existing code. Unchanged.

Two lines to capture. One webhook handler to re-run. Agent Doctor hooks into LLM SDKs, vector databases and tool calls at the library level, so your agent logic stays exactly as-is.

``` python
# Add these two lines, that's it
import agent_doctor
agent_doctor.init(api_key="sk-proj-...")

# Your existing agent logic, unchanged
# LLM calls, vector DB queries and tool calls are auto-captured

def handle_ticket(query, customer_id):
    # LLM call → auto-captured
    intent = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": query}]
    )

    # Vector DB lookup → auto-captured
    docs = pinecone_index.query(vector=embed(query), top_k=5)

    # Tool call → auto-captured
    policy = check_refund_policy(customer_id)

    # LLM call → auto-captured
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "system", "content": f"Docs: {docs} Policy: {policy}"},
                  {"role": "user", "content": query}]
    )
    return response

# Your webhook handler. payload is the run Agent Doctor sends.
def on_run(payload):
    with agent_doctor.run_context(run_id=payload.run_id, overrides=payload.overrides):
        result = handle_ticket(payload.query, payload.customer_id)
        # Every step auto-captured. Overrides auto-injected. Done.
```

1. 01**The dashboard sends the override** A new prompt, model or temperature for one node, posted to your webhook.
2. 02**Your servers run the agent**`run_context` injects the override at runtime. No code change, no redeploy, full access to your private data and tools.
3. 03**Only the trace comes back** Inputs, outputs and timings for each step land in the dashboard to compare.

Two lines in, every step of your pipeline is in the dashboard.

[Start free](https://theagentdoctor.com/signup)

## Probe finds the broken node by asking you a few questions.

Tell it what the agent should have said and what it said instead. It walks the trace with you, asking three or four plain-English questions, and names the exact step that caused the failure. No logs, no code, no debugging skills needed.

- A chat, so anyone on the team can run it
- Checks the pipeline by halves, so it needs only a few questions
- Highlights the broken node in the trace
- Hands the fix straight to the prompt optimizer

*"Customer requests refund for order #4821."*Does this routing look correct?

*"Order found. Status: delivered. Refund policy: eligible within 30 days."*Does this look correct?

**Root cause found:** response_composer

All upstream nodes are healthy. The response_composer node received correct refund eligibility data but generated a "call support" response instead of processing the refund.

**Suggestion:** The prompt lacks explicit instruction to confirm eligible refunds directly.

Open in Prompt Optimizer →

## Consistency checks find the step that answers differently every time.

Run your pipeline ten times with identical input. Per-node consistency scores show exactly which step is non-deterministic, and an AI report tells you why.

7/10 → Knowledge Base | 3/10 → Account Tools

MODERATE synthesize | Consistency: 0.72

STABLE validate | Consistency: 0.98

**Root cause:** Router prompt lacks explicit routing rules for password queries.

- Per-node consistency scores and decision divergence
- AI root cause report with a suggested fix
- Apply the fix, re-test, and verify from the same screen

## The optimizer writes, tests and ranks better prompts for you.

Pick a node and define what "good" looks like. The optimizer generates prompt variants, runs them against your examples, and ranks the results. It starts with GEPA, a genetic strategy that evolves prompts across generations, with more strategies on the way.

- Evolves prompts across generations and keeps the fittest
- Rephrases, expands, compresses and recombines winning traits
- Ranks every variant by its score on your examples, with token cost weighed in
- Shows the winning prompt side by side with the original prompt

## Why not just use a regular observability tool?

Most LLM tracing platforms run experiments on their own cloud, so your agent can't reach your private databases, RAG, or internal tools during a test. They also keep prompt iteration locked behind code, leaving your domain experts out.

## Common questions

`init()`. It auto-instruments your LLM calls, vector DB lookups, tool executions, and API requests at the library level, with no decorators or refactors in your agent logic. To re-run from the dashboard, add one webhook handler that calls your agent inside `run_context()`. After that, every prompt change is a re-run, not a redeploy.
## Test every agent fix. *No code changes. No redeploys.*

Install the SDK, open the dashboard, and let the whole team work on the agent.

Built with love in Khunti, Kudlum, Jamshedpur, Bangalore, Delhi, India
