cd /news/ai-agents/show-hn-agent-doctor-let-domain-expe… · home › topics › ai-agents › article
[ARTICLE · art-174129] src=theagentdoctor.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Show HN: Agent Doctor – let domain experts tune your agent without code changes

Agent Doctor launched as a Show HN tool that lets non-engineers such as doctors and lawyers override an AI agent's prompts, model and temperature from a dashboard and re-run the full pipeline on the customer's own servers, requiring only two lines of SDK code and one webhook handler with no code changes or redeploys. The tool hooks into LLM SDKs, vector databases and tool calls at the library level to capture every step, and developers review and ship the winning prompt with the next release.

key takeaway

Adding two lines of Agent Doctor SDK code and one webhook handler auto-captures every LLM call, vector lookup and tool call, letting domain experts override prompts, model and temperature from a dashboard and re-run the pipeline on your own servers without redeploying.

read5 min views3 publishedNov 22, 2026
Show HN: Agent Doctor – let domain experts tune your agent without code changes
Image: source

Two lines of SDK code capture every LLM call, vector lookup and tool call. Anyone on your team can then override a prompt, model or temperature from the dashboard and re-run it on your own servers, as many times as it takes. Ship the winner when it's proven.

Override any prompt and re-run the whole pipeline on your servers. #

A cardiologist with no coding background spots a wrong diagnosis, rewrites the faulty prompt from the dashboard, and re-runs the agent until the answer is right. The developer ships the proven prompt.

+ POSITIVE EXAMPLE:

Input: chest pressure, diaphoresis, jaw pain

Output: Unstable angina (ACS). Order troponin, 12-lead ECG.

- NEGATIVE EXAMPLE:

Input: chest pressure, diaphoresis, jaw pain

Output: General anxiety. Lifestyle changes. ← wrong"

unstable angina. Recommend immediate troponin levels, 12-lead ECG, and cardiology consult."

Developers build the pipeline and domain experts tune it. #

An engineer can wire up the agent, but they can't judge a sound medical diagnosis or a legally airtight clause. Agent Doctor gives the people who can a dashboard to iterate on prompts and parameters, while the developer keeps ownership of the code.

Builds the pipeline

Writes the agent code, connects to databases, sets up tools, deploys to production.

history = ehr_db.lookup(patient_id)

research = pubmed.search(symptoms)

diagnosis = llm.generate(...)

return treatment_plan(diagnosis)

Adds two lines & a webhook

Drops in the SDK and one webhook handler for re-runs. Back to building features.

agent_doctor.init(api_key="sk-...")

Reviews & ships

When the domain expert finds a winning prompt, the dev reviews it and ships it with the next release.

Sees every step visually

Opens the dashboard, sees the full pipeline trace. Every prompt, every output, every decision the agent made.

Tweaks prompts & parameters

Spots a bad prompt, rewrites it in plain English. Adjusts the temperature. Hits "Run". No code, no deploys.

  • "You are a board-certified cardiologist.

Prioritize differential diagnosis.

Cite relevant clinical guidelines."

Iterates until it's right

Compares runs side-by-side, uses Probe to debug, runs consistency checks. All from the dashboard.

Backend engineer Built the 6-node triage pipeline

rank_diagnoses → treatment_plan → safety_review

Cardiologist Iterates on prompts from the dashboard

Prompt: "Apply AHA guidelines for

cardiac symptom evaluation..."

Temp: 0.7 → 0.2

Your existing code. Unchanged. #

Two lines to capture. One webhook handler to re-run. Agent Doctor hooks into LLM SDKs, vector databases and tool calls at the library level, so your agent logic stays exactly as-is.

import agent_doctor
agent_doctor.init(api_key="sk-proj-...")


def handle_ticket(query, customer_id):
    intent = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": query}]
    )

    docs = pinecone_index.query(vector=embed(query), top_k=5)

    policy = check_refund_policy(customer_id)

    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "system", "content": f"Docs: {docs} Policy: {policy}"},
                  {"role": "user", "content": query}]
    )
    return response

def on_run(payload):
    with agent_doctor.run_context(run_id=payload.run_id, overrides=payload.overrides):
        result = handle_ticket(payload.query, payload.customer_id)
  1. 01The dashboard sends the override A new prompt, model or temperature for one node, posted to your webhook.
  2. 02Your servers run the agentrun_context injects the override at runtime. No code change, no redeploy, full access to your private data and tools.
  3. 03Only the trace comes back Inputs, outputs and timings for each step land in the dashboard to compare.

Two lines in, every step of your pipeline is in the dashboard.

Start free

Probe finds the broken node by asking you a few questions. #

Tell it what the agent should have said and what it said instead. It walks the trace with you, asking three or four plain-English questions, and names the exact step that caused the failure. No logs, no code, no debugging skills needed.

  • A chat, so anyone on the team can run it
  • Checks the pipeline by halves, so it needs only a few questions
  • Highlights the broken node in the trace
  • Hands the fix straight to the prompt optimizer

*"Customer requests refund for order #4821."*Does this routing look correct?

*"Order found. Status: delivered. Refund policy: eligible within 30 days."*Does this look correct?

Root cause found: response_composer

All upstream nodes are healthy. The response_composer node received correct refund eligibility data but generated a "call support" response instead of processing the refund.

Suggestion: The prompt lacks explicit instruction to confirm eligible refunds directly.

Open in Prompt Optimizer →

Consistency checks find the step that answers differently every time. #

Run your pipeline ten times with identical input. Per-node consistency scores show exactly which step is non-deterministic, and an AI report tells you why.

7/10 → Knowledge Base | 3/10 → Account Tools

MODERATE synthesize | Consistency: 0.72

STABLE validate | Consistency: 0.98

Root cause: Router prompt lacks explicit routing rules for password queries.

  • Per-node consistency scores and decision divergence
  • AI root cause report with a suggested fix
  • Apply the fix, re-test, and verify from the same screen

The optimizer writes, tests and ranks better prompts for you. #

Pick a node and define what "good" looks like. The optimizer generates prompt variants, runs them against your examples, and ranks the results. It starts with GEPA, a genetic strategy that evolves prompts across generations, with more strategies on the way.

  • Evolves prompts across generations and keeps the fittest
  • Rephrases, expands, compresses and recombines winning traits
  • Ranks every variant by its score on your examples, with token cost weighed in
  • Shows the winning prompt side by side with the original prompt

Why not just use a regular observability tool? #

Most LLM tracing platforms run experiments on their own cloud, so your agent can't reach your private databases, RAG, or internal tools during a test. They also keep prompt iteration locked behind code, leaving your domain experts out.

Common questions #

init(). It auto-instruments your LLM calls, vector DB lookups, tool executions, and API requests at the library level, with no decorators or refactors in your agent logic. To re-run from the dashboard, add one webhook handler that calls your agent inside run_context(). After that, every prompt change is a re-run, not a redeploy.

Test every agent fix. No code changes. No redeploys. #

Install the SDK, open the dashboard, and let the whole team work on the agent.

Built with love in Khunti, Kudlum, Jamshedpur, Bangalore, Delhi, India

── more in #ai-agents 4 stories · sorted by recency
── more on @agent doctor 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-agent-doctor…] indexed:0 read:5min 2026-11-22 · —