# Your MCP Server Returns the Source. Does the Answer Still Cite It?

> Source: <https://dev.to/ayanarshad02/your-mcp-server-returns-the-source-does-the-answer-still-cite-it-26n8>
> Published: 2026-10-17 07:26:58+00:00

*A benchmark for what happens to citations after a retrieval server hands its results to an AI agent it doesn't control.*

**Part 1 of 6 in the series *Citations at the MCP Boundary*.** This is where the series starts. The parts after this one go deeper into each finding; the full list is at the end of this post.

Here's a situation that's becoming normal very quickly.

A company turns its documentation into a search tool. It exposes that tool over MCP (the Model Context Protocol), so any agent can use it: Claude, ChatGPT, a coding assistant in someone's IDE, an agent a customer built with LangGraph or the OpenAI Agents SDK. The tool does its job well. For every query it returns the most relevant chunks of documentation, and each chunk comes with its source URL:

```
{
  "source_url": "https://fastapi.tiangolo.com/tutorial/body-updates/",
  "content": "# Body - Updates ..."
}
```

Then the agent takes over. It reads the chunks, decides what matters and writes the final answer.

The retrieval server never sees that answer. So the provider can be sure the right source went *in*, but has no idea whether a citation comes *out*.

For a documentation assistant that's a real problem. An answer about your API that's correct but uncited can't be checked, and the user can't click through to the page that explains the edge cases. An answer that cites the wrong page is worse: it looks verified when it isn't.

I couldn't find anyone measuring this, so I built a benchmark for it.

When a RAG knowledge base is consumed by an AI agent over MCP, instead of a single retrieve-then-answer call, do the citations survive?

The obvious hypothesis was that agents lose citations. An agent loops: it searches, reads, maybe searches again, and then writes. The sources from the first call end up far back in the context by the time the answer is written. It seemed natural that they'd get lost along the way.

That hypothesis turned out to be mostly wrong, and the way it was wrong is more useful than the hypothesis itself.

I built a complete, reproducible copy of this boundary:

`search_fastapi_knowledge_sources`, returning exactly
`{source_url, content}` over streamable HTTP. It mirrors the contract of a real hosted MCP
retrieval product, down to the naming pattern.`[url]` right after each claim):
Measuring "does it cite" turned out to need more care than I expected. An answer can be full of URLs and still be badly attributed. It can cite a real page that doesn't support the sentence, or cite a plausible FastAPI URL the agent never actually retrieved (models know what FastAPI URLs look like).

So **CSR (Citation Survival Rate)** requires every citation to pass two checks:

**CSR** = of the correct sources the agent actually retrieved, the fraction that end up in the final answer, cited on a claim the source genuinely supports.

Notice the condition: *the sources the agent actually retrieved*. If the agent never found the right page, that's a retrieval failure, and CSR leaves it out. CSR only measures what happens after the right source has crossed the boundary.

Here's what the metric said on the first pass:

| Pipeline (gpt-4o-mini) | CSR | 95% CI | 
|---|---|---|
| Single-shot RAG | **0.618** | [0.528, 0.708] | 
| LangGraph agent | **0.614** | [0.524, 0.705] | 
| OpenAI Agents SDK agent | **0.323** | [0.237, 0.414] | 

The LangGraph agent matched single-shot RAG (a difference of +0.000 question by question), so an agent loop by itself didn't cost citations. The OpenAI Agents SDK agent seemed to lose about half: in **66 of its 127 answers there was no citation in the `[url]` format the prompt asked for.**

I took that at face value for longer than I should have. When I finally read those 66 answers one by one, **most of them did cite their sources**:

`[source](https://...)`
`[source: https://...]`
My scorer only recognised the requested `[url]` format, so it counted every one of those Markdown links as a missing citation. I fixed the definition (written down before re-scoring: `[url]`, Markdown links and the other bracket format all count) and re-scored everything:

| Pipeline (gpt-4o-mini) | Only the requested format counted | Any citation format counted | 
|---|---|---|
| Single-shot RAG | 0.618 | 0.618 | 
| LangGraph agent | 0.614 | 0.643 | 
| OpenAI Agents SDK agent | 0.323 | **0.470** | 

Counted in any format, the SDK's remaining gap to single-shot on that run was −0.16 under the original scorer (significant) but −0.10 under a gpt-4o judge (not significant). One run couldn't settle it, so I ran every setup three times. A smaller loss held in every run and under both

scorers: **−0.12** (original scorer) and **−0.09** (gpt-4o judge), pooled, against about −0.30 counting only the requested format. So the drop is **about 60% format and 40% a real, smaller loss of citations.** At temperature 0 no loss is detectable.

The bigger and more stable effect is simpler:

**The OpenAI Agents SDK, on its default API, ignored the requested citation format in 51 of 127 answers (40%), and 53 and 46 in the two repeat runs. Single-shot RAG did that 0 times; the
LangGraph agent 8 times.**

That matters more than it sounds. Anything downstream that reads citations in a fixed format (a docs widget, a "view sources" panel, a guardrail, an eval like mine) sees nothing at all. The citation is gone *as far as the system is concerned*, even though a human reading the text could find it.

Part 2 has the full investigation. The short version: I ran five controlled experiments on the SDK agent, changing one thing at a time, with the plan written down *before* running anything:

`source_url` intact, and 98 of the 118 Markdown links point to pages the agent retrieved. The sources make it across the boundary.`[S3fa21c]`), ask for `[ID]` citations, and map IDs back to URLs after the answer is written. On my self-hosted mirror, wrong-format answers fell from about 50 to about 5 per run, the SDK agent's CSR rose by 0.10–0.13, and its gap to single-shot closed. (Part 6 has the numbers and the caveats, including that no human has audited those citations yet.)
A benchmark is only as good as its scorer, so I spent as long checking the measurement as running the experiment:

And the limits, stated plainly: one corpus, two models, three runs per gpt-4o-mini setup (one for the Claude runs), small human checks, and a self-hosted mirror of an MCP server rather than anyone's production system.

**Next:** [Part 2: One Agent Framework Seemed to Lose Half Its Citations. Most Were Just Reformatted.](//./02-one-framework-seemed-to-lose-half-its-citations.md)

*A new part is published every week.*

**Repo, traces and every number:** [csr-mcp-benchmark](//github.com/AyanArshad02/csr-mcp-benchmark)

*I'm an AI engineer who builds production RAG and agent systems and the evals that prove they work. I'm looking for a remote AI engineering role at a US or EU startup. If your team is building with RAG, agents or MCP, I'd love to talk.*