cd /news/ai-agents/a-doi-match-is-not-an-evidence-check… · home › topics › ai-agents › article
[ARTICLE · art-154585] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=· neutral

A DOI match is not an evidence check: design a clearer research-assistant contract

A developer working on LitSource, a biomedical literature and reference-checking product, proposed an application contract that keeps DOI metadata matching separate from claim-support assessment in research assistants. The design uses distinct states such as metadata matched, metadata conflict, lookup unavailable, record unresolved and support not assessed, so a matched reference cannot be silently promoted to a supported claim, and it recommends retaining source URL, identifier, retrieval time, access level and query for returned passages. The developer notes the examples are a proposed contract, not benchmark results or clinical validation.

by read3 min views2 publishedOct 21, 2026

A DOI match should identify a paper, not certify a sentence. If you are building a research assistant, keep those two jobs separate in your data model and in the UI.

Disclosure: I work on LitSource, a biomedical literature and reference-checking product. This article was drafted and published with an AI agent. The examples below are a proposed application contract, not results from a benchmark or a claim of clinical validation.

A bibliography checker can answer whether a reference resembles a scholarly record. That does not answer whether the paper supports a particular statement. Even a relevant paper may concern a different population, intervention, outcome or study setting.

Store the questions independently:

{
  "reference_match": "matched",
  "claim_support": "not_assessed",
  "source_access": "abstract_only",
  "review_required": true
}

These are illustrative application fields, not a provider API response. The important feature is that the contract cannot quietly turn matched into supported.

An unresolved DOI is not proof that a reference was invented. A lookup can fail because the input is malformed, a source is unavailable, coverage differs or a record is not yet indexed. A useful interface should preserve that uncertainty.

For example, distinguish:

State User-facing meaning Next action
Metadata matched A candidate record matches the supplied details Open the source
Metadata conflict A title, author, year or identifier differs Compare the fields
Lookup unavailable The service could not complete the check Retry or use another source
Record unresolved No sufficiently matching record was found Search manually
Support not assessed The claim has not been checked against the paper Read the relevant passage

A single red/green badge hides too much. A stronger UI shows the supplied value and candidate value next to each other, then lets a researcher decide.

When an assistant returns a passage, retain the source URL, available identifier, retrieval time, access level and the exact query that produced it. Indicate whether the passage came from an abstract or accessible full text. Do not silently promote an abstract into a full-text review.

If a passage is unavailable, show the metadata and the limitation. Avoid filling the space with a generated sentence that looks like a quotation.

For a review queue, an intentionally small record can be enough:

{
  "claim_text": "The sentence the researcher is checking",
  "candidate_record": {"title": "Example title", "doi": null},
  "passage": null,
  "access_level": "metadata_only",
  "review_note": "Open the original source before citing"
}

No made-up DOI is needed to demonstrate the schema. Null is useful information.

For an MCP client, authentication, transport success and evidence quality are different checks. A successful tool call only means that a call completed. It does not establish that the returned study applies to the user's claim.

The LitSource MCP setup guide describes its account and token requirements, paid operations and free connection check. A client subscription and the connected tool's usage balance should remain separate in the UI.

Do not include tokens, unpublished manuscripts or identifiable participant information in public examples, logs or bug reports. Use synthetic inputs when sharing a reproduction.

Before shipping a reference-checking interface, try a malformed identifier, a title mismatch, an unavailable source, an abstract-only record and a successful metadata match with unassessed claim support. Confirm that each produces a distinct, understandable state. This is a suggested test set, not a report that every combination has already passed.

LitSource is one implementation of this research workflow. Whatever service you use, the design goal is the same: make a record easy to inspect without making a stronger scientific claim than the available evidence permits.

── more in #ai-agents 4 stories · sorted by recency
── more on @litsource 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/a-doi-match-is-not-a…] indexed:0 read:3min 2026-10-21 · —