# Fix the Query Before the Reranker

> Source: <https://feniix-hq.net/blog/fix-the-query-before-the-reranker/>
> Published: 2026-11-20 00:00:00+00:00

**Recommendation:** Strip the chat-context header and keep only the user's latest message before sending a recall query to the reranker; trimming lifted MiniLM-L6 from 0.092 to 0.437 nDCG@10.
More in the playbook "Retrieval and RAG": <https://wpnews.pro/playbooks/rag-retrieval>

Article

# Fix the Query Before the Reranker

Trimming chat context from 40 Hindsight recall queries lifted the MiniLM-L6 reranker from 0.092 to 0.437 nDCG@10, about as much as switching to hs-rerank-1.

We run [Hindsight](https://github.com/vectorize-io/hindsight) 1.2.1 as the
memory of an AI assistant on a 16 GB M4 Mac mini. When recall quality
disappoints, the usual knobs are the cross-encoder reranker and how many
candidates it sees, and we measured both in
[Going Back to Hindsight’s Default Reranker](https://feniix-hq.net/blog/back-to-the-default-reranker/).
On our queries that carried recent chat, the text the assistant sent as
the query mattered as much as the reranker, and more than its cap.

Forty of our 100 evaluation queries begin with a chat-context header: the agent framework pastes the recent conversation in front of the user’s message. Cutting each of them down to the user’s latest message lifted MiniLM-L6 (the smaller reranker we ran before) from 0.092 to 0.437 nDCG@10 and hs-rerank-1 (Hindsight’s default) from 0.352 to 0.546.

That is about as much as switching from MiniLM-L6 to hs-rerank-1, and the two gains stack, though not in full: after trimming, hs-rerank-1’s lead over MiniLM-L6 is 0.110. Trimming also reduced how deep in the candidate list the reranker had to look, though not by as much as we expected.

Hindsight 1.2.1, cap 300. Blind graded relevance (Claude Opus for every grading round after the first; Claude Sonnet confirms), paired over the 40 queries. The trim rule and the pass bar were fixed before measuring.

To try both levers yourself, the
[reranker explainer](https://feniix-hq.net/demos/reranker/) lets you trim a query and move the
cap, first on twelve invented memories and then on all 100 queries.

## The chat header in recall queries

A recall query is written by the agent framework, not typed by a person.
When the assistant searches its memory while answering a chat message, the
framework sends Hindsight the recent conversation, with the user’s latest
message last. Context blocks are separated by a marker, and each chat line
starts with a header such as `#123 2026-11-18 20:44:32 UTC Ana (ana42):`.

An invented example, the same one the
[explainer](https://feniix-hq.net/demos/reranker/#toy) uses:

```
As sent:
#4101 2026-11-18 20:41:07 UTC Ana (ana42): the staging banner
broke again after the Friday deploy <ctx> #4102 2026-11-18
20:42:55 UTC Ben (benk): lunch is tacos today, and standup moved
to 10 <ctx> #4103 2026-11-18 20:44:32 UTC Ana (ana42): Which
database did we pick for the billing service?

Trimmed:
Which database did we pick for the billing service?
```

Forty of the 100 queries in our evaluation set look like this. In a week of live traffic (2026-11-12 to 11-19), 234 of 619 recall queries did (38%), from 99 distinct queries. A cross-encoder reads the query and each memory together, so with the header in front, most of what it reads as “the query” is conversation the user did not ask about.

We fixed the trim rule and checked it on all 40 queries before measuring. Our version recognized a wrapped query by the fixed text the framework puts at its start; the version below looks for the marker instead. The two give the same output except on a query that contains the marker without that opening, which ours left alone. The marker here is a placeholder:

``` python
import re

# The marker your agent framework puts between blocks of chat context.
# Ours is a short tag; replace it with whatever yours writes.
CONTEXT_SEPARATOR = "<ctx>"

# A chat-history line prefix such as
# "#123 2026-11-18 20:44:32 UTC Ana (ana42): "
CHAT_HEADER = re.compile(
    r"^#\d+\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}"
    r"\s+\S+\s+[^:()]*\([^)]*\):\s*"
)

def trim(query: str) -> str:
    """Reduce a chat-wrapped recall query to the user's latest message."""
    # Plain queries pass through untouched.
    if CONTEXT_SEPARATOR not in query:
        return query
    # The newest block is the last one.
    latest = query.rsplit(CONTEXT_SEPARATOR, 1)[-1].strip()
    # Drop who said it and when.
    return CHAT_HEADER.sub("", latest).strip()
```

On the 40 queries the median length went from 190.5 to 67 characters, and
none came out empty. The rule is specific to one framework’s format; the
point is to keep the user’s message and drop the rest. The explainer shows
[the same trim on a toy query](https://feniix-hq.net/demos/reranker/?form=trimmed#toy), with the
scores hs-rerank-1 gives each memory before and after.

Why the rule looks like this:

- It keeps the latest message because that is what the user asked; the earlier turns are context around it.
- It drops the `#id date name:` tag because a cross-encoder reads the query
and each memory as one input, so names, handles and timestamps become text
it tries to match.
- It is a fixed rule rather than a model rewrite of the query: that costs nothing at query time, gives the same output every time, and could be checked on all 40 queries before measuring. We did not test a model rewrite.
- It runs before recall rather than only before the reranker, because retrieval reads the query too; lever 1 measures the two together.

## Lever 1: the query

The chart shows nDCG@10 for the 40 headed queries, as sent and trimmed, for each model at cap 300. nDCG@10 scores the first ten results by their graded relevance, weighting the top positions most; 1.0 is the best possible order.

Both models improved, MiniLM-L6 the most: its share of queries with an answer at rank 1 went from 0.075 to 0.375. The result held with a second grader (Claude Sonnet: +0.395 and +0.215) and at cap 200 (+0.351 and +0.202).

Trimmed MiniLM-L6 scored 0.085 above untrimmed hs-rerank-1, with an interval of [−0.011, +0.179]. The interval includes zero, so we cannot separate the two: on these queries, cleaning the query was worth about as much as the model switch.

Trimming also changes what retrieval returns, so the gain is retrieval and reranking together; we did not separate them. Of 560 query–memory pairs in the new top 10s that had never been graded, 558 came from the trimmed queries’ candidate lists. We graded 559 of them (the missing one counts as not relevant) against the original query’s need, which is the user’s message either way.

These scores also use a larger set of graded memories than the default-reranker article’s, so compare them within this chart.

## Lever 2: the candidate cap

Hindsight reranks only the top N of the fused candidate list, where N is
`HINDSIGHT_API_RERANKER_MAX_CANDIDATES`. A smaller cap is faster, and it
costs quality only when relevant memories sit below N. On our pools,
that happened more often for headed queries than for plain ones.

At cap 300, hs-rerank-1 took 4.1 of each headed query’s top 10 from below rank 100 of the fused list. For the same queries trimmed it took 3.1, and for plain queries 1.4. Counting only relevant ones, the figures were 0.93, 0.72 and 0.38 per query. Trimming moved headed queries toward plain ones, not all the way. These counts were not part of a planned test.

The planned tests agree. Cutting the cap from 300 to 100 cost headed
queries as sent 0.063 nDCG@10, which failed our bar. On clean queries (the
60 plain ones plus the 40 trimmed) it cost 0.006, which passed with both
graders. On the 40 trimmed queries alone it cost 0.019, and the graders
disagreed on whether that passed. So cut the header first, then test the
cap on your own queries;
[Going Back to Hindsight’s Default Reranker](https://feniix-hq.net/blog/back-to-the-default-reranker/#cost-the-cpu-and-the-apple-gpu)
has the cost side.

## Lever 3: the model

Trimming does not replace a better reranker. hs-rerank-1 still scored above MiniLM-L6 at cap 300 on trimmed queries, where it was better on 24 queries and worse on 9, and on plain ones:

| Queries | hs-rerank-1 minus MiniLM-L6 | 95% interval | 
|---|---|---|
| 40 headed, trimmed | +0.110 | [+0.044, +0.173] | 
| 60 plain (first test, all 100 queries, split after the fact) | +0.090 | [+0.041, +0.146] | 

MiniLM-L6 also got less out of depth: of the items each model pulled from below rank 100 into its top 10, 25% were relevant for hs-rerank-1 and 10% for MiniLM-L6 (first test). On headed queries the two levers together took nDCG@10 from 0.092 (MiniLM-L6, as sent) to 0.546 (hs-rerank-1, trimmed).

## Long prompts

Recall queries can also be long task prompts that end in a question. We tested six synthetic ones, cut to their final question, with hs-rerank-1 on the CPU, one at a time, on a warm machine:

|  | As sent | Trimmed | 
|---|---|---|
| Length | 3,125–4,649 characters | about 80 characters | 
| Median rerank time | 46.3 s | 11.1 s | 
| Peak memory | 3.82 GiB | 2.34 GiB | 

None of our evaluation queries looked like that; a week of live traffic held one distinct long query (24 of 619 recalls), and it fitted neither trim pattern.

## Check your own queries

1. Sample a week of recall queries and count how many carry context the user did not type. Ours was 38%.
2. Freeze each query’s candidate list once and rerank it offline for every setting, so the settings differ only in what you changed.
3. Grade the top 10s blind, on a small scale such as 0 to 2, and use a second grader on a sample.
4. Compare settings paired by query, and write the pass bar down before you look.

If Hindsight calls a TEI-compatible reranker, the recording proxy in
[mps-rerank-server](https://github.com/feniix/mps-rerank-server) saves the
exact requests it sends, which covers step 1 and the query text for step 2.

## Limits

- All of this is one assistant’s traffic: 100 evaluation queries, 40 of them headed.
- The evaluation set kept only queries under 300 characters, so longer chat-wrapped queries were not tested.
- The graders are language models (the first round’s model was not recorded; every later round used Claude Opus), and no human checked their labels.
- The trim rule fits one framework’s format.
- The long-prompt numbers come from synthetic prompts on the CPU, and the depth counts in lever 2 were computed after the fact.
- The trim was measured offline, on replayed queries; it is not deployed in the assistant yet, so there is no live result.
