Route by Precedent, Not by Retraining

A retrieval-based k-nearest-neighbours (kNN) classifier that keeps each customer's labelled history in a searchable index can replace per-customer trained models for routing and categorisation tasks, according to the article. The approach uses one shared general-purpose embedding model to represent records while each customer maintains its own index of past decisions, so a new ticket is routed by the labels of its k=10 nearest historical precedents weighted by similarity. The article argues this avoids the permanent retraining burden of supervised classifiers, which suffer from cold-start customers, lopsided category distributions, categories added after training, and quiet accuracy decay from data and concept drift.

Most systems that sort, route, or categorise work arrive at the same question eventually: “Which bucket does this belong in?” A support request needs to reach the right team. An invoice needs a cost centre. A document needs a queue. At first, this sounds like a textbook supervised classification problem: collect examples, train a classifier, and let the model assign the label. That approach works well. But it carries an assumption that is easy to miss: the set of labels is relatively stable. In practice, it often isn’t. Different customers use different terminology. Teams reorganise. Categories are renamed, split, merged, or retired. New products introduce new kinds of requests. The system is expected to adapt, but the underlying model only knows the world it saw when it was trained. That is where a simple shift in perspective helps: instead of asking a model to memorise a customer’s past, keep the past available and search it when a new decision arrives. A classical ML classifier is usually built from labelled examples: this type of ticket goes to that team; this kind of transaction belongs to that cost centre. For one stable problem, that is a sensible arrangement. But in a multi-customer product, the work quickly multiplies. Each customer has its own labels, language, and distribution of work. In effect, you are not maintaining one classification problem. You are maintaining many of them, each changing independently. The cost is not only the first training run. A new customer has little or no labelled history, and there is no model until there is. The categories are lopsided: a few carry most of the volume while dozens have a handful of examples each — and those rare ones are exactly where a classifier is weakest. A model trained today cannot predict a category added tomorrow. And as the language of incoming work shifts, and categories change meaning, accuracy decays quietly between training runs data and concept drift . None of these issues is unusual on its own. Together, they turn retraining into a permanent operational burden: data preparation, evaluation, versioning, deployment, monitoring, and repeat. The useful question isn’t “How can we retrain faster?” It’s “Do we need to train at all?” People make decisions this way all the time. When an unusual request lands in a shared inbox, an experienced operator doesn’t reach for a rulebook. They remember — or look up — a similar request from last month and see how it was handled. The earlier case provides context, a likely answer, and often a reason for that answer. The same principle can be applied to automated classification. Rather than compressing a customer’s labelled history into a custom-trained model, keep that history searchable: For example, if eight of the ten closest historical support tickets were sent to the billing team, that is strong evidence that the new ticket should go there too — especially if those examples are very similar. Under the hood, this is a k-nearest-neighbours kNN classifier: one shared, general-purpose embedding model does the representing, and each customer gets its own index of history. It’s an old idea. What makes it practical now is that embeddings capture similarity in meaning, not just shared keywords. python from collections import defaultdictdef predict record, index, k=10 : neighbours = index.search embed record , k=k scores = defaultdict float for neighbour in neighbours: scores neighbour.label += neighbour.similarity label = max scores, key=scores.get confidence = scores label / sum scores.values return label, confidence, neighbours The important part is not the few lines of code. It is the decision model: a prediction is based on comparable precedents, and those precedents remain visible. A retrieval-based approach changes the shape of the problem. A customer becomes predictable as soon as there is enough labelled history to search. There is no separate training cycle to wait for. If a new category is introduced, it becomes available once relevant examples are labelled and added to the index. As terminology changes, newer examples start steering decisions. It shortens the delay between a change in the real world and a change in the system’s behaviour. It also creates a simpler operating model. Instead of maintaining a specialised classifier for every customer, you run one shared embedding model and one serving path. The customer-specific knowledge lives in that customer’s history. And unlike most model predictions, the result comes with its own evidence: “We routed this here because it closely resembled these earlier cases.” That matters in any setting where people need to trust, review, or override an automated decision. There is one obvious limitation: precedent only helps when precedent exists. A new customer with almost no historical records cannot benefit much from retrieval. That gap needs a bridge. One that works well is an LLM asked to classify into a small set of broad categories — a coarse call it can make reliably with no customer history at all. Simple rules the customer already uses, or human review while examples accumulate, can fill the gap too. The goal is not to pretend the system knows more than it does. It is to make the uncertainty explicit, use a temporary fallback, and gradually hand decisions over to retrieval as trustworthy history grows. Retrieval is not a universal replacement for trained classifiers. It reproduces bad labels just as faithfully as good ones. Old examples can point toward categories that no longer exist, so predictions should still be checked against the labels currently in use. And records that sound similar are not always supposed to be treated the same way; a general representation of meaning does not automatically understand the particular boundaries that matter to a business. There are also practical trade-offs. Searching a growing collection of past records has a cost, and maintaining that collection well matters. A conventional classifier is the better choice when one customer has a stable label set, abundant high-quality data, and clear boundaries between categories. The point is not that classifiers are obsolete. It is that retraining should not be the reflex. When categories change faster than you can retrain, stop trying to freeze them in place. Keep the history. Search it. Let precedent help make the next decision. The catch, of course, is that history becomes part of the model. And if that history is incomplete, stale, or unreliable, the system will inherit those flaws too. Route by Precedent, Not by Retraining https://pub.towardsai.net/route-by-precedent-not-by-retraining-928310d6e468 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.