We're building a universal product catalog, which means we process data for tens of millions of physical products every day. In simplest terms, we take raw unstructured data from products and transform it to fit our internal product taxonomy. A multi-stage pipeline performs data extraction, classification, and named-entity resolution to get structured product info and establish relationships across brands, variants, and canonical product offers across different retailers.
Whilst much of the public discourse today focuses on the frontier models, we care much more about the cheapest models that we can run at enormous scale.
The majority of our work is high quantity, low intelligence tasks. That makes Jev an interesting candidate. Jev is TypeSafe AI's decision model: instead of generating free-form text, it answers structured questions, such as choosing an option from a supplied list, and returns a confidence estimate. We tested it against our existing pipeline, starting with product categorization.
Using Jev to categorize products #
Category taxonomies take the form of a tree with subcategories getting more specific the further down you go. This often leads to hierarchical classification as the standard approach: find the root node(s) that match best, then which of their children match best, and repeat recursively until none of the children are a valid match or you've reached leaf nodes.
Take “Nike Sportswear Club Fleece.” We may find node “Clothing & Accessories” matches best at the root level, then its child “Clothing,” then its child “Outerwear,” and finally “Coats & Jackets.”
TypeSafe's cookbook pattern walked through doing this top-down walk with beam search, a similar pattern as described above but additionally maintaining K best paths as you traverse. We replicated this as our first attempt at integrating Jev, and while it looked promising initially, it broke down in some crucial cases early on.
Consider the case below, where our product is a “Youth Bicycle.” In both Shopify and Google's taxonomy, this sits at Sporting Goods > Outdoor Recreation > Cycling > Bicycles. It takes 3 layers before you start getting any semblance of a bicycle with “Cycling,” whereas other root nodes like “Vehicles & Parts” score much higher. Across the entire tree, the path we need falls out of our beam quickly, and we incorrectly match our “Youth Bicycle” to a “Motor Vehicle.”
Why don't we do a Jev call per node, since it's so cheap? Then, we certainly would've found our match with a direct comparison to Bicycles at a score of 0.94. The answer is that Jev may not be as cheap as you think. With thousands of nodes in the full category taxonomy, this would have increased our costs ~2,000× compared to our current production setup. Scoring all 6,000 categories reached 98.0% accuracy, at $1.08 per product.
After a few more experiments, we did find a solution with Jev that worked incredibly well. Our existing pipeline uses an embedding space of all categories, over which we query with product info. Recall@1 was low (sub-40%), but Recall@30 was high (above 99%). So, we return the top 30 results and use Gemma 4 31B to select the best option out of the results, resulting in 81.9% accuracy.
We swapped Jev in as a reranker on that last step. Once we had our top 30 results, we performed a Choice call over the returned categories. Here, we were able to achieve 97.2% accuracy with similar costs at $0.0005 per product.
TLDR: Jev improved our product categorization accuracy from 81.9% to 97.2% at roughly the same cost when we used it to choose from 30 prefiltered categories. Searching the full taxonomy with Jev was too expensive.
What we learned #
For those coming from frontier models like Astra and Fable, using Jev probably feels close to free. But, if Jev can solve tasks that you've been using frontier models on, you've likely been overpaying for inference for a while, and models like GLM-5.3-Flash or DeepSeek V4.1 Flash could have performed similarly for pennies on the dollar. For those benchmarking Jev against cheaper open source models, it's closer to a 50% discount than many multiples of savings. At Channel3, we use batched inference that makes the costs nearly identical to Jev.
Where Jev worked
- Picking from a short list. The more you can pre-filter options passed to Jev, the better it will perform.
- Zero-shot decisions. It performed well with no labeled examples or fine-tuning needed.
- Replacing small, cheap decisions. Our best results came when keeping our existing setup and swapping Jev in for Gemma.
- Replacing expensive models for simple decisions. For many use cases, Jev can be the “dumbest” model you need, and will save costs if you're using current OpenAI or Anthropic models.
Where it didn't
- Selection over a large search space. Sequential calls to narrow the space degraded performance. Jev struggles to “think ahead.”
- Improving costs on already-cheap inference. Jev does not provide meaningful cost savings compared to batched flash models like GLM-5.3-Flash.
Newsletter
Get new posts in your inbox #
Field notes from the rapidly changing world of AI commerce