Pinterest Replaces Journey Clustering with 4B Qwen3 LLM, Reports Improved Engagement

Pinterest replaced its multi-stage keyword clustering pipeline for user-journey inference with a fine-tuned 4-billion-parameter Qwen3 model, reporting a 1.1% improvement in email click-through and 1.3% higher push opens in initial A/B tests in the US and Canada. The model reads up to 150 chronological user actions across a 360-day window and generates ranked journey names, with Pinterest fine-tuning open-weight Qwen3 models from 0.6B to 8B parameters using synthetic training data and LoRA, then serving the 4B model on NVIDIA Dynamo and vLLM workers on L40S GPUs. Pinterest said the shift also produced statistically significant engagement and retention gains over the clustering-based system.

Pinterest has moved <https://medium.com/pinterest-engineering/from-activity-to-intent-generating-user-journeys-with-llms-9652b9417626 https://medium.com/pinterest-engineering/from-activity-to-intent-generating-user-journeys-with-llms-9652b9417626 its user-journey inference system from a multi-stage clustering pipeline to a fine-tuned 4-billion-parameter language model, reporting a 1.1% improvement in email click-through and 1.3% higher push opens. Rather than clustering keywords into interests, the model examines users' activity histories to identify longer-term goals and feeds those journeys into Pinterest's existing recommendation infrastructure. Pinterest defines a "journey" as a goal that connects interests, intent, and context across multiple sessions. Someone renovating a kitchen, for example, might search for cabinet designs, save countertop ideas, and later explore backsplash tiles or lighting. These interactions collectively suggest a coherent project rather than several unrelated interests. "User Journey" - Source: Pinterest The company's previous system extracted keywords from user activity, grouped them using embeddings and hierarchical clustering, named the resulting clusters, and ranked them with additional models and diversity heuristics. Although effective compared with earlier interest-based notifications, the approach could fragment related activities into separate journeys, produce generic labels, or mistake incidental engagement for sustained intent. The new system instead treats journey inference as a generation task. A fine-tuned LLM reads a chronological history of up to 150 user actions across a 360-day window, alongside a compact user profile, and generates a ranked set of journey names. Search and save actions carry more weight than incidental clicks, while the prompt instructs the model to consider repeated engagement, recency, and specificity. "Journey Inference" - Source: Pinterest The consolidation moved much of the previous system's ranking and filtering behavior into the prompt. Instructions discourage unrelated interests from being merged, suppress one-off activity and noisy automatically generated tags, and allow an empty result where there is insufficient evidence of a meaningful journey. Pinterest treats changes to these instructions as production launches rather than simple prompt edits. To build the system without an extensive human-labelled dataset, Pinterest used frontier models to generate synthetic training examples from its user activity histories. These examples were used to fine-tune smaller open-weight Qwen3 models through supervised fine-tuning and low-rank adaptation LoRA . The team tested models ranging from 0.6B to 8B parameters, selecting the 4B model for its balance of inference quality and serving throughput. Running the resulting model at Pinterest's scale presented a separate engineering challenge. The company deployed NVIDIA Dynamo in front of vLLM workers running on L40S GPUs, using continuous batching, PagedAttention, and prefix-cache-aware routing to improve inference efficiency. Journey Inference Architecture - Source: Pinterest However, one of the largest throughput improvements came from avoiding unnecessary inference altogether. The system tracks when each user's journeys were last generated and skips users processed within the preceding couple of days. Pinterest evaluates journey quality through side-by-side comparisons, a separate LLM judge assessing relevance and support from user activity, and online experiments. Initial A/B testing in the US and Canada produced the reported 1.1% email click-through and 1.3% push-open improvements over the clustering-based system, alongside statistically significant engagement and retention gains. The approach has parallels with Netflix's work <https://netflixtechblog.com/foundation-model-for-personalized-recommendation-1a0bd8e02d39 https://netflixtechblog.com/foundation-model-for-personalized-recommendation-1a0bd8e02d39 on foundation models for personalised recommendations. Netflix has described learning from long sequences of member interactions to create representations that can be reused by downstream recommendation models. While Netflix focuses on reusable preference representations and Pinterest generates explicit journey descriptions, both approaches use longer-term behavioural histories to enrich existing personalisation infrastructure rather than replace it entirely.