Netflix tests GenRec on 10% of traffic, reports 0.006% relative lift

Netflix engineers Ying Li, Arjun Rao and Shradha Sehgal tested GenRec, an LLM-backed ranker, on approximately 10% of traffic for four weeks and reported a statistically significant 0.006% relative improvement on its core online metric, according to an August 10, 2026 technical paper. The gain came while using about 40 times fewer labeled examples during GenRec's second training phase than Netflix's production model. The authors describe GenRec as an initial step toward an LLM-native recommendation stack, not a commercial product.

Netflix https://netflixtechblog.com/?ref=runtimewire engineers Ying Li https://www.linkedin.com/in/ying-li-94016850/?ref=runtimewire , Arjun Rao https://arjun.fyi/?ref=runtimewire and Shradha Sehgal https://www.linkedin.com/in/shradha-sehgal?ref=runtimewire tested GenRec, an LLM-backed ranker, against the streaming service's mature production recommender. Netflix tested GenRec on approximately 10% of traffic for four weeks and reported a statistically significant 0.006% relative improvement on its core online metric https://arxiv.org/pdf/2608.10257?ref=runtimewire . The result appeared in an August 10, 2026 technical paper https://arxiv.org/abs/2608.10257?ref=runtimewire , expanding on Netflix's July 30 engineering post https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire . Li, Rao and Sehgal achieved the gain while using about 40 times fewer labeled examples https://arxiv.org/pdf/2608.10257?ref=runtimewire during GenRec's second training phase than Netflix's production model. The comparison frames the experiment's tradeoff: a very small online gain against a heavily tuned baseline, paired with fewer frequently refreshed labels and a shorter serving context. The Netflix TechBlog post https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire lists Li, Rao and Sehgal as its authors. The technical paper https://arxiv.org/pdf/2608.10257?ref=runtimewire adds Rein Houthooft, Yaochen Zhu and Ashish Rastogi, with contributors from Netflix teams spanning member AI, its AI platform and serving, and product. Rao describes https://arjun.fyi/?ref=runtimewire his Netflix work as a mix of applied research and machine-learning engineering for ranking and page construction. A University of Illinois Urbana-Champaign profile https://siebelschool.illinois.edu/66486?ref=runtimewire describes Sehgal's graduate study and work involving machine-learning systems and LLM-based analogy mining. Li, Rao and Sehgal also co-authored a February 2026 paper on learned verbalization https://arxiv.org/abs/2602.20558?ref=runtimewire , the problem of turning raw interaction logs into language an LLM can use efficiently. GenRec carries that work into a full-catalog ranking system. The authors describe it as an initial step toward an LLM-native recommendation stack, rather than a commercial product for outside customers. The project converts member histories, title metadata and request context into text, then uses a Netflix-adapted foundation model and a catalog-aware scoring head to rank available titles. Netflix has not said that GenRec has replaced its recommender across the service, and the disclosed materials contain no external pricing or financing because GenRec is an internal engineering project. Moving recommendation into language Netflix's existing production systems https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire depend on thousands of hand-built features covering members, titles and interactions. Separate architectures handle sequence modeling, feature interactions and multiple objectives. Years of additions have made that machinery capable, while increasing the engineering required to introduce a content category or recommendation surface. That burden has grown as Netflix has expanded recommendation beyond movies and series to games, live programming and podcasts. Members generate hundreds of billions of interaction events https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire , including plays, viewing duration, explicit feedback, list additions and abandoned sessions. Netflix does not specify a period for that total, but the volume explains why replacing hand-built inputs with text still requires aggressive filtering before inference. GenRec changes the model's input. Member histories, title metadata and request context https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire are converted into natural-language or lightly structured descriptions. Those descriptions can include https://arxiv.org/pdf/2608.10257?ref=runtimewire viewing duration, explicit feedback, device, locale, time, subscription tenure and the surface where a recommendation will appear. The research team filters that history before it reaches the model. Long plays and positive ratings receive more detail https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire . Short or noisy interactions can be dropped. Repetitive behavior, including binge-watching sessions, can be compressed into a summary. Older activity receives less space than recent, higher-signal behavior. That choice turns prompt construction into a production engineering problem. Every additional token consumes GPU capacity and can raise latency. The full paper says Netflix reduced GenRec's context from about 5,000 tokens to roughly 1,700 https://arxiv.org/html/2608.10257?ref=runtimewire with negligible deterioration in its offline ranking metric. Because serving cost was approximately proportional to context length in this configuration, Netflix reported that cost fell to about one-third of the original level https://arxiv.org/html/2608.10257?ref=runtimewire . The model is trained in two phases https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire . The first adapts an open-source base model using proprietary Netflix data, teaching a shared foundation model about the catalog and member behavior. Netflix has not identified the base model. The second phase trains that foundation for ranking, using more frequently refreshed examples, recommendation objectives and reward signals. That separation reflects how quickly recommendation data ages. The Phase 1 foundation model is updated relatively infrequently, while Phase 2 is refreshed more often to track new content, popularity patterns and members' latest interests. Measured against a freshly trained Phase 1 model at the Phase 1 cutoff date, the second phase produced a roughly 35% to 50% gain in offline ranking metrics https://arxiv.org/html/2608.10257?ref=runtimewire . That comparison is distinct from GenRec's evaluation against Netflix's production ranker. The researchers also compared backbones on the order of 1 billion and 10 billion parameters https://arxiv.org/html/2608.10257?ref=runtimewire . The paper reports that offline mean reciprocal rank improved monotonically as the amount of Phase 2 training data increased for both model sizes. Netflix does not disclose the parameter count of the model used in the online experiment. A small online number with narrow limits In offline evaluation, GenRec improved mean reciprocal rank by about 1.6% relative to Netflix's production baseline https://arxiv.org/html/2608.10257?ref=runtimewire while using about 40 times fewer Phase 2 labeled examples. Mean reciprocal rank measures how high the first relevant result appears: a relevant title at position one receives more credit than one placed farther down the list. The 1.6% figure therefore describes ranking position in an offline test, rather than viewing, retention or revenue. Netflix then allocated approximately 10% of traffic to a four-week test on selected batch-computed recommendation surfaces. Netflix reported statistically significant gains in short-term and long-term measures https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire . The disclosed example was the 0.006% relative improvement https://arxiv.org/pdf/2608.10257?ref=runtimewire in its core online metric. The paper does not identify that online metric, its absolute baseline, the confidence interval or any effect on member retention, viewing or financial performance. Statistical significance indicates that Netflix's test detected an effect under its experimental design; it does not establish that the change had broad product or business impact. The relative percentage also cannot be converted into an absolute gain without the undisclosed baseline. The baseline is a ranker Netflix has tuned for years. Matching or modestly exceeding it with fewer frequently refreshed labels could help Netflix adapt ranking across movies, series, games, live programming and podcasts without building as much specialized machinery for each surface. The published test does not establish that GenRec will produce comparable results on real-time surfaces, in every country, across all member cohorts or at companies with different traffic and data profiles. Keeping an LLM inside the catalog A general-purpose language model presents several problems for recommendation. It can favor globally popular titles, invent titles that Netflix does not carry and ignore product rules governing how different content categories should be presented. Li, Rao, Sehgal and their collaborators added a catalog-aware scoring head https://arxiv.org/pdf/2608.10257?ref=runtimewire that can rank the available candidate set in a single forward pass https://arxiv.org/pdf/2608.10257?ref=runtimewire . The paper describes a pooled representation of member preferences and context that is combined with a learned embedding for each catalog item. Restricting the output to the catalog keeps GenRec from recommending nonexistent or unavailable content. The serving design also avoids autoregressive decoding. GenRec runs on Netflix's internal LLM infrastructure with vLLM in prefill-only mode https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire , consuming the member context and scoring candidates without generating recommendation text token by token. That reduces the inference burden of serving a decoder model when a member opens a product surface. Netflix uses reward-weighted training https://netflixtechblog.com/genrec-towards-llm-native-recommendation-at-netflix-f20be6f643e3?ref=runtimewire to steer ranking toward longer-term member satisfaction and business requirements. Other rewards rebalance exposure across movies, series, games, live programming and podcasts. The researchers tested reinforcement-learning methods https://arxiv.org/pdf/2608.10257?ref=runtimewire and reported additional preliminary gains over supervised fine-tuning. They selected the simpler weighted-loss approach because of training cost, operational stability and easier maintenance. The paper situates GenRec alongside industrial work https://arxiv.org/pdf/2608.10257?ref=runtimewire at Google, Spotify and Kuaishou on language-model backbones and generative retrieval for recommendation. Those projects address related ranking problems, although Netflix's comparison remains its own mature production system. GenRec's public results do not provide a cross-company benchmark. For smaller recommendation teams, the useful material lies in the engineering choices rather than the scale of Netflix's result: compress behavioral histories before inference, refresh ranking-specific training more often than the foundation model and constrain scores to a known catalog. Teams evaluating the design would still need to measure whether an LLM ranker beats a specialized model under their own latency, GPU, data and labeling constraints. Netflix's experiment offers no evidence that its serving economics transfer to a smaller operation. Li, Rao, Sehgal and their collaborators have shown that an LLM-backed ranker can compete with Netflix's incumbent model under a controlled serving design. Further evidence would require tests on real-time surfaces, a wider share of traffic or disclosed member outcomes. For now, the 0.006% relative online gain remains a narrow experimental result, while the larger contribution is a documented architecture for turning behavioral logs into catalog-constrained rankings.