{"slug": "the-30-second-update-behind-more-bookings", "title": "The 30-second update behind more bookings", "summary": "Airbnb cut guest-embedding staleness from roughly two days to 10–30 seconds by moving inference into its event stream, and reports about one-third of a percent more uncancelled bookings in an online A/B test. Engineers Pengyu Hou and fourteen colleagues kept the same Transformer and ranker described in the JourneyFormer paper, accepted at KDD 2026, and added Push Mode and near-real-time Model Transform to Airbnb's open-source Chronon feature platform so each new view or search triggers a refresh. Offline ranking quality rose 1.67% in NDCG over the nightly batch, though Airbnb did not publish sample size, duration or confidence intervals for the online result.", "body_md": "## Experiment summary\n\n[Airbnb](https://www.airbnb.com/) stopped refreshing guest embeddings once a night and started refreshing them seconds after each view or search. In an online A/B test, it reports about one-third of a percent more uncancelled bookings.\n\n- Airbnb’s search ranker used an embedding of each guest’s history that could be almost two days old.\n- The team kept the same Transformer and ranker and changed only when inference ran: a new view or search now triggers it.\n- Updates usually land in 10 to 30 seconds. Offline ranking quality rose 1.67% over the nightly batch.\n- Airbnb reports roughly one-third of a percent more uncancelled bookings in an online A/B test, without publishing sample size or duration.\n- An earlier real-time attempt looked better offline and failed online because it kept showing guests what they had just seen.\n\nA guest browses a dozen Airbnb listings on Tuesday afternoon and searches again that evening. The ranking system knows years of that person’s bookings, reviews and cancellations. Until recently, it could still miss everything they did that afternoon.\n\nThe model was capable. The schedule was the problem. Airbnb’s Transformer compressed each guest’s history into an embedding, but a nightly batch job refreshed it, and with processing lag the result could sit almost two days behind the guest.\n\nSo Airbnb did not build a bigger model. It changed when the existing one ran.\n\n## The model knew yesterday’s trip\n\nThe [JourneyFormer paper](https://arxiv.org/html/2606.19108v1), accepted at KDD 2026, describes the model being kept fresh. It reads two sequences per guest: up to 80 infrequent events from the past seven years, such as bookings, reviews and cancellations, and up to 200 listing views from the past 21 days.\n\nThe Transformer turns those sequences into a stored guest embedding. At search time, the live ranker combines it with the query and candidate listings. Splitting expensive encoding from live ranking keeps search fast. It also means the ranker can only be as current as the last time the embedding was written.\n\nThat matters because Airbnb optimises for a booking that survives cancellation, not a click. Views are plentiful and noisy. A guest might open a listing because it is lovely, because it helps them compare, or because it is wrong in an interesting way. The most recent views are often the best hint about which.\n\n## How Airbnb moved inference into the event stream\n\nIn the [engineering write-up](https://airbnb.tech/ai-ml/the-guest-journey-updated-in-real-time-extending-airbnbs-sequence-recommender-with-chronon/), [Pengyu Hou](https://www.linkedin.com/in/pengyuhou/) and fourteen colleagues describe two additions to Chronon, Airbnb’s open-source feature platform.\n\nPush Mode sends a notification the moment a streaming job commits a new feature value. Near-real-time Model Transform lets that stream call a deployed model and write the output back to the feature store.\n\nIn practice, a page view or search enters the stream and is merged into the guest’s sequences. Push Mode flags the change, the same Transformer runs, and the new embedding lands in the same low-latency store the batch job used. The ranker reads the same feature from the same place. Only its age changed: Airbnb says the delay fell from about two days to under a minute, usually 10 to 30 seconds.\n\n## What the Airbnb real-time recommendation test found\n\nOffline, the fresh embeddings improved Normalized Discounted Cumulative Gain by 1.67% over the nightly baseline. NDCG checks whether booked listings rank higher. It does not tell you whether anyone booked more.\n\nThe online A/B test answered that. Airbnb reports about one-third of a percent more uncancelled bookings. At Airbnb’s volume a fraction of a percent is real money, and the metric sits past clicks and booking requests, where the business is actually measured.\n\nThe write-up does not give the number of guests, treatment share, dates, duration, baseline rate or confidence interval. The JourneyFormer paper does describe Airbnb’s usual practice: guests are split by a fixed hash of their ID, tests typically run three weeks, and a launch needs a key business metric at p below 0.05 with no guardrail breaches. That is the platform’s standard, not a readout of this test.\n\n## Faster signals can repeat the wrong answer\n\nAirbnb had tried real-time sequence inference before. The JourneyFormer team saw an offline gain above 5%, and it vanished online. The fresher model kept resurfacing listings the guest had just viewed. That is retargeting, not discovery. [Allegro hit a similar wall](https://industrycontents.com/allegro-recommendation-system/) when a sensible recommendation in the wrong place became commercially awkward.\n\nShowing an attractive house again can lift a ranking score without helping anyone choose. The later result suggests freshness paid off only once the system stopped treating recent activity as an instruction to repeat it. Any team testing fresh features should track result diversity and repeat impressions from day one, and keep a batch backfill in case the stream drops events or drifts from the training data.\n\n## What a smaller team can copy\n\nYou do not need Airbnb’s recommender to use the idea. Find one feature whose age you can see and that feeds a commercial decision: a lead score updated nightly, a product affinity refreshed after checkout, a churn signal computed weekly. The window matters as much as the refresh rate, as [Teads found with a seven-day browsing memory](https://industrycontents.com/teads-user-foundation-model-experiment/).\n\nKeep the model fixed. Give a random treatment group event-triggered updates while control stays on the batch schedule, and serve both through the same feature schema and endpoint. That isolates freshness from a model change and makes rollback simple.\n\nMeasure the outcome the business cares about. For Airbnb, uncancelled bookings. For a sales team, qualified pipeline or closed revenue. For a subscription product, activation or expansion. Then log stream delay, failed updates, inference volume and cost per extra outcome, because running a model after every event is not free.\n\nThe [Chronon repository](https://github.com/airbnb/chronon) is public, covering batch and streaming features, backfills, low-latency serving and online-offline consistency. Airbnb’s compute bill and engineering time are not, and for a smaller team those numbers decide whether a fraction of a percent is worth it.\n\n| Experiment detail | What Airbnb published | \n|---|---|\n| Problem | Guest embeddings could lag recent browsing by almost two days. | \n| Control | Nightly batch embedding pipeline. | \n| Treatment | Event-triggered updates via Chronon Push Mode and Near-real-time Model Transform. | \n| AI role | The existing Transformer encoder, run after each relevant event. | \n| Latency | Under one minute, usually 10 to 30 seconds. | \n| Offline result | NDCG up 1.67%. | \n| Online result | About one-third of a percent more uncancelled bookings. | \n| Not published | Sample, allocation, dates, duration, confidence interval, guardrails and cost. | \n\n## Sources and disclosure\n\n- Airbnb Engineering, [The guest journey, updated in real time](https://airbnb.tech/ai-ml/the-guest-journey-updated-in-real-time-extending-airbnbs-sequence-recommender-with-chronon/)\n- Airbnb researchers, [JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling](https://arxiv.org/html/2606.19108v1) , KDD 2026\n- Airbnb, [Chronon open-source repository](https://github.com/airbnb/chronon)\n\nAirbnb designed, ran and reported the experiment, and Industry Contents found no independent replication. The KDD paper describes the earlier model and Airbnb’s testing standards, not the missing details of this test.\n\n## FAQ\n\n## What did Airbnb change?\n\nIt kept its sequence model and search ranker, and moved guest-embedding updates from a nightly batch to a streaming path triggered by each view or search.\n\n## What result did Airbnb report?\n\nEmbeddings usually refresh in 10 to 30 seconds, offline NDCG rose 1.67%, and an online A/B test showed about one-third of a percent more uncancelled bookings.\n\n## Why did the earlier real-time attempt fail?\n\nIt improved offline ranking by more than 5% but kept resurfacing listings guests had just viewed, so the gain did not appear online.\n\n## What can a smaller team copy?\n\nHold the model fixed, randomise only the update schedule, serve both paths through the same feature, and measure a commercial outcome alongside repeat exposure and inference cost.\n\nCompare more measured experiments in the [Signal Index](https://industrycontents.com/ic-lab/growth-signal-index/), browse the [Growth Signal archive](https://industrycontents.com/category/growth-signal/), or use the benchmark tools in the [IC Lab](https://industrycontents.com/ic-lab/).", "url": "https://wpnews.pro/news/the-30-second-update-behind-more-bookings", "canonical_source": "https://industrycontents.com/airbnb-real-time-recommendation-test/?utm_source=rss&utm_medium=feed&utm_campaign=editorial", "published_at": "2026-11-20 13:00:00+00:00", "updated_at": "2026-11-20 13:14:09.903535+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure", "mlops", "ai-research"], "entities": ["Airbnb", "Chronon", "JourneyFormer", "Pengyu Hou", "KDD 2026"], "also_reported_by": [], "takeaway": "Re-running the same Transformer on each new view or search via a feature-platform push stream cut embedding staleness from about two days to 10–30 seconds, lifting offline NDCG 1.67% and uncancelled bookings about one-third of a percent.", "recommendation": {"advice": "Trigger embedding recomputation from the event stream instead of a nightly batch, and validate offline gains with an online A/B test on the business metric before shipping.", "playbook": "production-ops", "playbook_url": "https://wpnews.pro/playbooks/production-ops"}, "alternates": {"html": "https://wpnews.pro/news/the-30-second-update-behind-more-bookings", "markdown": "https://wpnews.pro/news/the-30-second-update-behind-more-bookings.md", "text": "https://wpnews.pro/news/the-30-second-update-behind-more-bookings.txt", "jsonld": "https://wpnews.pro/news/the-30-second-update-behind-more-bookings.jsonld"}}