{"slug": "usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance", "title": "USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance", "summary": "ByteDance's USDB unified sequence storage system cut mean write latency by 85.8% and infrastructure cost by 42.1% after production migration, according to an arXiv paper submitted on 2 November 2026. USDB combines CascadeLSM time-based compaction, differentiated replicas sharing persistent files, partial dumps that reconstruct inference inputs, and a hybrid columnar format for projected reads, reusing maintenance results across ingestion, inference and training. In controlled comparisons against an Optimized RocksDB sequence baseline, USDB delivered faster long-history reads across projections and client loads, and aggregation pushdown sustained 8.5x the throughput of ordinary reads under a constrained receive budget.", "body_md": "# Computer Science > Databases\n\n  [Submitted on 2 Nov 2026]\n\n# Title:USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance\n\n[View PDF](https://arxiv.org/pdf/2611.02618)\n\n[HTML (experimental)](https://arxiv.org/html/2611.02618v1)\n\nAbstract:Recommendation models consume increasingly long, attribute-rich user histories, making small continuous updates and repeated large reads a central storage asymmetry. We present USDB, ByteDance's unified sequence storage practice for streaming and batch workloads. CascadeLSM time-based compaction stages materialize application-specific histories; differentiated replicas share persistent files; partial dumps reconstruct inference inputs from captured recent state and versioned history; and a hybrid columnar format accelerates projected reads. These designs reuse maintenance results across ingestion, inference, and training. Production migration records show 85.8% lower mean write latency, while normalized deployment accounting shows 42.1% lower infrastructure cost. Controlled comparisons with an Optimized RocksDB sequence baseline show faster long-history reads across projections and client loads. Aggregation pushdown sustains 8.5x the throughput of ordinary reads under a constrained receive budget. Production reconstruction and direct-file training results demonstrate how shared sequence maintenance supports fresh serving and repeated historical access.\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance", "canonical_source": "https://arxiv.org/abs/2611.02618", "published_at": "2026-11-03 16:40:02+00:00", "updated_at": "2026-11-03 16:53:23.814354+00:00", "lang": "en", "topics": ["ai-infrastructure", "machine-learning", "ai-research"], "entities": ["ByteDance", "USDB", "CascadeLSM", "RocksDB", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance", "markdown": "https://wpnews.pro/news/usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance.md", "text": "https://wpnews.pro/news/usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance.txt", "jsonld": "https://wpnews.pro/news/usdb-scaling-storage-for-next-generation-recommender-systems-at-bytedance.jsonld"}}