{"slug": "reduce-large-sorted-set-memory-with-a-packed-b-tree", "title": "Reduce large sorted set memory with a packed B+ tree", "summary": "Redis maintainer antirez proposed a draft pull request (#15635) that reimplements large sorted sets with a packed B+ tree, replacing the skiplist and dictionary to cut memory usage by 45–60% and speed up many operations by up to 70%, while keeping RDB/AOF formats unchanged. The PR, generated with AI assistance, is flagged as high risk because it alters core zset storage and hot paths across persistence, replication, modules, and defrag; Cursor Bugbot's automated review flagged a medium-severity issue where module iterator ranks can go stale after deletions or insertions.", "body_md": "# Reduce large sorted set memory with a packed B+ tree - #15635\n\n[antirez](/antirez)wants to merge 1 commit into\n\n## Conversation\n\nHi, this is a draft implementation that reimplements large sorted sets by replacing the skiplist and general-purpose dictionary with one packed B+ tree (score ordered) and a compact member index, that is, a specialized hash table.\n\nNo changes to RDB / AOF. Memory drops between 45 - 60% in many common use cases. Many operations are now faster, sometimes *much* faster (70% faster). ZSCAN can be slower, but not much slower.\n\nPlease tell me if you like this direction, I'll do more QA and refine the PR more.\n\n**AI Disclaimer** This PR was generated with massive AI help.\n\nNote\n\n**High Risk**\n\nCore zset storage and hot paths change across persistence, replication, modules, and defrag; regressions would affect data integrity, memory, and latency at scale despite extensive QA described in the PR.\n\n**Overview**\n\n**Large sorted sets** no longer default to skiplist + dictionary once they exceed listpack limits. They use a new ** OBJ_ENCODING_BTREE** (\n\n`btree`\n\nin `OBJECT ENCODING`\n\n): a score-ordered B+ tree with a compact open-addressing member index (hash tag + leaf id), packed scores in leaves, and **external allocation for members ≥320 bytes** so one huge member does not force a whole-key skiplist fallback.\n\n** zset_btree.c** is wired into the server build;\n\n**routes create/convert/add/del/rank/range/set-ops through the btree API, with skiplist retained for explicit conversion and temporary bulk-command paths.**\n\n`t_zset.c`\n\n**RDB** still writes\n\n`RDB_TYPE_ZSET_2`\n\n; loads large sets via **into btree (or listpack) instead of building skiplist first.**\n\n`zsetAdd`\n\n**AOF rewrite**,\n\n**ZSCAN**,\n\n**active defrag**,\n\n**fork dismiss**,\n\n**modules**(range/scan iterators),\n\n**geo**,\n\n**sort**, and\n\n**DEBUG DIGEST** gain btree branches.\n\nA large ** ZSET_MEMSAVING_ALT_REPORT.md** documents design, benchmarks (~45–60% memory on cited workloads), review fixes (ZSCAN cursor/reply, scan completeness), and follow-ups (direct bulk B+ build, compaction). Observable command results and on-disk types stay compatible; encoding,\n\n`MEMORY USAGE`\n\n, ZSCAN order/cursors, and internal scan behavior can differ within SCAN’s contract.Reviewed by Cursor Bugbot for commit\n\n`5b1e9dd`. Bugbot is set up for automated code reviews on this repo. Configure here.\n\n**reviewed**\n\n[cursor](/apps/cursor)BotAug 13, 2026\n\n###\n**\n**[cursor](/apps/cursor)\nBot\nleft a comment\n\n**left a comment**\n\n[cursor](/apps/cursor)Bot\n\nThere was a problem hiding this comment.\n\nCursor Bugbot has reviewed your changes using high effort and found 1 potential issue.\n\nReviewed by Cursor Bugbot for commit\n\n`5b1e9dd`. Configure here.\n\n[src/module.c](/redis/redis/pull/15635/files/5b1e9dd012b2959093351d6c92160430730b0347#diff-6109c354d7e009093f811238069b581bcb9bdbfc638d7d089814031776801632)\n\n| if (!valid) goto end; | ||\n| } | ||\n| key->u.zset.current = (void *)(uintptr_t)(rank + 1); | ||\n| return 1; |\n\nThere was a problem hiding this comment.\n\n### Module iterator ranks go stale\n\n**Medium Severity**\n\nB+ tree module range iteration stores a numeric rank in `current`\n\n. After a module callback deletes or inserts members with a lower rank, that saved rank still looks valid but now names a different element. `RM_ZsetRangeCurrentElement`\n\ncan therefore return the wrong member, and `RM_ZsetRangeNext`\n\n/ `RM_ZsetRangePrev`\n\ncan skip survivors. Out-of-range ranks end cleanly, but in-range shifted ranks do not.\n\n## Additional Locations (2)\n\nReviewed by Cursor Bugbot for commit\n\n`5b1e9dd`. Configure here.\n\n[Learn more about bidirectional Unicode characters](https://github.co/hiddenchars)", "url": "https://wpnews.pro/news/reduce-large-sorted-set-memory-with-a-packed-b-tree", "canonical_source": "https://github.com/redis/redis/pull/15635", "published_at": "2026-08-13 11:21:22+00:00", "updated_at": "2026-08-13 11:42:00.382385+00:00", "lang": "en", "topics": ["ai-tools", "ai-research"], "entities": ["antirez", "Redis", "Cursor Bugbot", "OBJ_ENCODING_BTREE", "zset_btree.c", "t_zset.c", "RDB_TYPE_ZSET_2"], "alternates": {"html": "https://wpnews.pro/news/reduce-large-sorted-set-memory-with-a-packed-b-tree", "markdown": "https://wpnews.pro/news/reduce-large-sorted-set-memory-with-a-packed-b-tree.md", "text": "https://wpnews.pro/news/reduce-large-sorted-set-memory-with-a-packed-b-tree.txt", "jsonld": "https://wpnews.pro/news/reduce-large-sorted-set-memory-with-a-packed-b-tree.jsonld"}}