{"slug": "optimizing-memory-usage-in-a-markdown-parser", "title": "Optimizing memory usage in a markdown parser", "summary": "A developer optimizing a C++ port of the markdown-rs parser reduced AST node size from 232 bytes to 16 bytes, cutting memory usage by up to 75.2% across benchmark shapes. The optimization involved arena allocation, string growth, struct field reordering, and packing booleans, with the largest improvement in the entities benchmark dropping from 660.2 KB to 163.8 KB.", "body_md": "I’m porting gpui-component (a Rust UI component library built on GPUI) to C++ as gpui-cpp. By which I mean: my friend Claude does the porting, I’m just directing.\n\nIt uses markdown-rs (a CommonMark + GFM parser) markdown parser so I ported it too.\n\nThen I optimized it.\n\nThis post describes what I did with the intention of teaching other how to optimize C++ code.\n\nThe starting point\n\nThere are 2 kinds of markdown parser:\n\nthose that stream nodes as they parse\n\nthose that build an AST in memory\n\nmarkdown-rs builds an AST. The game is about minimizing the size of AST node.\n\nIn Rust there are various kinds of nodes, the largest being 152 bytes.\n\nClaude generated a single Node struct of 232 bytes.\n\nI got it down to 16 bytes.\n\nHere’s the initial Node struct, before optimizations:\n\nWhere the 232 went: 8 string fields at 16 bytes each (a char* plus a length), two growable vectors at 24 bytes each (children and table alignments), a 24-byte unist Position (line, column and offset at each end), six bools one to a byte, and the padding all of that dragged in.\n\nEvery node in the tree pays for every field, whichever kind it is. A Text node uses one string field and nothing else.\n\nArena allocator\n\nIt’s important that all allocations are done in an arena.\n\nNodes in a parse tree all have the same lifetime which makes it a perfect use for an arena: a bump allocator that can only grow. The only way to free memory is to reset the arena.\n\nThis is different than calling malloc() to allocate each node individually and then having to call free().\n\nIt makes it easy to measure memory usage: check the arena size after parsing.\n\nIt also allows optimization tricks like compressing pointers.\n\nHow I measured\n\nbun cmd/bench.ts markdown parses 64 KB of markdown in four shapes and reports the arena bytes the parse allocated:\n\nprose — paragraphs, emphasis, links\n\nnested lists — deep blockquotes and lists\n\ngfm tables — tables all the way down\n\nentities — text that is mostly &-style character references\n\nThe number is the whole arena: nodes, the tokenizer’s event list, and the strings. Not just sizeof(Node) × node count.\n\nWe also measure parsing time to make sure we don’t trade size for speed.\n\nSome strings had to grow. Arena allocator doesn’t provide freeing or reallocation. You can only allocate new strings, which wastes memory by leaving dead copies of the string we were appending to.\n\nWe can grow the last allocated string and that’s what this change does. Luckily, most appends were done to the last string.\n\nArenaStrAppend checks whether the string ends exactly where the arena’s next allocation would begin. If it does, the new bytes are pushed straight onto it and nothing is copied.\n\nDecoding HTML entities (e.g. &) broke that optimization by doing an allocation before appending to the string.\n\nWe switched to decoding entities into a 4-byte stack buffer which enabled optimized append.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n1285.9 KB\n\n1285.9 KB\n\n+0.0%\n\n-21.9%\n\nnested lists\n\n1067.9 KB\n\n867.5 KB\n\n729.2 KB\n\n-15.9%\n\n-31.7%\n\ngfm tables\n\n2926.0 KB\n\n2269.7 KB\n\n2269.7 KB\n\n+0.0%\n\n-22.4%\n\nentities\n\n660.2 KB\n\n626.2 KB\n\n163.8 KB\n\n-73.8%\n\n-75.2%\n\n3. Re-order struct fields, pack the bools (5c0ce6e)\n\nUnless told to pack the layout of the struct, C++ compilers align struct fields to the size of the largest primitive type. If you sandwich a bool between 2 uint64_t values, the bool will occupy 8 bytes (sizeof(uint64_t)) instead of 1 byte as it should.\n\nOur Node had such wasted space due to padding. My friend Claude was careless.\n\nA simple fix is to re-arrange fields, putting the largest first.\n\nWe also had six bool field which we packed into a uint8_t flags field.\n\nResult: 168 → 144 bytes, with no padding at all.\n\nWe’re beating Rust version now.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n1285.9 KB\n\n1150.9 KB\n\n-10.5%\n\n-30.1%\n\nnested lists\n\n1067.9 KB\n\n729.2 KB\n\n654.0 KB\n\n-10.3%\n\n-38.8%\n\ngfm tables\n\n2926.0 KB\n\n2269.7 KB\n\n2023.6 KB\n\n-10.8%\n\n-30.8%\n\nentities\n\n660.2 KB\n\n163.8 KB\n\n151.0 KB\n\n-7.8%\n\n-77.1%\n\nFree bytes: same fields, same code, different order.\n\nWe compress pointer for all objects allocated in the arena, like we compressed a pointer to the string.\n\nArenaVec<Node*> children held 8-byte addresses; ArenaPtr<T> is a 4-byte offset into the arena’s position space, resolved by ArenaAtOffset. Zero is null, which costs nothing because no allocation ever lands at offset zero.\n\nThe Node itself doesn’t change size — a vector handle is the same three words whatever it holds — so all of the saving is in the child arrays.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n1150.9 KB\n\n1091.9 KB\n\n-5.1%\n\n-33.7%\n\nnested lists\n\n1067.9 KB\n\n654.0 KB\n\n611.6 KB\n\n-6.5%\n\n-42.7%\n\ngfm tables\n\n2926.0 KB\n\n2023.6 KB\n\n1866.9 KB\n\n-7.7%\n\n-36.2%\n\nentities\n\n660.2 KB\n\n151.0 KB\n\n144.9 KB\n\n-4.0%\n\n-78.1%\n\nThese shapes rank by children-per-node rather than by node count, which is why tables moved most.\n\nArenaStr was an offset and a length in 8 bytes. Now it’s the offset alone — 4 bytes — and the length is varint-encoded at the beginning of the string data:\n\n```\n[varint len][string bytes][NUL]\n```\n\nThere are many varint encoding schemes. This one is for unsigned number and codes number < 128 as a single byte.\n\nMost strings are below that threshold, so they use a single byte for the varint length, saving roughly 3 bytes per string.\n\nNode shrinks from 144 → 112 bytes.\n\nCaveat: An offset-and-length string can point at a slice of another string, and a length-prefixed one can’t. We weren’t doing it so it doesn’t apply here.\n\nSome nodes have children that were stored as a growable vector. Empty vector was 24 bytes in the node.\n\nWe replaced it with a ring of compressed pointers: the parent names its last child, each child names the next one, and the last child wraps back to the first.\n\nWe use a ring and not just a linked list because appending is the only thing the parser does to a child list. A single linked list requires walking the list to find the end, while a ring does not.\n\nSaving: 96 → 80 bytes.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n828.0 KB\n\n619.0 KB\n\n-25.2%\n\n-62.4%\n\nnested lists\n\n1067.9 KB\n\n462.2 KB\n\n308.8 KB\n\n-33.2%\n\n-71.1%\n\ngfm tables\n\n2926.0 KB\n\n1379.6 KB\n\n898.7 KB\n\n-34.9%\n\n-69.3%\n\nentities\n\n660.2 KB\n\n120.0 KB\n\n98.9 KB\n\n-17.6%\n\n-85.0%\n\nCaveat: accessing a child by index would require a walk through the ring, so indexing in a loop would be quadratic. In our code we only ask for the first or the last.\n\nFor tables we were storing column alignments in a separate vector on every node, even though only Table nodes have them. Another 24 bytes per node.\n\nWe switched to a compressed pointer which points to an optimized representation of the column alignments.\n\nThere are four alignments (left, right, center, none), so a column needs 2 bits:\n\n```\n[varint count][2 bits a column, four to a byte]\n```\n\nThe whole list is known when the table is entered, so it’s counted, allocated once and filled. For an 8-column table that’s 3 bytes in the arena and a 4-byte offset in the node.\n\nSaving: 80 → 60 bytes.\n\nWe saved more than the 20 bytes because with the last pointer-holding member gone alignof(Node) fell from 8 to 4.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n619.0 KB\n\n519.3 KB\n\n-16.1%\n\n-68.5%\n\nnested lists\n\n1067.9 KB\n\n308.8 KB\n\n256.3 KB\n\n-17.0%\n\n-76.0%\n\ngfm tables\n\n2926.0 KB\n\n898.7 KB\n\n710.3 KB\n\n-21.0%\n\n-75.7%\n\nentities\n\n660.2 KB\n\n98.9 KB\n\n89.2 KB\n\n-9.8%\n\n-86.5%\n\nThe block is pushed byte-aligned rather than through the general allocator, which rounds to 8 and would have handed back exactly what the varint saved.\n\nWe had 8 strings that were not all used by all nodes.\n\nInstead of figuring out how many strings we need at most, I created a linked list of strings in the arena. They are different than regular strings in that they carry a 4 byte compressed pointer to the next string within the arena and the kind of the strings.\n\n```\n[u32 next][u8 kind][varint len][len bytes][NUL]\n```\n\nWe can add as many kinds of strings as we need but we only pay for used strings + 5 byte per-string overhead.\n\nSome nodes don’t have any strings.\n\nNew records go on the head, so storing is O(1), and the walk that finds a kind is at most 8 long and is almost always 1 or 0. In-place growth still works, because a record being the newest thing in the arena is the same condition it always was.\n\nAt this point I decided that I didn’t need the position so I removed it. Other markdown parsers don’t carry it around so it doesn’t seem very useful.\n\nI reduced overhead of perKind by converting it to a record in the string list from step 12 — varint-encoded, under its own kind byte.\n\nA List, Heading or Table pays ~8 bytes for it; every other node pays nothing, where a field cost 4 bytes on all of them.\n\nSavings: 24 → 16 bytes.\n\nFor safety arena allocator aligns allocations to 8 bytes but a 16 bytes Node can be allocated at 4 bytes, which we did.\n\nThis reduces wasted space between allocations.\n\nshape\n\nstart\n\nbefore\n\nafter\n\nvs before\n\nvs start\n\nprose\n\n1646.1 KB\n\n321.2 KB\n\n272.0 KB\n\n-15.3%\n\n-83.5%\n\nnested lists\n\n1067.9 KB\n\n136.5 KB\n\n110.2 KB\n\n-19.3%\n\n-89.7%\n\ngfm tables\n\n2926.0 KB\n\n341.9 KB\n\n250.5 KB\n\n-26.7%\n\n-91.4%\n\nentities\n\n660.2 KB\n\n70.4 KB\n\n65.6 KB\n\n-6.8%\n\n-90.1%\n\nEnd results\n\nThe results are pretty dramatic:\n\nsizeof(Node)\n\nprose\n\nnested\n\ntables\n\nentities\n\nstart\n\n232\n\n1646.1 KB\n\n1067.9 KB\n\n2926.0 KB\n\n660.2 KB\n\nend\n\n16\n\n272.0 KB\n\n110.2 KB\n\n250.5 KB\n\n65.6 KB\n\n-93%\n\n-83.5%\n\n-89.7%\n\n-91.4%\n\n-90.1%\n\nA parse of 64 KB of prose cost 25.7× the source in arena bytes. It costs 4.2× now. The entities shape went from 10.3× to 1.02×.\n\nThe speed was unchanged. Fastest of 3 runs:\n\nprose 8.47 → 8.22 ms\n\nnested 9.45 → 9.26 ms\n\ntables 12.88 → 12.92 ms\n\nentities 5.90 → 5.85 ms\n\nThose are within margin of error.\n\nThe phase of building the tree got a measurable speed up: 0.397 → 0.302 ms, about 24% faster.\n\nThis is from allocating less and touching fewer cache lines.\n\nThis is not visible on micro benchmarks, but using less memory will slightly speed up the rest of the application.\n\nLessons learned\n\nArranging struct fields by size is good. It costs literally nothing.\n\nPointer compression is good. 8 bytes become 4 bytes and the cost of converting back and forth is negligible, as Google shown in their v8 blog post and is re-inforced by our benchmarks\n\nVarint-encoding is good. Most strings are short so varint encoding can save 3 bytes per string on average.\n\nMoving rare fields out of line is good. The way we reduced 8 strings into an out-of-line list. Only pays off if savings is bigger than the cost of additional metadata.\n\nsizeof only drops when the saving crosses an alignment boundary. Two of our changes didn’t reduce size of Node struct but it paid off in later optimizations.\n\nThe allocator’s alignment is part of sizeof. A 28-byte struct from an 8-aligned bump allocator is 32 bytes.\n\nWe need benchmarks. You can’t improve what you can’t measure. Our benchmarks measured both memory usage and speed, to ensure we didn’t regress speed to save memory.", "url": "https://wpnews.pro/news/optimizing-memory-usage-in-a-markdown-parser", "canonical_source": "https://blog.kowalczyk.info/a-n8wf/optimizing-memory-use-in-markdown-parser.html", "published_at": "2026-08-23 11:51:43+00:00", "updated_at": "2026-08-23 12:12:45.257956+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["gpui-component", "gpui-cpp", "markdown-rs", "Claude"], "alternates": {"html": "https://wpnews.pro/news/optimizing-memory-usage-in-a-markdown-parser", "markdown": "https://wpnews.pro/news/optimizing-memory-usage-in-a-markdown-parser.md", "text": "https://wpnews.pro/news/optimizing-memory-usage-in-a-markdown-parser.txt", "jsonld": "https://wpnews.pro/news/optimizing-memory-usage-in-a-markdown-parser.jsonld"}}