1BRC in Zig! My program ran in 1.12 seconds

A Zig developer reported completing the One Billion Row Challenge (1BRC) in Zig with a program that ran in 1.12 seconds, publishing the implementation write-up on mattymo.dev. The developer said a MapReduce approach beat a single hash map guarded by a mutex, which "resulted in poor performance" and bugs, and commenters noted that Zig 0.16 and 0.17 disabled auto-vectorization due to an LLVM 21 miscompilation not fixed until LLVM 23, so the program should get faster on Zig 0.18 when auto-vectorization is reenabled.

Hi Zig community I recently did the one billion challenge 1BRC with Zig and wanted to share my blog post. I’ve been using Zig in all my recent projects and I am amazed at how quickly I have begun to feel productive. I think it’s a testament to the simplicity and focus on code readability. Check out my post here : One Billion Row Challenge https://mattymo.dev/posts/one-billion-row-challenge Also, I am super open to feedback. Thanks for reading 9 Likes Would the recent talk about using SIMD for the Zig tokenizer give you any ideas for SIMD searching of semicolons? tholmes https://ziggit.dev/u/tholmes 3 Before I went with a MapReduce implementation for the multi-threaded approach, I tried using one hash map with a mutex. However, this resulted in poor performance and it also caused a lot bugs skill issue I know . This roughly matches my experience with mutex. Recently I wrote a program to convert DDS textures to VTF Valve Texture Format . I was able to use multithreading to optimise reading the DDS textures from the input directory since I was decoding DXT1 and DXT5 with code I wrote myself / stole off the Internet , but to write the output VTF files I was depending on VTFLib https://nemstools.github.io/pages/VTFLib-Download.html , which is not threadsafe. What this meant was that in order to safely write a VTF, I had to use a mutex to halt all other threads while the external library did its work, creating an unnecessary performance bottleneck. My solution was to have the multi-threaded DDS parser code populate an ArrayList of structures that contained pixels, width, height and format of the output VTF, then do a single-threaded loop over that ArrayList to make calls to VTFLib and actually save the files. Just so you know Zig 0.16 and 0.17 have disabled auto vectorization because of a miscompilation in LLVM 21 that didn’t got fixed until LLVM 23. So your program should get faster on 0.18 when auto vectorization is reenabled. I’m also interested in your changes to the std reader