Correction for Listing 6.5 in Build a Reasoning Model From Scratch

Thanks everyone for all the kind words and feedback. Super happy that you are enjoying Build a Reasoning Model (From Scratch)!

Unfortunately, there’s a small typo in Listing 6.5 on page 198 (see the video below). The line torch.manual_seed(0)

should be torch.manual_seed(5) .

This is a tiny change, but it’s needed to reproduce the generated response in Listing 6.5 and the corresponding log-probability outputs later in Chapter 6. If you use 0, the code is of course still correct but the numeric results may be a bit different, which could be confusing. Anyway, it should be a small 1 character change :).

This will be fixed in the next printing.

I am sorry about the oversight, and I hope this note saves you some debugging time.

Source: lightly edited website version of my Substack note.

Read Next #

LLMs From Scratch Reaches 100,000 GitHub Stars Short note celebrating the LLMs-from-scratch repository passing 100,000 GitHub stars and summarizing its learning materials.

Kimi K3 Architecture Notes Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

A Few Notable Open-Weight Models This Week Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.