# Show HN: Reconstruct distributed LLM training traces

> Source: <https://trace.vladsavinov.com/>
> Published: 2026-08-27 08:58:12+00:00

When we train large language models, there are a lot of systems challenges and different sharding schemes one can use. While there are many great resources on scaling LLMs out there ([https://huggingface.co/spaces/nanotron/ultrascale-playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook) or [https://jax-ml.github.io/scaling-book/](https://jax-ml.github.io/scaling-book/)), I felt like there was still a gap when it comes to visualising different forms of parallelism and building intuition around overlaps and execution order for a distributed training run

The idea is to make it easier to visualise FSDP/Tensor Parallel/Expert Parallel/Context parallel and reason about it - you can drag and drop compute kernels and collectives to create DDP/TP/FSDP/EP/CP traces based on real torchtitan profiles.

Comments URL: [https://news.ycombinator.com/item?id=49461808](https://news.ycombinator.com/item?id=49461808)

Points: 1

# Comments: 0