{"slug": "show-hn-reconstruct-distributed-llm-training-traces", "title": "Show HN: Reconstruct distributed LLM training traces", "summary": "A developer has released a tool to reconstruct and visualize distributed LLM training traces, enabling users to drag and drop compute kernels and collectives to create DDP, TP, FSDP, EP, and CP traces based on real torchtitan profiles. The tool aims to fill a gap in visualizing different forms of parallelism and building intuition around overlaps and execution order for distributed training runs.", "body_md": "When we train large language models, there are a lot of systems challenges and different sharding schemes one can use. While there are many great resources on scaling LLMs out there ([https://huggingface.co/spaces/nanotron/ultrascale-playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook) or [https://jax-ml.github.io/scaling-book/](https://jax-ml.github.io/scaling-book/)), I felt like there was still a gap when it comes to visualising different forms of parallelism and building intuition around overlaps and execution order for a distributed training run\n\nThe idea is to make it easier to visualise FSDP/Tensor Parallel/Expert Parallel/Context parallel and reason about it - you can drag and drop compute kernels and collectives to create DDP/TP/FSDP/EP/CP traces based on real torchtitan profiles.\n\nComments URL: [https://news.ycombinator.com/item?id=49461808](https://news.ycombinator.com/item?id=49461808)\n\nPoints: 1\n\n# Comments: 0", "url": "https://wpnews.pro/news/show-hn-reconstruct-distributed-llm-training-traces", "canonical_source": "https://trace.vladsavinov.com/", "published_at": "2026-08-27 08:58:12+00:00", "updated_at": "2026-08-27 09:18:46.227908+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools", "machine-learning"], "entities": ["torchtitan", "Hugging Face", "JAX"], "alternates": {"html": "https://wpnews.pro/news/show-hn-reconstruct-distributed-llm-training-traces", "markdown": "https://wpnews.pro/news/show-hn-reconstruct-distributed-llm-training-traces.md", "text": "https://wpnews.pro/news/show-hn-reconstruct-distributed-llm-training-traces.txt", "jsonld": "https://wpnews.pro/news/show-hn-reconstruct-distributed-llm-training-traces.jsonld"}}