{"slug": "tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas", "title": "Tracing causal structure in LLM-generated text: a different lens on the Dallas circuit", "summary": "A new approach to mechanistic interpretability traces causal structure in LLM-generated text rather than internal activations, producing attribution graphs that resemble reasoning trajectories. The method, applied to Qwen3-1.7B using gradient attribution and sparse pruning, requires almost no semantic knowledge and could help study information propagation during inference.", "body_md": "The classic \"[Dallas](https://transformer-circuits.pub/2025/attribution-graphs/biology.html#dives-tracing)\" example from Anthropic focuses on an **internal circuit** in an LLM.\n\nI became curious about what the same underlying process looks like when viewed through the **generated reasoning trace **instead of hidden activations. The resulting attribution graph looks much more structured than I expected:\n\nBelow is an animated version:\n\nUsing simple gradient attribution, DAG tracing, and sparse pruning on Qwen3-1.7B, the resulting graph already resembles a '**reasoning trajectory**'.\n\nThis feels like a different perspective on mechanistic interpretability: instead of analyzing internal circuits directly, it explores causal structure within the generated language itself through context-attribution DAGs. *P.S. There is some related work, such as the ***Thought Anchors*** series. However, the goals are different, and their approach relies on prior semantic knowledge, whereas this approach requires almost no semantic knowledge.*\n\nI'm curious whether this perspective can become useful beyond visualization—for example, as a way to study how information propagates through language during inference, and perhaps, more broadly, the informational dynamics of language and LLMs.\n\n*Source code:* [https://github.com/dqy08/InfoLens](https://github.com/dqy08/InfoLens)", "url": "https://wpnews.pro/news/tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas", "canonical_source": "https://www.lesswrong.com/posts/uBeXWTAHbyf68yu36/tracing-causal-structure-in-llm-generated-text-a-different", "published_at": "2026-07-20 14:22:09+00:00", "updated_at": "2026-07-20 14:45:36.699113+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools"], "entities": ["Anthropic", "Qwen3-1.7B", "InfoLens"], "alternates": {"html": "https://wpnews.pro/news/tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas", "markdown": "https://wpnews.pro/news/tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas.md", "text": "https://wpnews.pro/news/tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas.txt", "jsonld": "https://wpnews.pro/news/tracing-causal-structure-in-llm-generated-text-a-different-lens-on-the-dallas.jsonld"}}