Stop making your stakeholders do mental math during eval reviews

A new visualization tool, inspect_viz, enables AI teams to generate heatmaps from evaluation logs, replacing line-by-line log reviews with color-coded matrices that highlight model performance patterns. The tool, which works with inspect_ai and pandas, helps stakeholders instantly spot model ceilings, track skill gains, and compare configurations, as demonstrated by examples like pairing '3.6-flash' with a 'gemini_api_skill' at a 1.0 score.

Stop making your stakeholders do mental math during eval reviews This is where heatmaps become a critical part of your AI workflow. Instead of forcing people to read line-by-line log values, a heatmap uses color intensity to represent numerical magnitude—like accuracy scores. The human eye is much better at spotting a bright yellow patch in a sea of purple than it is at comparing "0.87" vs "0.89" in a spreadsheet. Moving beyond basic Inspect dashboards While tools like inspect view are great for a quick deep dive into specific logs or sorting metrics to see model vs. model differences, they aren't built for high-level storytelling. If you are running complex evaluations with multiple independent and dependent variables, you need a 2D visual matrix to make the patterns jump out. A well-constructed heatmap allows you to: Spot model ceilings: Instantly see where a model hits a performance plateau across different skills. Track skill gains: Visually confirm if adding a specific prompt engineering technique or tool-use skill actually moves the needle. Compare configurations: Line up models on one axis and skills on the other to see the entire performance spectrum at a single glance. How to generate heatmaps with inspect viz To implement this in your own pipeline, you need the inspect viz library installed alongside inspect ai and pandas . I have been playing around with the implementation, and the core logic relies on the scores heatmap function. If you want to build this from scratch, you can look at the way the data is marshaled into the visualization. The script essentially takes your evaluation logs and wraps them into a horizontal or vertical orientation to make the axes readable. Here is the basic command to run a heatmap generation script if you have your logs ready: python3 inspect viz heatmap.py logs -o heatmap.html Analyzing the output When you generate the heatmap, you typically end up with a matrix where: The Y-axis Vertical : Represents your models or specific configurations. This allows you to scan down a single column to see how a specific skill performs across different model versions. The X-axis Horizontal : Represents the skills or test cases. Scanning across a row shows you exactly how a single model's performance fluctuates as the task complexity changes. In a real-world deployment scenario, this is how you prove value. For example, if you see a high-intensity color like a bright yellow for a specific combination—say, 3.6-flash paired with a specific gemini api skill at a 1.0 score—it becomes an immediate talking point. You aren't just saying "the model is better"; you are showing a visual heat signature of success.If you are building out a rigorous LLM agent evaluation framework, don't settle for raw logs. If you can't visualize the delta between your iterations, you aren't really seeing the full picture. Next AI coding agents create weird 15-minute gaps in your day → /en/threads/7666/