{"slug": "surfacing-the-methodology", "title": "Surfacing the Methodology", "summary": "Plotly Studio has launched a new \"Methodology\" feature that surfaces data sources, columns, formulas, and assumptions directly in the interface to help users verify AI-generated analytics. The feature uses a separate AI agent to review code and results without prior session context, aiming to prevent mental atrophy and reduce AI hallucinations. This design addresses the growing challenge of reviewing AI outputs in data products by providing an implicit verification checklist.", "body_md": "# Surfacing the Methodology\n\nWe recently designed a new feature in [Plotly Studio](https://plotly.com/studio) called \"Methodology\".\nIt surfaces the data source, columns, formulas, assumptions, and approach\nfront and center in the interface. We designed this so that it'd be easier\nto verify and trust the results from the agentic loop and to prevent a feeling\nof mental atrophy that is becoming all to common in our AI experiences.\n\nIn many AI products and productivity tools, AI is generating the majority of the output and our role has transformed from creating things to reviewing things.\n\nAnd in data products, reviewing and validating that the AI did the write thing is essential work.\n\nIn Plotly Studio, we've been thinking a lot about how we can make it easier to verify what the AI generated.\n\nWe've found that designing verification UX also solves another key problem in our use of AI - mental atrophy. AI can often generate too much and too fast for us to comprehend, and the default product experiences can often feel overwhelming. Verification ends up being a way to slow it all down.\n\nIn the most recent release of Plotly Studio, you'll find a new \"Methodology\" button below each step. The \"Methodology\" will display an easy to scan but detailed, thorough view of the most important aspects of the data analysis step.\n\nHere's what it looks like:\n\nIn Plotly Studio, all of the analytics are backed by code. We've long believed in the power of code-based analytics. But many of the tools that we use for working with code in software engineering are ill-equiped for working with data.\n\nReview is one of those. Each of these steps has 20-100 lines of code. While this is a managable set of code to review, it's not always easy to pull out the data-related details of the code. The Methodology section helps put it in context. It also provides an implicit review checklist, by surfacing the important things front and center like any assumptions that were made or if any data points were hard-coded.\n\nMethodology is created by a second agent of the main agentic loop and context. This has a two benefits:\n\n- It doesn't slow down the main agentic loop.\n- It isn't \"influenced\" by the previous context.\n\nAI sessions can sometimes get \"stuck in their ways\". They can fib or make up data in order to satisify user's request. By creating the Methodology as a separate AI call, we can avoid this contextual history and steering. The Methodology agent simply reviews the code and the result without any prior knowledge of the original user's request or the history of the session. This gives the results quite a bit more neutrality and consistency.\n\nMethodology is now available in Plotly Studio. Give it a try and\n[let me know](/about) what you think.", "url": "https://wpnews.pro/news/surfacing-the-methodology", "canonical_source": "https://chris-parmer.com/methodology/", "published_at": "2026-06-01 00:00:00+00:00", "updated_at": "2026-06-03 06:36:39.917370+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "ai-agents", "generative-ai"], "entities": ["Plotly Studio"], "alternates": {"html": "https://wpnews.pro/news/surfacing-the-methodology", "markdown": "https://wpnews.pro/news/surfacing-the-methodology.md", "text": "https://wpnews.pro/news/surfacing-the-methodology.txt", "jsonld": "https://wpnews.pro/news/surfacing-the-methodology.jsonld"}}