{"slug": "most-developers-test-their-code-why-don-t-they-test-their-ai", "title": "Most Developers Test Their Code. Why Don't They Test Their AI?", "summary": "A developer argues that AI applications are software and should be tested with the same rigor as traditional code, advocating for evaluation datasets, prompt testing, and context testing. The developer highlights that probabilistic AI outputs require systematic evaluation rather than relying on intuition or 'it worked once' optimism.", "body_md": "Here's something I've been thinking about while building AI systems:\n\nDevelopers are obsessed with testing code.\n\nWe write unit tests.\n\nIntegration tests.\n\nEnd-to-end tests.\n\nCI pipelines.\n\nCode reviews.\n\nLinting.\n\nType checking.\n\nBut then we build an AI feature and suddenly the testing strategy becomes:\n\n\"I tried it three times and it seems pretty good.\"\n\nThat's not testing.\n\nThat's optimism.\n\nAnd I think this is becoming one of the biggest weaknesses in AI development.\n\n**AI Applications Are Software**\n\nConsider a simple AI coding assistant.\n\nThe workflow might look like this:\n\nUser Request\n\n↓\n\nContext Retrieval\n\n↓\n\nPrompt\n\n↓\n\nLLM\n\n↓\n\nGenerated Code\n\n↓\n\nValidation\n\nEvery component can fail.\n\nThe retrieval can return the wrong files.\n\nThe context can be incomplete.\n\nThe prompt can be ambiguous.\n\nThe model can hallucinate.\n\nThe generated code can contain bugs.\n\nYet many AI applications have no automated way of detecting these failures.\n\nWe wouldn't accept that standard from a normal API.\n\nWhy should AI be different?\n\n**\"It Worked Once\" Means Almost Nothing**\n\nSuppose you're building a system that converts natural language into SQL.\n\nYou test:\n\n\"Show me the top 10 customers by revenue.\"\n\nThe model generates:\n\n```\nSELECT customer_name, SUM(revenue) AS total_revenue\nFROM sales\nGROUP BY customer_name\nORDER BY total_revenue DESC\nLIMIT 10;\n```\n\nLooks good.\n\nYou ship it.\n\nThen a user asks:\n\n\"Show me the top 10 customers by revenue in 2025,\n\nexcluding cancelled orders.\"\n\nSuddenly your system may produce completely different behavior.\n\nAI outputs are probabilistic.\n\nThat means testing one input isn't enough.\n\nBuild an Evaluation Dataset\n\nOne of the simplest things an AI builder can do is create a small evaluation dataset.\n\nFor example:\n\n```\ntest_cases = [\n    {\n        \"input\": \"Find the top 10 customers by revenue.\",\n        \"expected_contains\": [\"GROUP BY\", \"ORDER BY\", \"LIMIT\"]\n    },\n    {\n        \"input\": \"Find revenue for 2025 excluding cancelled orders.\",\n        \"expected_contains\": [\"2025\", \"cancelled\"]\n    },\n]\n```\n\nNow you can run your AI system against the same cases whenever you change:\n\nThat changes everything.\n\nYou're no longer asking:\n\n\"Does this feel better?\"\n\nYou're asking:\n\n\"Did performance improve?\"\n\n**Prompts Need Tests Too**\n\nThis is one reason I don't think prompt engineering is disappearing.\n\nA production prompt isn't just something you write once.\n\nIt is part of the system.\n\nAnd if you change it, you should know whether the change improved the output.\n\nI discussed the importance of this broader discipline in [The Real Reason Prompt Engineering Isn't Going Away](https://dev.to/jaideepparashar/the-real-reason-prompt-engineering-isnt-going-away-2koo).\n\nThe next step is to connect prompt engineering with evaluation.\n\nThink of it like software:\n\nPrompt v1\n\n↓\n\nEvaluation\n\n↓\n\nResults\n\n↓\n\nPrompt v2\n\n↓\n\nEvaluation\n\n↓\n\nCompare\n\nThat's much more reliable than changing prompts based on intuition.\n\n**Context Needs Testing Too**\n\nHere's another problem.\n\nYou can have a perfect prompt and still get a terrible answer because the AI received the wrong context.\n\nImagine a coding assistant receives:\n\nPrompt:\n\n\"Fix the authentication bug.\"\n\nContext:\n\n5 unrelated files\n\n+\n\noutdated documentation\n\n+\n\nwrong configuration\n\nThe model may generate perfectly reasonable code based on completely incorrect information.\n\nThis is why I believe context engineering is becoming just as important as prompt engineering.\n\nI wrote about this in Why Context Engineering Is More Important Than Prompt Engineering.\n\nThe lesson is simple:\n\nDon't only test what you ask the model. Test what you give the model.\n\n**Workflows Need Evaluation**\n\nThis becomes even more important when AI is part of a larger workflow.\n\nConsider:\n\nUser\n\n↓\n\nRetriever\n\n↓\n\nLLM\n\n↓\n\nTool Call\n\n↓\n\nValidation\n\n↓\n\nFinal Response\n\nWhere did the failure happen?\n\nYou need to know.\n\nWas the retrieval wrong?\n\nDid the model select the wrong tool?\n\nDid the API return bad data?\n\nDid validation fail?\n\nThis is one reason I've argued that workflows often matter more than agents.\n\nA well-defined workflow gives you clear places to measure and debug.\n\nI explored that argument in [Why I Think Workflows Matter More Than Agents](https://dev.to/jaideepparashar/why-i-think-workflows-matter-more-than-agents-3p82).\n\n**Start Small**\n\nYou don't need an expensive AI evaluation platform to begin.\n\nStart with 20–50 representative test cases.\n\nFor each case, record:\n\nThen run the dataset whenever you make a significant change.\n\nOver time, your evaluation dataset becomes one of the most valuable assets in your AI project.\n\nIt captures what \"good\" actually means.\n\n**My AI Evaluation Rule**\n\nI've started thinking about AI systems in three layers:\n\nBasic functionality.\n\nReliability.\n\nEngineering maturity.\n\nThe third question is where many AI projects struggle.\n\nIf you cannot measure improvement, you're mostly guessing.\n\n**Evaluation Is the Missing Layer**\n\nThe AI industry has spent enormous effort improving:\n\nBut evaluation deserves the same attention.\n\nBecause eventually every AI system needs to answer one uncomfortable question:\n\n\"How do you know it works?\"\n\nNot:\n\n\"The demo looked impressive.\"\n\nNot:\n\n\"The model is highly capable.\"\n\nNot:\n\n\"Users seem to like it.\"\n\nShow me the evaluation.\n\nThat's the engineering mindset I want to see more often in AI.\n\n**Build This Into Your GitHub Workflow**\n\nI'm also adding a simple AI evaluation starter to my companion AI Builder resources.\n\nA useful structure is:\n\nai-evaluation/\n\n├── README.md\n\n├── test_cases.json\n\n├── evaluate.py\n\n├── results.csv\n\n└── prompts/\n\n├── v1.txt\n\n└── v2.txt\n\nThe idea is straightforward:\n\nPrompt changes → Run tests → Record results → Compare versions.\n\nThis turns experimentation into an engineering process.\n\n**Final Thoughts**\n\nI don't think AI development should be:\n\nPrompt\n\n↓\n\nLooks good\n\n↓\n\nShip\n\nIt should look more like:\n\nBuild\n\n↓\n\nEvaluate\n\n↓\n\nMeasure\n\n↓\n\nImprove\n\n↓\n\nEvaluate Again\n\n↓\n\nDeploy\n\nThat's how we build reliable software.\n\nAnd I believe that's how we need to start building reliable AI.\n\nThe future of AI engineering won't belong only to people who know how to make models produce impressive outputs.\n\nIt will belong to developers who can measure, reproduce, debug, and improve those outputs.\n\nBecause the most important question in AI isn't:\n\n**\"Can the model do it?\"**\n\nIt's:\n\n**\"Can I prove that my system does it reliably?\"**", "url": "https://wpnews.pro/news/most-developers-test-their-code-why-don-t-they-test-their-ai", "canonical_source": "https://dev.to/jaideepparashar/most-developers-test-their-code-why-dont-they-test-their-ai-1kp3", "published_at": "2026-08-21 05:21:43+00:00", "updated_at": "2026-08-21 05:44:31.639866+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-products", "developer-tools"], "entities": ["Jaideep Parashar"], "alternates": {"html": "https://wpnews.pro/news/most-developers-test-their-code-why-don-t-they-test-their-ai", "markdown": "https://wpnews.pro/news/most-developers-test-their-code-why-don-t-they-test-their-ai.md", "text": "https://wpnews.pro/news/most-developers-test-their-code-why-don-t-they-test-their-ai.txt", "jsonld": "https://wpnews.pro/news/most-developers-test-their-code-why-don-t-they-test-their-ai.jsonld"}}