{"slug": "7-checks-to-run-before-trusting-ai-generated-code", "title": "7 Checks to Run Before Trusting AI-Generated Code", "summary": "A 2025 study of 576,000 LLM-generated code samples documented package hallucinations in Python and JavaScript at a 5.2% rate for commercial models and 21.7% for open-source models, creating exposure to \"slopsquatting,\" in which attackers register commonly hallucinated package names. The article, citing GitHub's responsible-use guidance for Copilot, lays out seven checks for AI-generated code, starting with inspecting the git diff for scope creep, running compilers, linters and type checkers, and independently verifying every new dependency and API before merging.", "body_md": "# 7 Checks to Run Before Trusting AI-Generated Code\n\nAI makes writing code cheap. Verifying it is still your job.\n\nAI coding assistants can generate a large amount of working code in a few minutes. This increases productivity but it also makes them dangerous. Bad AI-generated code does not necessarily look bad. It may compile, pass tests, and use clean variable names while still making the wrong assumption about an API, authorization rule, edge case, or dependency. **[GitHub warns](https://docs.github.com/en/copilot/responsible-use/code-review)** that AI-generated code can be semantically incorrect and recommends carefully reviewing and testing it before merging. So instead of asking \"Does this AI-generated code look good?\", run these **7 checks**.\n\nDoes this AI-generated code look good?\n\n## 1. Inspecting the Diff Before Running Anything\n\nYour first check should not be ChatGPT, Copilot, Claude Code, or another model.\n\nIt should be Git.\n\nWhen coding agents are given access to an entire repository, they sometimes modify more than the task requires. A small bug fix can unexpectedly become a refactor, dependency update, configuration change, and formatting sweep.\n\nStart with:\n\n```\ngit status\ngit diff --stat\ngit diff\n```\n\n`git diff --stat` gives you an immediate picture of the scope.\n\nSuppose you asked the agent:\n\n```\nAdd validation to the email signup endpoint.\n```\n\nYou expected changes in two files. But the diff says:\n\n```\n12 files changed, 437 insertions(+), 186 deletions(-)\n```\n\nStop there.\n\nBefore examining whether the code works, ask why the task required hundreds of changed lines.\n\nWhile reviewing the diff, look specifically for:\n\n- files unrelated to the task\n- deleted tests, disabled validation, and new configuration values\n- permission changes\n- changes to dependency files\n- large refactors hidden inside a feature\n- duplicated helper functions and error handling that ignores failures\n\nThe goal is simple: **make sure the AI solved the task you gave it rather than a larger task it invented.**\n\n## 2. Making the Compiler, Linter, and Type Checker Judge It\n\nHumans are terrible compilers.\n\nDo not spend ten minutes visually checking syntax that a deterministic tool can check in seconds.\n\nFor a Python project, your first automated checks might look like:\n\n```\npython -m compileall .\nruff check .\nmypy .\n```\n\nFor JavaScript or TypeScript:\n\n```\nnpm run lint\nnpx tsc --noEmit\nnpm run build\n```\n\nStatic type checking is particularly useful with AI-generated code because models can confidently connect values that happen to look compatible but are not.\n\nFor example:\n\n``` php\ndef get_user_id(user: dict[str, str]) -> int:\n    return user[\"id\"]\n```\n\nThe code looks harmless. But the declared dictionary contains strings while the function claims to return an integer. A type checker can flag this immediately.\n\n## 3. Verifying Every New Dependency and API\n\nThis deserves special attention because it is an AI-specific failure mode.\n\nModels sometimes invent package names, and those names can look completely plausible.\n\nFor example, an AI might suggest:\n\n```\npip install fastapi-jwt-auth-plus\n```\n\nDo not assume a dependency exists simply because the name sounds reasonable.\n\nA **[2025 study](https://arxiv.org/abs/2406.10279)** analyzed 576,000 large language model (LLM)-generated code samples and documented package hallucinations across Python and JavaScript — 5.2% for commercial models and 21.7% for open-source models. This is important because hallucinated package names create an opportunity known as **slopsquatting**: someone can register a package under a commonly hallucinated name and potentially trick developers into installing it. So whenever AI adds a dependency, verify it independently.\n\nFor Python:\n\n```\npip index versions package-name\n```\n\nFor npm:\n\n```\nnpm view package-name\n```\n\nThen check:\n\n- Does the package actually exist?\n- Is this the official package? Who maintains it? When was it last updated?\n- Does the project genuinely need it?\n- Does the version match the documentation the code appears to use?\n- Did the lockfile suddenly change substantially?\n- Does it introduce known vulnerabilities?\n\nDo the same for APIs.\n\nIf the AI writes:\n\n```\nclient.responses.stream_async(...)\n```\n\ndo not assume the method exists because it resembles the rest of an SDK.\n\nOpen the current official documentation and confirm:\n\n```\nclass → method → parameters → return type → supported version\n```\n\nThis is especially important for rapidly changing AI SDKs, cloud APIs, and JavaScript frameworks. Every newly introduced package and every unfamiliar external API should be traceable to an authoritative registry or official documentation.\n\n## 4. Testing the Behavior, Not the Implementation\n\nNow run the existing test suite.\n\nFor example:\n\n```\npytest -q\n```\n\nBut a green test suite is only the beginning. The more important question is:\n\n**What important case is not being tested?**\n\nSuppose the AI generated:\n\n``` python\ndef apply_discount(price, discount):\n    return price - (price * discount / 100)\n```\n\nAnd generated this test:\n\n``` python\ndef test_discount():\n    assert apply_discount(100, 10) == 90\n```\n\nGreat.\n\nBut what about:\n\n```\ndiscount = 0\ndiscount = 100\ndiscount = -10\ndiscount = 150\nprice = 0\nprice = None\nprice = \"100\"\n```\n\nAI frequently generates the happy path because the happy path is also what appears most prominently in examples and documentation. For every generated feature, test at least four categories:\n\n```\nNormal input\nBoundary input\nInvalid input\nFailure condition\n```\n\nFor an API endpoint, that might mean:\n\n```\nValid request\nMissing field\nMalformed field\nUnauthorized request\nDatabase/service failure\n```\n\nAlso make sure the AI did not \"fix\" a failing test by deleting it or weakening the assertion. For critical functions, consider writing the test **before showing the implementation to another model**. The feature should survive tests for what happens when the user, network, database, file system, and input data do something unexpected.\n\n## 5. Attacking the Security Boundaries\n\nFunctional correctness and security are different tests. This function may work:\n\n```\nquery = f\"SELECT * FROM users WHERE email = '{email}'\"\ncursor.execute(query)\n```\n\nIt may even pass all your unit tests.\n\nIt is still unsafe.\n\nWhenever AI-generated code touches user-controlled input, authentication, databases, files, shell commands, or external URLs, switch from a functional mindset to an adversarial one.\n\nSearch the patch for areas involving:\n\n```\nSQL queries\nauthentication\nauthorization\nfile paths\nsubprocess execution\nHTML rendering\ndeserialization\nredirects\nHTTP requests\ntokens\npasswords\nAPI keys\n```\n\nAsk questions like:\n\n```\nCan the user control this string?\nCan they access somebody else's resource?\nCan this path escape its intended directory?\nCan this query be parameterized?\nCan this command execute arbitrary input?\nWhat happens if this URL points to an internal service?\nAre permissions checked server-side?\n```\n\nThen run static security tooling where appropriate.\n\nOne general-purpose option is **[Semgrep](https://semgrep.dev/)**:\n\n```\nsemgrep scan --config auto\n```\n\nSemgrep documents this configuration for automatically selecting relevant scanning rules.\n\nPython projects can also use **[Bandit](https://bandit.readthedocs.io/)**:\n\n```\nbandit -r .\n```\n\nBandit analyzes Python's abstract syntax tree for common security problems.\n\nRepositories using GitHub can enable **[CodeQL](https://codeql.github.com/)** code scanning, which can analyze supported languages on pull requests and protected branches.\n\nOne more check is important: **Search for secrets.** Look for:\n\n```\nAPI keys\ndatabase passwords\nOAuth tokens\nprivate keys\ncloud credentials\nconnection strings\n```\n\nGitHub's **[secret scanning](https://docs.github.com/en/code-security/secret-scanning)** exists specifically to detect credentials accidentally committed into repositories. An AI assistant should never turn this:\n\n```\napi_key = os.environ[\"API_KEY\"]\n```\n\ninto:\n\n```\napi_key = \"sk-...\"\n```\n\njust because it saw a credential in its environment or context. No untrusted input should cross a security boundary without you understanding exactly how it is validated, escaped, authorized, or isolated.\n\n## 6. Following the Failure Path\n\nAI-generated code often looks strong when everything works. So deliberately examine what happens when everything does **not** work.\n\nConsider this generated code:\n\n```\ntry:\n    response = requests.get(url)\n    return response.json()\nexcept Exception:\n    return {}\n```\n\nIt appears robust because it contains error handling. But in reality, it can hide:\n\n- timeouts\n- connection failures\n- authentication errors\n- invalid JSON\n- HTTP 500 errors\n- programming bugs\n\nAnd every one of them becomes:\n\n```\n{}\n```\n\nThis makes debugging considerably harder. Look for broad patterns such as:\n\n```\nexcept Exception:\n    pass\ncatch (e) {\n  console.log(e);\n}\n```\n\nor:\n\n```\nreturn None\n```\n\nbeing used for multiple unrelated failures.\n\nAlso inspect:\n\n#### Retries\n\nDoes the AI retry forever?\n\n```\nwhile True:\n    try:\n        ...\n    except Exception:\n        continue\n```\n\n#### Timeouts\n\nDoes a network request have one?\n\n```\nrequests.get(url, timeout=10)\n```\n\n#### Transactions\n\nIf operation three fails, are operations one and two rolled back?\n\n#### Async Code\n\nAre tasks awaited correctly?\n\n#### Shared State\n\nCould two requests modify the same data simultaneously?\n\n#### Cleanup\n\nAre database connections, temporary files, sockets, and locks released after an error?\n\nA useful test is to deliberately break each external dependency. For example:\n\n```\nTurn off the database.\nReturn HTTP 500 from the API.\nProvide corrupt JSON.\nRemove file permissions.\nForce a request timeout.\nSubmit the same request twice simultaneously.\n```\n\nObserve what your application does. The question is not:\n\n```\nDoes it fail?\n```\n\nIt is:\n\n```\nDoes it fail predictably, safely, and visibly?\n```\n\n## 7. Asking Whether the Code Should Exist at All\n\n \n\nThe final check is the one developers often skip. Suppose the AI generated this:\n\n``` python\ndef transform_items(items):\n    output = []\n\n    for item in items:\n        transformed_item = transform_single_item(item)\n        output.append(transformed_item)\n\n    return output\n```\n\nAnd:\n\n``` python\ndef transform_single_item(item):\n    return item[\"name\"]\n```\n\nThe whole thing could simply be:\n\n```\n[item[\"name\"] for item in items]\n```\n\nThis is not about shorter code. AI coding agents can generate unnecessary:\n\n- wrappers, classes, and helper functions\n- configuration layers and abstractions\n- comments, fallback logic, and duplicated utilities\n\nBefore merging, ask:\n\n```\nDid this feature introduce a new abstraction?\nCould an existing function already do this?\nDid we add a library for something available in the standard library?\nDid the AI duplicate an existing helper?\nCan this implementation be reduced by 30% without losing clarity?\nWould I understand this code six months from now?\n```\n\nBut do not optimize for the smallest possible line count.\n\nOptimize for the smallest amount of **new complexity** necessary to solve the problem.\n\nThis is also where human understanding becomes the final gate.\n\nTake the most important generated function and explain it without the AI.\n\nIf you cannot explain:\n\n```\nwhat goes in\nwhat comes out\nwhat state changes\nwhat can fail\nwhy this approach was chosen\n```\n\nthen you are not ready to own that code in production.\n\n## Final Thoughts\n\nAI coding tools dramatically reduce the cost of producing code. They do not reduce the cost of proving that code is correct. The wrong response is to distrust every line an AI writes.\n\nThe best AI coding workflow is therefore not **generate and trust**.\n\nIt is **generate and verify**.\n\nAI can be the fastest programmer on your team. Your job is to make sure it still has to pass the same gates as everyone else.\n\n \n\n \n\n**[**\\[Kanwal Mehreen\\](https://www.linkedin.com/in/kanwal-mehreen1/)**](https://www.linkedin.com/in/kanwal-mehreen1/)** is a machine learning engineer and a technical writer with a profound passion for data science and the intersection of AI with medicine. She co-authored the ebook \"Maximizing Productivity with ChatGPT\". As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She's also recognized as a Teradata Diversity in Tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having founded FEMCodes to empower women in STEM fields.", "url": "https://wpnews.pro/news/7-checks-to-run-before-trusting-ai-generated-code", "canonical_source": "https://www.kdnuggets.com/7-checks-to-run-before-trusting-ai-generated-code", "published_at": "2026-10-19 12:00:43+00:00", "updated_at": "2026-10-19 12:17:59.332751+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety", "developer-tools", "artificial-intelligence"], "entities": ["GitHub", "Copilot", "ChatGPT", "Claude Code", "Python", "JavaScript", "TypeScript"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/7-checks-to-run-before-trusting-ai-generated-code", "markdown": "https://wpnews.pro/news/7-checks-to-run-before-trusting-ai-generated-code.md", "text": "https://wpnews.pro/news/7-checks-to-run-before-trusting-ai-generated-code.txt", "jsonld": "https://wpnews.pro/news/7-checks-to-run-before-trusting-ai-generated-code.jsonld"}}