{"slug": "we-cannot-100-trust-ai-generated-output-a-concrete-example", "title": "We Cannot 100% Trust AI-Generated Output: A Concrete Example", "summary": "A developer found that Claude, an AI assistant from Anthropic, incorrectly flagged zero-length bugs in the WinReg C++ library's string getters, failing to account for the RegGetValueW API's null-termination guarantee. Claude later corrected itself after the developer pointed out the error, highlighting that AI-generated code cannot be trusted 100% and may contain hidden bugs or wrong assumptions.", "body_md": "Recently I asked Claude to review the code of my [WinReg C++ library](https://github.com/GiovanniDicanio/WinReg) (which is a **C++ high-level wrapper** around the low-level C-interface **Windows Registry** API).\n\nAs a result of its analysis, Claude reported that there were zero-length bugs in my code, in particular Claude stated that zero-length REG_SZ/REG_EXPAND_SZ values crash the GetStringValue, GetExpandStringValue, TryGetStringValue and TryGetExpandStringValue methods of the RegKey class.\n\nIn particular, Claude noted that I correctly guarded against dataSize == 0 in the binary-returning getters (like RegKey::GetBinaryValue), but the string getters do not have such guard; they *unconditionally *do:\n\n```\nresult.resize((dataSize / sizeof(wchar_t)) - 1);\n```\n\nClaude specified that REG_SZ and REG_EXPAND_SZ values can be legitimately stored with cbData == 0 (zero bytes, no NUL at all; different from an empty string made by a single NUL ‘\\0’).\n\nIf you substitute zero for the dataSize variable in the above statement, you end up with:\n\n```\nresult.resize(SIZE_MAX);\n```\n\nwhich would throw a std::length_error exception.\n\nClaude proposed to fix the above code using the same edge-case check logic I had already implemented in the binary getters to guard against the zero-length case:\n\n```\nif (dataSize == 0)\n{\n    result.clear();\n}\nelse\n{\n    result.resize((dataSize / sizeof(wchar_t)) - 1);\n}\n```\n\nMy WinReg library is quite battle-tested, and there were bugs related to some edge cases that I had already fixed, so I was curious, and tried writing a zero-length string value in the registry, and read it back with my existing code. And I noted that (at least in Windows 11 where I tested my code) the dataSize == 0 condition was *not *hit at run-time.\n\nThat is because in my C++ code I invoke the **RegGetValue(W)** API, which by contract *guarantees* to return a NUL-terminated string, even if the string stored in the registry doesn’t have a NUL-terminator. (This is not the case for older APIs like [RegQueryValueEx](https://learn.microsoft.com/en-us/windows/win32/api/winreg/nf-winreg-regqueryvalueexw).)\n\nI replied to Claude pointing that out, and Claude corrected itself:\n\nYou’re right, and thanks for the pushback — I should have accounted for\n\n`RegGetValueW`\n\n‘s null-termination guarantee before flagging that as a bug.\n\nThose AI tools can be very powerful, but the key *takeway* here is that we should not forget that they are just ** tools**, and we should not trust AI-generated output and code 100%, because bugs and wrong assumptions can be hidden in that AI code, too.", "url": "https://wpnews.pro/news/we-cannot-100-trust-ai-generated-output-a-concrete-example", "canonical_source": "https://giodicanio.com/2026/07/28/why-we-should-double-check-the-ai-output-a-bug-which-wasnt/", "published_at": "2026-07-28 16:42:57+00:00", "updated_at": "2026-07-28 16:52:32.346623+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-safety"], "entities": ["Claude", "Anthropic", "WinReg C++ library", "Giovanni Dicanio", "Windows Registry", "RegGetValueW", "Windows 11"], "alternates": {"html": "https://wpnews.pro/news/we-cannot-100-trust-ai-generated-output-a-concrete-example", "markdown": "https://wpnews.pro/news/we-cannot-100-trust-ai-generated-output-a-concrete-example.md", "text": "https://wpnews.pro/news/we-cannot-100-trust-ai-generated-output-a-concrete-example.txt", "jsonld": "https://wpnews.pro/news/we-cannot-100-trust-ai-generated-output-a-concrete-example.jsonld"}}