cd /news/artificial-intelligence/we-cannot-100-trust-ai-generated-out… · home topics artificial-intelligence article
[ARTICLE · art-77297] src=giodicanio.com ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

We Cannot 100% Trust AI-Generated Output: A Concrete Example

A developer found that Claude, an AI assistant from Anthropic, incorrectly flagged zero-length bugs in the WinReg C++ library's string getters, failing to account for the RegGetValueW API's null-termination guarantee. Claude later corrected itself after the developer pointed out the error, highlighting that AI-generated code cannot be trusted 100% and may contain hidden bugs or wrong assumptions.

read2 min views1 publishedJul 28, 2026
We Cannot 100% Trust AI-Generated Output: A Concrete Example
Image: source

Recently I asked Claude to review the code of my WinReg C++ library (which is a C++ high-level wrapper around the low-level C-interface Windows Registry API).

As a result of its analysis, Claude reported that there were zero-length bugs in my code, in particular Claude stated that zero-length REG_SZ/REG_EXPAND_SZ values crash the GetStringValue, GetExpandStringValue, TryGetStringValue and TryGetExpandStringValue methods of the RegKey class.

In particular, Claude noted that I correctly guarded against dataSize == 0 in the binary-returning getters (like RegKey::GetBinaryValue), but the string getters do not have such guard; they *unconditionally *do:

result.resize((dataSize / sizeof(wchar_t)) - 1);

Claude specified that REG_SZ and REG_EXPAND_SZ values can be legitimately stored with cbData == 0 (zero bytes, no NUL at all; different from an empty string made by a single NUL ‘\0’).

If you substitute zero for the dataSize variable in the above statement, you end up with:

result.resize(SIZE_MAX);

which would throw a std::length_error exception.

Claude proposed to fix the above code using the same edge-case check logic I had already implemented in the binary getters to guard against the zero-length case:

if (dataSize == 0)
{
    result.clear();
}
else
{
    result.resize((dataSize / sizeof(wchar_t)) - 1);
}

My WinReg library is quite battle-tested, and there were bugs related to some edge cases that I had already fixed, so I was curious, and tried writing a zero-length string value in the registry, and read it back with my existing code. And I noted that (at least in Windows 11 where I tested my code) the dataSize == 0 condition was *not *hit at run-time.

That is because in my C++ code I invoke the RegGetValue(W) API, which by contract guarantees to return a NUL-terminated string, even if the string stored in the registry doesn’t have a NUL-terminator. (This is not the case for older APIs like RegQueryValueEx.)

I replied to Claude pointing that out, and Claude corrected itself:

You’re right, and thanks for the pushback — I should have accounted for

RegGetValueW

‘s null-termination guarantee before flagging that as a bug.

Those AI tools can be very powerful, but the key takeway here is that we should not forget that they are just ** tools**, and we should not trust AI-generated output and code 100%, because bugs and wrong assumptions can be hidden in that AI code, too.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @claude 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-cannot-100-trust-…] indexed:0 read:2min 2026-07-28 ·