philh 1 That seems like probably a very different failure mode to me?
Like, the kind of bug that allows arbitrary file deletion in a program that’s intended to be able to delete specific files, is very different from the kind of bug that allows arbitrary file deletion in a program that’s not intended to be able to delete files. I’m imagining you were working on the first kind of program, and ychr is the second.
I’m imagining that ychr has a handful of uses for IO: reading files, mutable variables (IORef/MVar/STM/…), exceptions, printing to stdout/stderr. Maybe one or two I’m forgetting. But no writing to the filesystem, and no network access. Assuming that’s right (which it may well not be!) then I think “ruling out hidden destructive bugs to sufficient degree of confidence” isn’t all that difficult.
1 Like
I think our imagination is irrelevant when dealing with slop. And it doesn’t matter what you prompted. It’s known that LLMs sometimes ignore instructions and that unrelated code can make it into the implementation.
I don’t think any amount of effects system or safe haskell would convince me.
Either you own the output and understand it or you’re playing code roulette.
philh 3 I’m not imagining details about what the LLM generated, I’m imagining details about how to check what the LLM generated.
Like, if you know that a codebase shouldn’t have any IO at all, then checking it doesn’t is fairly straightforward. Do you agree with that?
1 Like
gregbm 4 How do I know your program doesn’t delete my home directory?
Same way you’d know for human-generated code on hackage, probably.
Speaking of which, what defenses does hackage have against malicious packages?
That’s a pretty strange way to dismiss the “lack of agency” issue. And I don’t think it can be taken seriously.
- Empirical research across Fortune 50 enterprises found that AI-assisted developers produce commits at three to four times the rate of their peers but introduce security findings at 10x the rate, creating a security debt that accumulates faster than organizations can remediate it [7].
Our study suggests that securing vibe-coded appli-
cations requires treating vibe coding as a workflow-level security problem more than a code-generation problem
LLMs change the game here fundamentally sadly. They rework distributed trust because there is less of a hurdle of effort to get something compiling and plausibly working.
I see this myself in industry with junior devs. Before, they would get stuck and be unable to cut a PR that looked reasonable. Now, the LLM can give them that by default. But the LLM PRs have some pretty silly issues that trace back to the juniors’ lack of understanding.
gregbm 7 With regards to trusting code on hackage, I am more worried about intentionally-introduced malware than I am about vibecoded packages, so I would be interested in a verification mechanism that isn’t just scanning for signs of AI-generated code.
1 Like
Models can very well be compromised by state actors.