Anti-LLM Sentiment Considered Harmful

Contentious debate surrounding the use of large language models (LLMs) in software development, particularly criticizing those who dismiss LLM tools without serious hands-on experience. It highlights the Rust programming language community's adoption of a nuanced LLM policy that permits only solicited, high-quality, and well-reviewed LLM-generated code contributions, effectively functioning as a soft ban. The author also shares a personal perspective that LLMs are best used as assistive devices for specific tasks, especially in strongly typed languages, while emphasizing the need for better tooling to improve reliability and collaboration.

To bring the conversation back on topic, most of the pushback here seems to be coming from people who haven’t used coding agents seriously and are mostly repeating common tropes about LLMs. Do my “deep concerns about ethics and sustainability” count as “common tropes”? Do I need to invest time into doing something that I consider deeply unethical and completely wasteful so that my concerns are taken seriously? Should I start collecting rigorous statistics on how many LLM PRs I had to prove to be garbage? On a happier side note about related news, I recommend looking at Rust development of their LLM policy essentially a very softly elaborated ban, which is nice to see , and especially some of the related slashback on the avoidance of discussion of ethical concerns. A very unfortunately-worded disturbing trigger statement is now edited out of the PR, but perhaps it will be visible in history or in various related posts content warning for F words . Yikes, that discussion looks like quite the prickly bush. I can imagine it must be very difficult to discuss a topic like that where so many people have such strong, and such opposing views. I’m willing to bet that there cannot exist a policy on something this controversial without at least some people calling the authors names over it e.g. “coward” . I suppose that’s just what you have to deal with if you’re in a leading role of such a big project. Some hard decisions will need to be made that will make a lot of people extremely unhappy. Both on the policy itself as well as the discussion about the policy. Looking at the policy itself, it seems like they tried to put a lot of nuance in. It is neither a full ban not a full toleration of LLM use. Solicited, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure. “Solicited” means that a reviewer has communicated ahead of time that they are willing to review an LLM-created PR. New contributors cannot use an LLM unless they first talk with a reviewer. This must be the same reviewer who will be assigned to the PR. This sounds pretty much like a soft ban. Someone from the team can essentially say “here’s this problem I don’t wanna bother with, but am willing to accept an LLM PR if it’s good enough”. It requires prior consent or advertisement, so to speak . hey i noticed this thread is getting long and i have some fun thoughts that i think land in the middle. For those who dont know me, i did a huge amount of contribs 2006- 2020ish in haskell space my current stance on llms is: they are not a substitute for domain literacy or competency. they actually perform better in a task that uses a strongly static typed programming language and type checking as one success gate for the task. vs weakly/unsound / dynamic etc they are a wonderful assistive device for precisely the steps in building cool stuff where ive often frozen/couldn’t figure out the best way to realize an idea i have in my head. namely the model half asses it then i get mad and fix it. the tools for using these models currently suck and no one has realized that the runner /harness around the model can have an incredible impact on quality of work and agentic reliability broadly ive been working on a whole bunch of tools to hoist using these models into being much more reliable, and ive been using haskell to do quite a bit of this. there is so much low hanging fruit to build better tools im probably gonna open source some pieces, like i have a pretty nice little kit for connecting to almost every single model provider anyways: no matter what its def the case that things are changing, so i think the best way to move forward is do cool shit and incentivize building tools that make it easier for collab to be as nice as we historically recall theres a lot of other problems in the world that are trickier to address, my focus has been figuring out how to make these tools the most reliable assistive device possible. ive been dog fooding a bunch of stuff around llms using stuff on my cartazio gh org repos, and im hoping to standup a pleasant very fancy 2nd gen llm harness etc as saas this summer. so im biased, but honestly the more reliable they get in my setups the easier it is fir me to build and share all sorts of cool stuff Could you give a couple examples of what you mean? I’ve been following along this interesting thread too, as a Haskell curious developer. It seems to me that what people really resist and rightly so is “slop”. However, IMO, “ai tools” are just tools, you can do great or horrible things with them. If I had to evaluate it, I’d say I can go 3 times as fast when coding with AI-assistance, with equivalent or better quality. And I use a simple prompt to initialize a “tdd workflow”, such that I review micro changes and add them to my git stages, only to commit “logical units of work”. With that workflow, I’ve been very happy with where I’ve landed… So, to me, a good AI flow is TDD + good prompts transformed into skills if replay-ability is required But you seem to suggest that there’s something more to that? Not to derail this thread, but I’d be interested to hear your opinions on the matter. Note that I’ve explored alternative CLI agents such as “pi”, “opencode”, etc. , and their extensions such as “omp” and so on , different models including Chinese ones, etc. And I haven’t felt any of them moved the needle in the grand scheme of things. I see more micro improvements that don’t really change the fundamentals of programming. I’d be interested to hear anyone else’s opinion on the matter of course, including Haskell-specific considerations. so, try out my oh punkin pi repo and tell me what it feels lkme vibe wise etc and email me back if you like or dislike anything. im semi deliberately usualky working in a way that kinda maps to treating llms like smart hardworking clinically insane intern that you need to monitor. if im having to spell out a spec on turnthe crank stuff im doing something wrong. if only because specs should be subordinate to user feedback