Why agentic AI needs better experts

A developer used OpenAI's Codex AI agent to refactor the uutils sed program to handle raw bytes instead of characters, improving compatibility and performance. The agent handled 78 prompts, but 61 required expert guidance for improvements, highlighting the need for human oversight. The session cost an estimated $80 in tokens and had an environmental impact of ~326 kg CO₂e and ~12,871 L of water.

Over the past few days I changed https://github.com/uutils/sed/pull/487 the way the uutils https://uutils.org/ project’s sed https://github.com/uutils/sed/ program handles data to default from characters to raw bytes. This improves compatibility with GNU sed and also performance. Given the change’s size and extent 13 changed files, 1740 insertions, 609 deletions , I worked with an AI agent OpenAI Codex , which allowed me to experience first-hand both its power and limitations. On the one hand, Codex guided me correctly toward the task and handled it successfully both through the initial prompt and through refinements. It handled expertly a lot of tedious grunt work, saving me time and effort. At all stages the modified code compiled and run successfully. Having a large set of unit and integration tests as well as CI checks helped guide the agent. On the other hand, I realized that to obtain production-quality code, Codex’s work needed careful reviewing and extensive expert guidance. At the end of this post I have included all prompts that I issued, with the ones that guided Codex to improve the code formatted in bold. From the 78 prompts and sub-prompts 61 78% asked for improvements. So, To be fair, agents continuously improve, and many of the prompts I gave, such as those involving comments and unit tests, could have easily be handled by a more capable agent or even with a suitable initial agent configuration. But, as coding agents improve, the tasks we give them will also become more demanding, as will become the required reviewing and expert guidance. Finally, there’s AI’s monetary and environment cost, which may mean that not all work that can be done by an AI agent is worth doing it thus. I made several changes on my own, mainly to save time and waste. At the end of the session, Codex using gpt-5.5 reported Token usage: total=4,298,673 input=4,113,242 + 110,110,848 cached output=185,431 reasoning 30,920 . ChatGPT calculates https://chatgpt.com/share/6a4e17e0-df70-83eb-9a23-f894b8582b5 that with current prices the session’s tokens would cost about $80. I used Codex through my OpenAI subscription and current token costs are probably subsidized, but the figure gives the true cost’s likely magnitude. My guess is that for sustained use the cost would mirror that of a developer salary. This is not unreasonable given the benefit I received, but organizations certainly need to weigh that cost into their budgets. I was surprised by the session’s large environmental impact. ChatGPT based on Mistral’s Le Chat reporting gave me ~326 kg CO₂e and ~12,871 L of water usage. To put these values into perspective the CO₂ emissions are similar to a 2,5000 km trip in an efficient diesel/gasoline car or one short/medium-haul passenger round trip. And if we price these externalities, the CO₂ emissions cost €26 through the EU ETS carbon price or $42–117 through the US EPA social cost of carbon. The cost of the water used in the local Athens domestic water tariff would be ~€4.50–41. Again, these costs are not astronomical, but also far from negligible. Below are the prompts I gave during the session. Those associated with improvements I spotted are formatted in bold. Note that some of the unit tests I asked were missed in previous versions, so they’re code improvements requiring human prompting but not the agent’s fault. sed compatibility score? fast optimizations. Transliteration field named fast into byte fast . script text to bytes ? script text to bytes is the wrong approach. Handle the script as bytes. as str ? cases shouldn’t we report an appropriate input runtime error where applicable? ScriptCharProvider::new initializations simply deal with bytes? git subst current bytes current ; git subst next line bytes next line to make the code mirror the original. But I broke it. Please fix it. current char shouldn’t current be current bytes ? provider.rs deal with bytes rather than “logical characters” ? Byte sequences should be converted into characters / strings only when required, e.g. for FancyRegex and transliterate when processing UTF-8 input. I added character mode in the context to help you with that. parse transliteration to return a variant based on context.character mode , so bytes or string. Then parsed script text to utf8 wouldn’t be needed. Also shouldn’t the regex pattern start life as bytes and only be upgraded to string for FancyRegex ? Also, shouldn’t LiteralMatcher operate on bytes? NEEDS RE -based logic? ByteRegex matcher? HEAD . Also when adding new functions always prefix them with a documentation comment. Regex::new should get passed context.character mode and only create a string from UTF-8 if needed when mode is Utf8 . In Byte mode Regex::new should fail with an error back-references are not supported in byte mode if it needs to construct a FancyRegex matcher. Also, add integration tests to test s and y command behavior under diverse input / scrips / LC ALL settings. Transliteration lookup fn into lookup char to contrast with lookup byte . push script char should follow the bytes.push only in Bytes mode, otherwise it can create invalid Unicode scripts. In Unicode mode it should push the character’s corresponding Unicode representation. line.current return a char. extend from slice line.current bytes consider using the more readable push line.current byte . ERR constant for “transliteration strings are not the same length” push script char only in delimited parser. ParsedTransliteration into command.rs . self.utf8 verified.set true ; be in set to string ? I’ll fix it . Regex Bytes 0x7f as \x ? Can’t regex bytes handle them as is? Just tell me. Match::from bytes avoid constructing text in Bytes contexts? retreat ? new ucmd provide a way to specify the environment? character mode to byte regex pattern . Just examine character mode when building the regex. Match::from bytes constructing text in Bytes contexts. byte regex pattern let cmd str = pattern Don’t convert into UTF-8. Rather adjust and continue the flow with bytes using OsStr / OsString . r and w commands. list should also work correctly in byte mode without converting into a String. Be careful to avoid duplication in the way characters / bytes are shown. retreat new ucmd .env ... rather than the custom run sed with locale . In general follow the logic of existing tests. Regex::new issue a USimpleError rather than construct one from scratch os string from bytes try from utf8 and issue an error on failure. os string from bytes put it in delimited parser writer rather than duplicating it. readable char readable ascii byte . write list item . write list item calls len for each new item, making its complexity N². Encapsulate the function and the length in a class to avoid this. NEEDS RE.is match and NEEDS FANCY RE.is match . It’s enough to test only for the first, right? Transliteration::is byte safe . os string from bytes , parsed bytes to utf8 , parse character class with ' ' , parse transliteration for mode , Transliteration::apply match , Translitaration::from bytes , lookup byte , write line bytes , the regex::bytes::RegexBuilder::new path, Regex::captures iter , and Regex::find . is byte identity field? If so, just loose the trivial getter. pub crate ? unterminated substitute replacement \q in compile replacement ~step in an RE address parse command ending compile replacement compile replacement compile replacement compile subst command compile trans command at EOL in parse character class : in parse character class . parse transliteration bytes prompts.md with the prompts I issued in this session. Format code identifiers in backticks. Format prompts that ask for corrections to previous work in bold.Last modified: Wednesday, July 8, 2026 1:30 pm Unless otherwise expressly stated, all original material on this page created by Diomidis Spinellis is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License https://creativecommons.org/licenses/by-nc/4.0/ .