{"slug": "a-code-inspection-of-serious-llm-generated-code", "title": "A code inspection of 'serious' LLM-generated code", "summary": "An IfSQ code inspection of a 140,000-line C++ visual modelling application written almost entirely by Claude Code found that LLM-generated code contains serious defects, though LLMs outperformed IfSQ at unearthing certain problems, according to Lab Note #6 from ea.rna.nl. The author reports that more than 50% of his energy now goes to quality and defects rather than features, and that fixing the flagged problems is possible but gradually degrades the effectiveness of coding with LLMs. The project also includes about 15,000 lines of stable documentation and 30,000 lines of temporary documentation, and the author concludes that LLM agents \"will in no way be able to replace actual human engineers.", "body_md": "*Special note for subscribers who get this in the mail: this is an extremely long post because it contains two entire python scripts I use (and that some may want to download) and a large document that is part of my ‘contextual harness’. There is nothing I can do about that. This one is easier to read on my site where these long segments are within a small scroll box.*\n\nTo find out more about the reality of all the (generally overblown) promises about LLMs, I have been doing a very serious experiment on the poster child for Generative AI: *coding*. I wrote [one critical post](https://ea.rna.nl/2026/06/07/anthropic-openai-may-be-spending-more-than-1000-for-every-100-you-pay-them/), and I am close to writing up my conclusions, but I am hitting some work that I have to do first before I can publish that (e.g. create a demo video which is a lot of work, and before that I need to finish some final functionalities and improvements.\n\nThis is Lab Note #6. Some basic explanations about my setup are in [Lab Note #1](https://ea.rna.nl/2026/09/26/serious-coding-with-llms-lab-notes-2026-09-25-codex-vs-claude-code-usage-limit-resets/). Some basic technical notes (on tokens, context) are in [Lab Note #5](https://ea.rna.nl/2026/10/21/serious-coding-with-llms-lab-notes-2026-10-21-the-throttling-of-subscriptions-by-openai-and-anthropic-is-really-happening/).\n\n**Experiment**: As an experiment to learn about the limits of LLM-coding, I am building a professional visual modelling application from the ground up. It can handle multiple grammars (definable in JSON). It has state of the art graphics. It is rich in what users can influence and how (e.g. schemes, style/colour/geometry properties). It is responsive. It is well documented. It has a solid software architecture. Every line of code (and many of the other texts) have been generated by the LLM (so far almost exclusively Claude Code). It runs on macOS and Windows (Linux will be added later). If I finish it and I am satisfied, it will be open sourced. This is a serious sized project: 140k lines C++, about 15k lines stable documentation and 30k lines of temporary documentation (analysis, design and planning for substantial changes over the last half year), so about one third of what is in play is not code but documentation. **Warning**: *That I am doing this should not be read as either pro or contra coding with LLM-agents.* I am highly critical of the hype, but I am trying to establish the limits of LLM-coding based on as serious a use case as possible. I can say already that my experience so far has let me conclude that LLM-agents will in no way be able to replace actual human engineers. **Note**, these lab notes do not explain all technical details. I generally try to write for a general audience, but if you’re not a software engineer you’ll need to kind of ‘read over’ everything you do not understand, but you can still get the gist of it.\n\nThis is an extremely long and detailed note. Sorry for that. So, let’s do a\n\n**TL;DR**\n\nA review based on the IfSQ method brought to light that LLMs are better than IfSQ in unearthing certain problems, but also that the code that LLMs generate have serious problems that were unearthed by IfSQ. Attacking those problems is possible, but it slowly degrades the effectiveness of coding with LLMs.\n\nFor a software engineer, the experience of coding with LLMs remains a somewhat awe-inspiring experience, but that is in part typical shallow human judgement. Underneath, reality bites. By now, more than 50% of my energy goes to quality and defects, not features. I can do more than I ever did, but for how long?*Next time I will report on how much better the code got after Claude reworked it based on the inspection results.*\n\nAnd if you get this by mail as a — very appreciated — subscriber, this one you should read on the web as the large amounts of code aren’t in scroll-boxes in that email.\n\n## Before we start: Slop, Sludge and ‘Vakwerk’\n\nA month ago, Michael Larabel wrote [AI Made A Lot Of “Hideous” Code But Found Major Bottlenecks For Faster Linux Compilation](https://www.phoronix.com/news/AI-To-Faster-Linux-Kernel-Comp). In it, he relates the observations by a Linux kernel engineer, Lorenzo Stoakes, about the combination of the effects of LLM-coding that has struck many: the code may work but it can be *hideous*. Another example is this item from a Linux kernel graphics development [GitLab repository: iMac 5K shows only 1 tile (2560×2880) instead of a 2 tile display (5120×2880)](https://gitlab.freedesktop.org/drm/amd/-/work_items/4455#note_3728748), which started out with a 1500 lines of code LLM-assisted patch, which was brought back to 50 lines and might even become only 25 lines of code. This kind of echoes the paper OpenAI released on a quasi-Riemann proof which is an important result, but the 200-page paper produced by the LLM would be rejected by a journal because, while the math checks out (mechanically checked by LEAN), it was extremely poorly written. A 49-page version from OpenAI with a slightly different proof with a smaller but still important result was edited by a human. According to [reports](https://www.quantamagazine.org/ai-proved-the-quasi-riemann-hypothesis-what-happens-now-20261019/) it might be possible to bring that size substantially down further.\n\nWe all know AI-slop by now. AI-generated results that are so poor that — often without humans noticing — they do more harm than good. But in mathematics we have those results mentioned above. Results that check out (are correct) but the *form* is poor. I am reminded of the use of the word ‘sludge’ as defined by Cass R. Sunstein of Harvard University [here](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3379367). Sludge as a form of friction. \n\nIf LLMs generate, we might classify them into the following three categories, broadly:\n\n1. AI-**Slop** : Generated results that are incorrect. The cheap model results are often that;\n2. AI-**Sludge** : Generated results that are correct, but roundabout, bloated, ‘kludgy’ way;\n3. AI-**Vakwerk** : This is what we want (and are being sold…).Vakwerk means ‘result of a skilful job’. But it has a second meaning that is ‘truss’ which is a sturdy structure. I have tried to come up with something in English or well known in English, but even with all the suggestions an LLM could make I found none better than this. All languages have the meaning, but none like Dutch. So, let’s propose this as a loan-word. (I considered ‘AI-solidity’ (my #2), ‘AI-art’, and many more).\n\nMy project is an experiment to get **vakwerk** (*solid* results) from coding with LLMs, which in themselves are stochastic, so there is a fundamental contradiction we are confronting.\n\n## The Code Audit\n\nAs I wrote in Lab Note #4, I recently had the code from Claude inspected at [IfSQ Level 2](https://ifsq.org/level-2.html). IfSQ stands for “The Institute for Software Quality” and it is a method based on visual inspection by an experienced engineer that provides a measure of *maintainability*: analysability, modifiability, testability, modularity, reusability.\n\nIfSQ is directed at inspecting code created by humans and *to be maintained by humans*. The latter is important as I expect the economics of LLMs is going to be a fundamental problem that limits their usefulness in many (if not most, we will see) cases. LLMs are hideously inefficient in more ways than one, but [they are currently extremely heavily subsidised](https://ea.rna.nl/2026/06/07/anthropic-openai-may-be-spending-more-than-1000-for-every-100-you-pay-them/), mostly by venture capital and private credit. So, my project has a stated goal: the code must be maintainable by humans (I try to be prepared for when the serious models become unaffordable for me when the party is over, and we also might take future software engineers’ conundrums into account — more on that a next time).\n\nIn IfSQ, a code sample is collected and audited on a couple of aspects:\n\n1. Work in Progress (**WIP** ), such as TODO statements in comments in the code, disabled code, an empty block left open for later;\n2. Structured Programming (**SP** ): too long routines, too deep nesting, too complex routine;\n3. Single Point of Maintenance (**SPM** ): Magic Numbers or Strings, Copy/Paste programming, the latter is known by its acronym**DRY** : Do not Repeat Yourself. Copy-pasting code is a recipe for maintenance issues as changing the code easily leads to forgetting one or two copies;\n4. Defensive Programming (**DP** ): Parameters not checked, return value ignored, unexpected state not trapped (e.g. a switch statement without a default: clause). You can easily get false positives here, there are many`if` statements that are fine without an`else` . “Do this if that, else do nothing” is a fine flow. So, you need to inspect which is the case and that isn’t mechanical, it requires understanding code (hence human in the loop).\n5. Causes for Concern (**CfC** ): Apparent missing functionality, apparent wrong results, hard to maintain. These are not part of a Level 2 inspection, but they can be part of a more heavy ‘review’. I’m getting back to that below.\n\nAn overall rating goes from AAA (best), via AA,A,B,C,D,E,F,FF, to FFF (worst), based on the estimated number of defects per thousand lines of code. I’ve worked on such an audit myself once, about twenty years ago and I’ve been using one during my time as Lead Architect for the Judiciary of The Netherlands two decades ago. I then found IfSQ to be one of the better ways to audit code quality, so I decided to ask originator of this method — [Graham Bolton](http://linkedin.com/in/gbolton) — if he wanted to play along today. He did.\n\nGraham has created some tools to support this form of an audit, but in the end the judgement remains a human judgement. Supported by the tools, however, he could quickly inspect about half of the code.\n\nThe idea of this kind of audit is not just simply a measurement. It is a result that is meant to be discussed with the engineers. When we used such an audit twenty years ago, you could see the junior engineers fight its findings (“come on, how bad is that”) while experienced, senior engineers shifted nervously in their chairs as they recognised how bad the findings were for the future of a system. Graham confronted them with what they already understood but had ignored. So, I worded the result of the audit and gave that back to Claude to work with.\n\nCaveat: Graham did a quick and rough version of the inspection (still a lot of work — thanks, Graham!) so the tools he uses bring up candidates to look at, but as a human he has to check these if the candidates are really issues. So, we will have some false positives as Graham did a quick scan and not a one-by-one deep analysis of the candidates. As an example, an `if` statement not followed by an `else` statement is a *potential* issue (an issue with defensive programming, an unexpected result is not handled and the program simply runs on in a broken state after which everything can go wrong). A `switch` statement without a `default:` label is too. But there are many situations where this is not true. Using a hard coded number (e.g. distance from border: `6` points) in code is a bad sign (for maintenance, because if you change it in one place, will you not forget it to change it somewhere else), while code that finds the center of a box by dividing height and width by `2.0` is not an issue.\n\nThe result of the IFSQ inspection on my LLM-generated code was interesting and helpful. To start with: the overall score was FF.\n\nWhich looks bad, obviously.\n\nHere is the overall result:\n\nSo, why was it so bad? What did Graham’s IfSQ audit flag? And given that the audit is based on human creators, how bad is this result if you just keep using an LLM which maybe isn’t bothered by these scores at all? Let’s answer all of these.\n\n### IfSQ finding: Work in Progress (WIP): *Perfect*\n\nIfSQ’s WIP-detection is very limited. It looks for things like dead code, TODO statements in comments, that sort of thing. That is behaviour that is typical for (poor or too much under stress) coders, not for LLMs generating code. An LLM is ‘often wrong, never in doubt’, so it will not leave something like that in the code like an uncertain human would.\n\n### IfSQ finding: Defensive Programming (DP): *OK*\n\nMy code quality document that is part of Claude’s context (which I will share), already is built to direct Claude at producing robust and self-documenting code. As the development has proceeded I have often had to correct Claude to improve the quality, and such findings then found their way into my ‘stable documentation’. In the case of `if` and `else`, my instructions before the audit were already that any `if` should either have an `else`, or a comment documenting why an ‘else’ is not required here. Example:\n\n```\nif (!EACMP_replaceFieldStateValue(ordinaryViewFields, EACMP_ViewField::CanvasBounds, std::optional())) { // A state of another record type; no 'else' branch. EACMP_log(EACMP_LogLevel::Error, \"Clipboard view snapshot lacks its canvas limit\"); return false;}\n```\n\nThere is no ‘else’ branch here because the routine returns, nothing else needs to be done in that case. When I looked into a few of these obstacles found, I saw several false positives which a thorough human review would have spotted. Say that about 1/3 of the DP-3 findings were real based on my quick scan.\n\n### IfSQ finding: Single Point of Maintenance (SPM): *Bad*\n\nThe results of SPM surprised me in part. Not the hard coded numbers and the ‘string literals’. I knew they were there and most of these were things like keys in JSON, which are strings, and I use JSON schema to check correctness in the data. But that isn’t really a good way to be ready for human maintenance. While an LLM will definitely not mistype ‘default’ as ‘defualt’, humans might. And `default` will be caught by the compiler, string literals will not, so using them is a liability. I already knew I had to fix that.\n\n### IfSQ finding: Structured Programming (SP): *Disastrous*\n\nThe LLM-generated code is — despite all the attention paid by me to quality, probably more than 50% — extremely complicated. Take this snippet from inside a large routine:\n\n```\n// The container's own subtree and its ancestors are never candidates.std::unordered_set containerSubtree;std::function collectSubtree = [&](const EACMP_Uuid& parentUUID) { // depth-first over the nesting forest below parentUUID for (const auto& childPtr : view->viewElements()) { // each possible child if (childPtr->parentViewElement() != parentUUID) { continue; } // not a child here if (!containerSubtree.insert(childPtr->uuid()).second) { continue; } // already collected collectSubtree(childPtr->uuid()); } };collectSubtree(containerViewElementUUID);\n```\n\nI defines a variable, the defines a routine inside the outer routine and uses that. That gets you a lot of depth in your logic. Even harder to read: sometimes these extra routines were defined as anonymous ‘lambdas’. A routine is called with parameters, and one of these parameters is the address of an on-the-fly defined function, i.e. the `[&]() {` part of the first line is the start of a complete function definition *inside* the parameter list of the function call:\n\n``` php\nconst auto outcome = undoStack->execute(\"Create Parts\", [&]() { // Resolve targets after conflicting previews have been cancelled. EACMP_Landscape* landscape = context->landscapeForModel(modelUUID); EACMP_Model* model = landscape ? landscape->modelByUUID(modelUUID) : nullptr; if (!model || !model->elementByUUID(rootElementUUID)) { // GR-3: the root must still exist; I-1 supplies its grammar EACMP_log(EACMP_LogLevel::Error, \"EACMP_executeBornWithParts: model or root element unresolved\"); return EACMP_ReplayOutcome::Refused; }\n```\n\nYou can guess how complex that can become.\n\n### How did Claude react?\n\nQuite well, in fact. I first gave it this:\n\n```\nLook at eacmplib/src/label_geometry.cpp.A switch statement starts at line 18.types.h says:// Label area placement: horizontal axis. Shape = the label area is the shape's own labelArea// rect (scaled), not a placed box — flush band labels; always set on BOTH axes together.enum class EACMP_HPlacement {OutsideLeft, InsideLeft, HCenter, InsideRight, OutsideRight, Manual, Shape};The final case statement says:case EACMP_HPlacement::Shape: // should never reach: shape placement is resolved to the shape's own rect upstreamNow with if/else statements we have the rule that there always is an else, unless it is clear there doesn't need to be one in which case this is documented in the code. (Note: we seem to (sometimes?) miss that \"else if\" is not the same as a final \"else\" so there are if switches that end with \"else if\" but without either a final \"else\" or a comment explaining why it isn't needed).Here I see a few problematic issues:1. There is no default: entry point. The 'should never reach' begs the question: under what circumstances can it be reached? Is \"shape\" (the JSON string version of EACMP_HPlacement::Shape). \"shape\" is in the grammar-reference, but its behaviour is not explained or obvious (as the others are) and it 'should never be reached'.2. If 'should never be reached' is a programming error (which seems to be the case with an enum) it should follow the hard rules for not accepting them.It seems \"shape\" acts here is a kind of broken \"default\" that should never be reached. But it is allowed in the grammar.It also seems some grammar keys are turned into enums in the code and others are not. I think they all should beSwitch statements should always have a \"default\" unless there is a good reason in which case there should be a comment (like 'no else').A second related issue is that there are many 'magic strings' and 'magic numbers' in the code. Dividing by 2 might be acceptable to get at a center of something geometrically, but numbers like '6' (for a default distance), or 255 instead of MAX_COLOR_BYTEVALUE or \"key\" for a string key in JSON are brittle from a human maintenance perspective. These could be created in a #define to keep everything in a single location, or like I have already seen for some in enums, with translating strings from to enums (which is more robust)I want you to create a report docs/20261017-codequality.md on these kind issues with the patterns you find in the C++ code and suggestions for how to fix these and keep track of these (audit)\n```\n\nClaude generated this first version of a document (Its finding ‘F1’ is funny, it found out that the audit it had built itself was pretty broken):\n\nGraham reacted to this when I showed him: “My impression is that your agent has reacted better to the findings than any development team I ever spoke with, and that it shows a remarkable amount of common sense in its analysis. Glad to see that the review led to the agent finding actual bugs and fixing them!”. This document became the ‘temporary documentation’ for the tasks regarding code quality as is my usual modus operandi.\n\nI had Claude address the quality problems with the flows (like unreachable code etc.). I had it address the ‘magic strings’ and ‘magic numbers’ it had used widely in the code.\n\n### Addressing DRY (Do not Repeat Yourself, Copy-Paste coding)\n\nI had experienced DRY already in code generated by Claude already: e.g. some user actions in the application were reachable in three ways: via the main menu, a context menu, and a properties panel. At some point, Claude had changed two out of three (so when I used the third it didn’t work properly). You can simply fix the problem, but a better way is to fix the situation in a way that the problem cannot happen as easily in the future: remove as much DRY as you can.\n\nAt the time, I had Claude create a script that could detect DRY (you get all these temporary side tasks):\n\n``` bash\n#!/usr/bin/env python3# MIT License## Copyright (c) 2026 Gerben Wierda## Permission is hereby granted, free of charge, to any person obtaining a copy# of this software and associated documentation files (the \"Software\"), to deal# in the Software without restriction, including without limitation the rights# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell# copies of the Software, and to permit persons to whom the Software is# furnished to do so, subject to the following conditions:## The above copyright notice and this permission notice shall be included in all# copies or substantial portions of the Software.## THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE# SOFTWARE.\"\"\"rna-finddry - find and report copy-pasted blocks of lines in source files.Usage: rna-finddry.py --length=N [--length=N ...] [--suffix=S ...] [--ignore=PATTERN ...] [--cut-line-after=PATTERN ...] [--cut-between START END ...] [--string-literal=PATTERN ...] [--verbose] [--frequent-first | --longest-first] [--reverse-report] [--max-report=N] [--summary] PATH ... rna-finddry.py --usage rna-finddry.py --versionText matched by the --cut options (e.g. comments, but not inside stringliterals) is removed, then lines are normalised (whitespace compacted andtrimmed, line endings unified); blank lines, and lines matching an --ignorepattern, are invisible to the search. For each length N, every maximal blockof at least N lines that occurs more than once is reported, most frequentfirst (or longest first with --longest-first), optionally limited andreversed. --summary adds totals per length at the end.\"\"\"import argparseimport bisectimport enumimport osimport reimport sysfrom array import arrayfrom collections import namedtuple# sourcefiles.py, embedded by install.py: registered as module sourcefiles for the import belowimport sysimport typesdef _register_embedded_module(module_name, source): \"\"\"Run an embedded module's source and register it, so that importing it finds this copy.\"\"\" module = types.ModuleType(module_name) exec(compile(source, f\"\", \"exec\"), module.__dict__) sys.modules[module_name] = module_register_embedded_module('sourcefiles', ( b'\"\"\"sourcefiles - parts shared by rna-finddry and rna-findcompl.\\n' b'\\n' b'Messages, the --usage action, command-line path checks, collecting and reading\\n' b'the source files, and limiting a report. install.py embeds this module in the\\n' b'installed programs, so that they run without it.\\n' b'\"\"\"\\n' b'\\n' b'import argparse\\n' b'import os\\n' b'import signal\\n' b'import stat\\n' b'import sys\\n' b'\\n' b'MODULE_VERSION = 1 # started at 1; increase by one on every change to this file\\n' b'UNRELEASED = \"unreleased\" # a program\\'s RELEASE_DATE when run from the source tree; install.py sets the real date\\n' b'EXIT_ERROR = 2 # also the status argparse uses for command-line errors\\n' b'\\n' b'FILE_COLLECTION_MANUAL = \"\"\"\\\\\\n' b' Directories are scanned recursively, following symlinks. Hidden directories\\n' b' (name starting with \".\") and binary files (containing a NUL byte) are\\n' b' skipped. A file or directory reached by several paths is scanned once.\\n' b'\"\"\"\\n' b'\\n' b'\\n' b'class MessageReporter:\\n' b' \"\"\"Writes warnings and errors on stderr, prefixed by the program name.\"\"\"\\n' b'\\n' b' def __init__(self, program_name):\\n' b' self.program_name = program_name\\n' b'\\n' b' def warning(self, message):\\n' b' \"\"\"Print a warning on stderr.\"\"\"\\n' b' print(f\"{self.program_name}: warning: {message}\", file=sys.stderr)\\n' b'\\n' b' def error(self, message):\\n' b' \"\"\"Print an error on stderr.\"\"\"\\n' b' print(f\"{self.program_name}: error: {message}\", file=sys.stderr)\\n' b'\\n' b'\\n' b'class ManualAction(argparse.Action):\\n' b' \"\"\"argparse action for --usage: print the help followed by the manual (keyword `manual`), then exit.\"\"\"\\n' b'\\n' b' def __init__(self, option_strings, manual, dest=argparse.SUPPRESS, default=argparse.SUPPRESS, help=None):\\n' b' super().__init__(option_strings=option_strings, dest=dest, default=default, nargs=0, help=help)\\n' b' self.manual = manual\\n' b'\\n' b' def __call__(self, parser, namespace, values, option_string=None):\\n' b' \"\"\"Print and exit successfully; runs before required arguments are checked, like -h.\"\"\"\\n' b' sys.stdout.write(parser.format_help() + self.manual)\\n' b' parser.exit()\\n' b'\\n' b'\\n' b'def version_text(program_name, program_version, release_date):\\n' b' \"\"\"Return the --version text; release_date is an installation date (yyyy-mm-dd) or UNRELEASED.\"\"\"\\n' b' if release_date == UNRELEASED: # run from the source tree, not installed\\n' b' return f\"{program_name} version {program_version}, unreleased\"\\n' b' return f\"{program_name} version {program_version}, released on {release_date}\"\\n' b'\\n' b'\\n' b'def install_default_signal_handlers():\\n' b' \"\"\"End quietly on a closed pipe and on Ctrl-C, as a C program does, instead of with a traceback.\"\"\"\\n' b' signal.signal(signal.SIGPIPE, signal.SIG_DFL)\\n' b' signal.signal(signal.SIGINT, signal.SIG_DFL)\\n' b'\\n' b'\\n' b'def check_max_report(parser, max_report):\\n' b' \"\"\"Validate --max-report (argparse\\'s None: not given); return (limit_report_size, max_report_size).\"\"\"\\n' b' limit_report_size = max_report is not None\\n' b' max_report_size = 0 # read only when limit_report_size\\n' b' if limit_report_size: # a limit was given: it must be positive\\n' b' if max_report < 1:\\n' b' parser.error(f\"--max-report must be at least 1, got {max_report}\")\\n' b' max_report_size = max_report\\n' b' return limit_report_size, max_report_size\\n' b'\\n' b'\\n' b'def check_arguments(paths, messages):\\n' b' \"\"\"Stat every command-line path; return (all valid, [(path, stat)]) and report each invalid one.\"\"\"\\n' b' all_valid = True\\n' b' checked_arguments = []\\n' b' for path in paths: # must exist and be a directory or regular file\\n' b' try:\\n' b' path_stat = os.stat(path)\\n' b' except OSError as failure:\\n' b' messages.error(f\"cannot access {path}: {failure.strerror}\")\\n' b' all_valid = False\\n' b' continue\\n' b' if not (stat.S_ISDIR(path_stat.st_mode) or stat.S_ISREG(path_stat.st_mode)):\\n' b' messages.error(f\"{path} is neither a file nor a directory\")\\n' b' all_valid = False\\n' b' continue\\n' b' checked_arguments.append((path, path_stat))\\n' b' return all_valid, checked_arguments\\n' b'\\n' b'\\n' b'class SourceCollector:\\n' b' \"\"\"Collects the files to scan; follows symlinks and visits every directory and file once.\\n' b'\\n' b' is_wanted_file_name(name) selects the files found while scanning a\\n' b' directory; files named on the command line are always collected.\\n' b' \"\"\"\\n' b'\\n' b' def __init__(self, is_wanted_file_name, messages):\\n' b' self.is_wanted_file_name = is_wanted_file_name\\n' b' self.messages = messages\\n' b' self.visited_directories = set() # (device, inode)\\n' b' self.visited_files = set() # (device, inode)\\n' b' self.source_files = [] # (path, named on command line)\\n' b'\\n' b' def add_argument(self, path, path_stat):\\n' b' \"\"\"Add a command-line path; named paths bypass the file-name and hidden-directory filters.\"\"\"\\n' b' if stat.S_ISDIR(path_stat.st_mode): # directory: scan recursively\\n' b' self._walk(path, path_stat)\\n' b' else: # regular file, as ensured by check_arguments\\n' b' self._add_file(path, path_stat, True)\\n' b'\\n' b' def _add_file(self, path, file_stat, is_explicit):\\n' b' \"\"\"Record a file unless it was already reached by another path.\"\"\"\\n' b' identity = (file_stat.st_dev, file_stat.st_ino)\\n' b' if identity in self.visited_files: # hard link, symlink or repeated argument\\n' b' return\\n' b' self.visited_files.add(identity)\\n' b' self.source_files.append((path, is_explicit))\\n' b'\\n' b' def _mark_directory_visited(self, directory_stat):\\n' b' \"\"\"Mark a directory visited; return False if it already was.\"\"\"\\n' b' identity = (directory_stat.st_dev, directory_stat.st_ino)\\n' b' if identity in self.visited_directories: # symlink loop or repeated argument\\n' b' return False\\n' b' self.visited_directories.add(identity)\\n' b' return True\\n' b'\\n' b' def _walk(self, root_path, root_stat):\\n' b' \"\"\"Scan a directory tree depth-first, in name order.\"\"\"\\n' b' if not self._mark_directory_visited(root_stat): # already scanned via another path\\n' b' return\\n' b' pending = [root_path]\\n' b' while pending: # one directory per iteration\\n' b' directory_path = pending.pop()\\n' b' try:\\n' b' with os.scandir(directory_path) as iterator:\\n' b' entries = sorted(iterator, key=lambda entry: entry.name)\\n' b' except OSError as failure:\\n' b' self.messages.warning(f\"cannot read directory {directory_path}: {failure.strerror}\")\\n' b' continue\\n' b' subdirectories = []\\n' b' for entry in entries: # classify each entry, following symlinks\\n' b' try:\\n' b' entry_stat = entry.stat()\\n' b' except OSError as failure:\\n' b' self.messages.warning(f\"cannot access {entry.path}: {failure.strerror}\")\\n' b' continue\\n' b' if stat.S_ISDIR(entry_stat.st_mode): # descend into new, non-hidden directories\\n' b' if not entry.name.startswith(\".\") and self._mark_directory_visited(entry_stat):\\n' b' subdirectories.append(entry.path)\\n' b' elif stat.S_ISREG(entry_stat.st_mode): # regular file: apply the file-name filter\\n' b' if self.is_wanted_file_name(entry.name):\\n' b' self._add_file(entry.path, entry_stat, False)\\n' b' else: # fifo, socket or device: never a source file\\n' b' self.messages.warning(f\"skipping special file {entry.path}\")\\n' b' pending.extend(reversed(subdirectories)) # so that pop() yields them in name order\\n' b'\\n' b'\\n' b'def read_source_files(source_files, messages):\\n' b' \"\"\"Yield (path, content) of each readable, non-binary source file; warn about unreadable ones.\"\"\"\\n' b' for path, is_explicit in source_files: # one file per iteration\\n' b' try:\\n' b' with open(path, \"rb\") as source:\\n' b' content = source.read()\\n' b' except OSError as failure:\\n' b' messages.warning(f\"cannot read {path}: {failure.strerror}\")\\n' b' continue\\n' b' if b\"\\\\0\" in content: # binary file: skipped, noted only when named explicitly\\n' b' if is_explicit:\\n' b' messages.warning(f\"skipping binary file {path}\")\\n' b' continue\\n' b' yield path, content\\n' b'\\n' b'\\n' b'def select_shown_reports(reports, limit_report_size, max_report_size, reverse_report):\\n' b' \"\"\"Return the reports to list: the most important ones if limited, reversed if asked.\"\"\"\\n' b' shown_reports = reports\\n' b' if limit_report_size: # keep only the most important\\n' b' shown_reports = shown_reports[:max_report_size]\\n' b' if reverse_report: # most important last\\n' b' shown_reports = shown_reports[::-1]\\n' b' return shown_reports\\n'))from sourcefiles import (EXIT_ERROR, FILE_COLLECTION_MANUAL, UNRELEASED, ManualAction, MessageReporter, SourceCollector, check_arguments, check_max_report, install_default_signal_handlers, read_source_files, select_shown_reports, version_text)PROGRAM_NAME = \"rna-finddry\"PROGRAM_VERSION = 1 # started at 1; increase by one on every change to this fileRELEASE_DATE = '2026-10-25' # set by install.pyMINIMUM_LENGTH = 2EXIT_NO_DUPLICATES = 0EXIT_DUPLICATES_FOUND = 1CUT_LINE_AFTER_OPTION = \"--cut-line-after\"CUT_BETWEEN_OPTION = \"--cut-between\"STRING_LITERAL_OPTION = \"--string-literal\"END_OF_LINE_PATTERN = re.compile(rb\"(?=\\n)|\\Z\") # end of a --cut-line-after cut: before the newline, or end of fileMATCH_END_PATTERN = re.compile(b\"\") # end of a --string-literal region: right where its match endsmessages = MessageReporter(PROGRAM_NAME)Settings = namedtuple( \"Settings\", \"lengths accept_all_suffixes suffixes ignore_patterns region_rules verbose report_order reverse_report\" \" limit_report_size max_report_size summary paths\")RegionRule = namedtuple(\"RegionRule\", \"action start_pattern end_pattern description\")Report = namedtuple(\"Report\", \"count starts locations block_length\") # copies ordered by (path, line)LengthStatistics = namedtuple(\"LengthStatistics\", \"length report_count surplus_line_count\")class ReportOrder(enum.Enum): \"\"\"Order of the DRYs within a length's report, most important first.\"\"\" MOST_COPIES = \"most copies first\" LONGEST = \"longest first\"class RegionAction(enum.Enum): \"\"\"What happens to a region found by the --cut and --string-literal options.\"\"\" CUT = \"cut\" KEEP = \"keep\"MANUAL = r\"\"\"details:normalisation Line endings (CRLF, CR, LF) are unified. Then the --cut options remove text (see cutting). In each line, runs of whitespace become one space and leading and trailing whitespace is removed. Blank lines, and lines matching an --ignore pattern, are invisible to the search: a block runs across them and they do not count towards its length.cutting --cut-line-after cuts from its match to the end of that line. --cut-between cuts from its START match up to and including the next END match, across lines; without an END it cuts to the end of the file, with a warning. --string-literal marks each match of its pattern as a string literal, which is kept as it is: e.g. '\"(\\\\.|[^\"\\\\\\n])*\"' for C strings, so that \"http://x\" survives --cut-line-after '//'. Each file is scanned from the start: at each point the region (cut or literal) that starts first wins (on a tie, the option given first) and scanning resumes after it, so a cut start inside a literal, or a quote inside cut text, has no effect. Patterns are Python regular expressions in multi-line mode (^ and $ match at every line). Line numbers are unaffected; lines left blank are blank lines.files\"\"\" + FILE_COLLECTION_MANUAL + r\"\"\" --suffix and the hidden-directory rule apply only to what is found while scanning: paths named on the command line are always scanned.DRY blocks For each --length N, every block of at least N lines is reported that occurs more than once and cannot be extended up or down without losing a copy. Copies within one file do not overlap. When part of a block occurs more often than the whole block, both are reported.output Per --length, in command-line order, for each block: DRY (level N) COUNT lines=LENGTH TEXT with --verbose: the normalised lines of the block PATH: LINE one per copy; LINE is the first line of the copy Blocks are ordered by COUNT, most first (default, or --frequent-first; with --longest-first: by LENGTH, longest first), then by path and line. --max-report N lists only the first N blocks of each length; --reverse-report lists them in reverse order, so the most important come last. --summary and the exit status always count all blocks.summary (--summary) After all blocks: Summary: TOTAL lines of code (non-blank) Summary (level N): DRYs: BLOCKS, DRY lines: SURPLUS, per 1000 lines of code: RATE SURPLUS counts the lines in every copy except the first listed one, each line once. TOTAL includes ignored lines, but not lines left blank by cutting.exit status 0 no DRY found 1 DRY found 2 error: invalid argument, or a path that is missing or neither file nor directory Unreadable files and directories are skipped with a warning on stderr.\"\"\"class AppendRegionAction(argparse.Action): \"\"\"argparse action for the --cut and --string-literal options: collects them, tagged by option, in order.\"\"\" def __call__(self, parser, namespace, values, option_string=None): \"\"\"Append (option, values) to a new list, leaving the shared default untouched.\"\"\" setattr(namespace, self.dest, getattr(namespace, self.dest) + [(option_string, values)])def compile_pattern(parser, option, pattern, flags): \"\"\"Compile a command-line regular expression; empty and invalid ones end in parser.error.\"\"\" if pattern == \"\": # would match everywhere parser.error(f\"{option} pattern must not be empty\") try: # re offers no way to validate a pattern other than compiling it return re.compile(os.fsencode(pattern), flags) except re.error as failure: parser.error(f\"invalid {option} pattern {pattern!r}: {failure}\")def parse_arguments(argv): \"\"\"Parse and validate the command line; exits with EXIT_ERROR on invalid input.\"\"\" parser = argparse.ArgumentParser( prog=PROGRAM_NAME, allow_abbrev=False, description=\"Find and report copy-pasted blocks of lines.\") parser.add_argument( \"--length\", action=\"append\", type=int, required=True, metavar=\"N\", help=f\"report repeated blocks of at least N lines, not counting blank and ignored lines\" f\" (N >= {MINIMUM_LENGTH}); may be repeated\") parser.add_argument( \"--suffix\", action=\"append\", metavar=\"STRING\", help=\"scan only files whose name ends in STRING; may be repeated; default: all files\") parser.add_argument( \"--ignore\", action=\"append\", default=[], metavar=\"PATTERN\", help=\"leave lines matching regular expression PATTERN (searched in the normalised line) out of\" \" DRY detection; they still count as lines of code; may be repeated\") parser.add_argument( CUT_LINE_AFTER_OPTION, dest=\"region_options\", action=AppendRegionAction, default=[], metavar=\"PATTERN\", help=\"cut from a match of regular expression PATTERN to the end of the line, e.g. '//'; may be repeated\") parser.add_argument( CUT_BETWEEN_OPTION, dest=\"region_options\", action=AppendRegionAction, default=[], nargs=2, metavar=(\"START\", \"END\"), help=\"cut from a match of START up to and including the next match of END, across lines,\" \" e.g. '/\\\\*' '\\\\*/'; may be repeated\") parser.add_argument( STRING_LITERAL_OPTION, dest=\"region_options\", action=AppendRegionAction, default=[], metavar=\"PATTERN\", help=r\"\"\"keep each match of regular expression PATTERN as a string literal, untouched by cuts,\"\"\" r\"\"\" e.g. '\"(\\\\.|[^\"\\\\\\n])*\"' for C strings; may be repeated\"\"\") parser.add_argument(\"--verbose\", action=\"store_true\", help=\"also print the repeated text\") order_group = parser.add_mutually_exclusive_group() order_group.add_argument( \"--frequent-first\", dest=\"report_order\", action=\"store_const\", const=ReportOrder.MOST_COPIES, default=ReportOrder.MOST_COPIES, help=\"order the DRYs by number of copies, most first (the default)\") order_group.add_argument( \"--longest-first\", dest=\"report_order\", action=\"store_const\", const=ReportOrder.LONGEST, help=\"order the DRYs by block length, longest first\") parser.add_argument( \"--reverse-report\", action=\"store_true\", help=\"list the DRYs in reverse order, so the most important come last\") parser.add_argument( \"--max-report\", type=int, metavar=\"N\", help=\"list at most N DRYs per length, the most important ones; --summary and exit status still count all\") parser.add_argument( \"--summary\", action=\"store_true\", help=\"end with, per length, the number of DRYs and of DRY lines (lines in copies beyond the first)\" \" per 1000 non-blank lines\") parser.add_argument( \"--usage\", action=ManualAction, manual=MANUAL, help=\"show this help plus a description of behaviour, output and exit status, and exit\") parser.add_argument( \"--version\", action=\"version\", version=version_text(\"%(prog)s\", PROGRAM_VERSION, RELEASE_DATE), help=\"show the version and release date, and exit\") parser.add_argument(\"paths\", nargs=\"+\", metavar=\"PATH\", help=\"file, or directory to scan recursively\") arguments = parser.parse_args(argv) seen_lengths = set() for length in arguments.length: # each length must be valid and given once if length < MINIMUM_LENGTH: parser.error(f\"--length must be at least {MINIMUM_LENGTH}, got {length}\") if length in seen_lengths: parser.error(f\"--length {length} given more than once\") seen_lengths.add(length) accept_all_suffixes = arguments.suffix is None suffixes = () if not accept_all_suffixes: # suffixes given: none may be empty if \"\" in arguments.suffix: parser.error(\"--suffix must not be empty\") suffixes = tuple(arguments.suffix) ignore_patterns = [compile_pattern(parser, \"--ignore\", pattern, 0) for pattern in arguments.ignore] region_rules = [] for option, values in arguments.region_options: # in command-line order, which breaks ties between regions if option == CUT_LINE_AFTER_OPTION: # cut from a match to the end of its line start_pattern = compile_pattern(parser, option, values, re.MULTILINE) region_rules.append( RegionRule(RegionAction.CUT, start_pattern, END_OF_LINE_PATTERN, f\"{option} {values}\")) elif option == CUT_BETWEEN_OPTION: # cut from a start match through the next end match start_text, end_text = values region_rules.append( RegionRule(RegionAction.CUT, compile_pattern(parser, option, start_text, re.MULTILINE), compile_pattern(parser, option, end_text, re.MULTILINE), f\"{option} {start_text} {end_text}\")) else: # STRING_LITERAL_OPTION: keep the match itself literal_pattern = compile_pattern(parser, option, values, re.MULTILINE) region_rules.append( RegionRule(RegionAction.KEEP, literal_pattern, MATCH_END_PATTERN, f\"{option} {values}\")) limit_report_size, max_report_size = check_max_report(parser, arguments.max_report) return Settings(arguments.length, accept_all_suffixes, suffixes, ignore_patterns, region_rules, arguments.verbose, arguments.report_order, arguments.reverse_report, limit_report_size, max_report_size, arguments.summary, arguments.paths)def apply_cuts(text, region_rules, path): \"\"\"Remove all cut regions from text, keeping their newlines so that line numbers stay valid. Scans left to right: the region (cut or kept string literal) whose start matches first wins (ties: the first rule), and scanning resumes after it, so starts inside a region have no effect. \"\"\" pieces = [] position = 0 upcoming_starts = [rule.start_pattern.search(text, 0) for rule in region_rules] # re's None: no further match while True: # one region per iteration for index, rule in enumerate(region_rules): # re-search starts that the scan has passed match = upcoming_starts[index] if match is not None and match.start() < position: upcoming_starts[index] = rule.start_pattern.search(text, position) candidates = [(match.start(), index) for index, match in enumerate(upcoming_starts) if match is not None and match.start() >= position] if len(candidates) == 0: # no more regions: keep the rest pieces.append(text[position:]) break region_start, rule_index = min(candidates) rule = region_rules[rule_index] end_match = rule.end_pattern.search(text, upcoming_starts[rule_index].end()) is_unterminated = end_match is None # only a cut can be: a literal always ends region_end = len(text) if is_unterminated else end_match.end() if region_end == region_start: # an empty match is no region: try this rule's next match instead if region_start < len(text): upcoming_starts[rule_index] = rule.start_pattern.search(text, region_start + 1) else: # at the very end, where re would match again: no further match upcoming_starts[rule_index] = None continue if is_unterminated: # cut to the end of the file line_number = text.count(b\"\\n\", 0, region_start) + 1 messages.warning(f\"{path}: {line_number}: no end found for {rule.description}; cut to end of file\") if rule.action == RegionAction.KEEP: # string literal: kept as it is pieces.append(text[position:region_end]) else: # RegionAction.CUT: only its newlines remain pieces.append(text[position:region_start]) pieces.append(b\"\\n\" * text.count(b\"\\n\", region_start, region_end)) position = region_end return b\"\".join(pieces)class Corpus: \"\"\"All scanned files as one sequence of interned, normalised, non-blank, non-ignored lines. A position is an index into line_ids. Every file is preceded and followed by a sentinel: a negative id unique to that file boundary. Hence no window or block spans two files, copies in different files never overlap, and a copy at the start or end of a file has a neighbour that matches nothing. \"\"\" def __init__(self, ignore_patterns, region_rules): self.ignore_patterns = ignore_patterns self.region_rules = region_rules self.line_ids = array(\"q\") self.line_numbers = array(\"q\") # original 1-based line number; sentinel slots hold 0 and are never read self.file_paths = [] self.file_start_positions = [] self.line_texts = [] # line id -> normalised text self.line_counts = [] # line id -> occurrences in the corpus self.line_id_by_text = {} self.code_line_count = 0 # non-blank lines in all files, ignored ones included self.sentinel_count = 0 self._append_sentinel() def _append_sentinel(self): \"\"\"Append a file boundary that differs from every line and every other boundary.\"\"\" self.sentinel_count += 1 self.line_ids.append(-self.sentinel_count) self.line_numbers.append(0) def add_file(self, path, content): \"\"\"Apply the cuts, normalise the lines of a file and append the non-blank, non-ignored ones.\"\"\" self.file_paths.append(path) self.file_start_positions.append(len(self.line_ids)) text = content.replace(b\"\\r\\n\", b\"\\n\").replace(b\"\\r\", b\"\\n\") lines = apply_cuts(text, self.region_rules, path).split(b\"\\n\") for line_index, raw_line in enumerate(lines): # intern each non-blank, non-ignored line words = raw_line.split() if not words: # blank line: not part of any block continue text = b\" \".join(words) self.code_line_count += 1 if any(pattern.search(text) for pattern in self.ignore_patterns): # ignored: counted, but not part of any block continue if text in self.line_id_by_text: # known line line_id = self.line_id_by_text[text] else: # first occurrence line_id = len(self.line_texts) self.line_id_by_text[text] = line_id self.line_texts.append(text) self.line_counts.append(0) self.line_counts[line_id] += 1 self.line_ids.append(line_id) self.line_numbers.append(line_index + 1) self._append_sentinel() def location(self, position): \"\"\"Return (path, original line number) of a position.\"\"\" file_index = bisect.bisect_right(self.file_start_positions, position) - 1 return self.file_paths[file_index], self.line_numbers[position]def build_corpus(source_files, ignore_patterns, region_rules): \"\"\"Read and normalise all source files, skipping unreadable and binary ones.\"\"\" corpus = Corpus(ignore_patterns, region_rules) for path, content in read_source_files(source_files, messages): corpus.add_file(path, content) return corpusdef find_window_groups(corpus, length): \"\"\"Group the start positions of all windows of `length` lines by content; return groups of 2+.\"\"\" line_ids = corpus.line_ids line_counts = corpus.line_counts windows = {} run_length = 0 # consecutive lines, ending here, that occur more than once in the corpus for position, line_id in enumerate(line_ids): # slide the window if line_id < 0 or line_counts[line_id] < 2: # sentinel or unique line: no repeated window spans it run_length = 0 continue run_length += 1 if run_length >= length: # a full window of repeatable lines ends here start = position - length + 1 key = tuple(line_ids[start:position + 1]) if key in windows: # seen before windows[key].append(start) else: # first time windows[key] = [start] return [starts for starts in windows.values() if len(starts) >= 2]def non_overlapping_starts(starts, block_length): \"\"\"Return the copies that remain when overlapping ones are dropped, earliest first.\"\"\" kept_starts = [starts[0]] for start in starts[1:]: # keep a copy only if it begins after the last kept one ends if start >= kept_starts[-1] + block_length: kept_starts.append(start) return kept_startsdef has_common_predecessor(line_ids, starts): \"\"\"True if all copies are preceded by the same line, i.e. the block extends leftwards.\"\"\" predecessor = line_ids[starts[0] - 1] return all(line_ids[start - 1] == predecessor for start in starts)def group_by_next_line(line_ids, starts, block_length): \"\"\"Partition copies by the line that follows them, keeping position order.\"\"\" groups = {} for start in starts: # copies at the end of a file meet a unique sentinel and stay alone next_line_id = line_ids[start + block_length] if next_line_id in groups: # seen before groups[next_line_id].append(start) else: # first time groups[next_line_id] = [start] return groups.values()def find_maximal_blocks(line_ids, window_group, length): \"\"\"Return (kept starts, block length) of every maximal repeated block beginning with this window group. The window group is extended line by line; copies split off where their next lines differ. A block is reported when it cannot be extended leftwards (no common predecessor) nor rightwards without losing copies. \"\"\" blocks = [] pending = [(window_group, length, non_overlapping_starts(window_group, length))] while pending: # one candidate block per iteration starts, block_length, kept_starts = pending.pop() if len(kept_starts) < 2: # overlapping copies of a single block: no repeat continue if has_common_predecessor(line_ids, starts): # this block and all its extensions are found from an earlier window continue is_right_maximal = True for continuation in group_by_next_line(line_ids, starts, block_length): # try each longer block if len(continuation) < 2: # a single copy is no repeat continue continuation_kept = non_overlapping_starts(continuation, block_length + 1) if len(continuation_kept) == len(kept_starts): # longer block with as many copies is_right_maximal = False pending.append((continuation, block_length + 1, continuation_kept)) if is_right_maximal: # report this block blocks.append((kept_starts, block_length)) return blocksdef find_repeated_blocks(corpus, length): \"\"\"Return every maximal block of at least `length` lines that occurs more than once.\"\"\" blocks = [] for window_group in find_window_groups(corpus, length): # each block begins with exactly one window group blocks.extend(find_maximal_blocks(corpus.line_ids, window_group, length)) return blocksdef build_reports(corpus, blocks, report_order): \"\"\"Turn blocks into reports, most important first in the given order, then by path and line.\"\"\" reports = [] for kept_starts, block_length in blocks: # order each block's copies by file location starts = sorted(kept_starts, key=corpus.location) locations = [corpus.location(start) for start in starts] reports.append(Report(len(starts), starts, locations, block_length)) if report_order == ReportOrder.LONGEST: # longest blocks first, then most copies reports.sort(key=lambda report: (-report.block_length, -report.count, report.locations[0])) else: # ReportOrder.MOST_COPIES: most copies first, then longest blocks reports.sort(key=lambda report: (-report.count, report.locations[0], -report.block_length)) return reportsdef write_report(output, corpus, length, reports, verbose): \"\"\"Write the report for one length.\"\"\" for report in reports: # header, optional text, one line per copy output.write(f\"DRY (level {length}) {report.count} lines={report.block_length}\\n\".encode()) if verbose: # the normalised text of the block first_start = report.starts[0] for position in range(first_start, first_start + report.block_length): output.write(corpus.line_texts[corpus.line_ids[position]] + b\"\\n\") for path, line_number in report.locations: output.write(os.fsencode(path) + f\": {line_number}\\n\".encode())def count_surplus_lines(corpus, reports): \"\"\"Count the lines in copies beyond the first of each report; a line in several reports counts once.\"\"\" is_surplus = bytearray(len(corpus.line_ids)) for report in reports: # the first listed copy is taken as the original for start in report.starts[1:]: is_surplus[start:start + report.block_length] = b\"\\1\" * report.block_length return is_surplus.count(1)def write_summary(output, code_line_count, statistics): \"\"\"Write, per length, the number of DRYs and of DRY lines per 1000 lines of code.\"\"\" output.write(f\"Summary: {code_line_count} lines of code (non-blank)\\n\".encode()) for entry in statistics: # one line per length if code_line_count > 0: # the rate is defined rate = f\"{1000 * entry.surplus_line_count / code_line_count:.1f}\" else: # nothing scanned: no rate rate = \"undefined\" output.write((f\"Summary (level {entry.length}): DRYs: {entry.report_count}, \" f\"DRY lines: {entry.surplus_line_count}, per 1000 lines of code: {rate}\\n\").encode())def main(argv): \"\"\"Scan the given paths and report repeated blocks; return the exit status.\"\"\" install_default_signal_handlers() settings = parse_arguments(argv) all_valid, checked_arguments = check_arguments(settings.paths, messages) if not all_valid: # a command-line path is missing or unusable return EXIT_ERROR collector = SourceCollector( lambda name: settings.accept_all_suffixes or name.endswith(settings.suffixes), messages) for path, path_stat in checked_arguments: collector.add_argument(path, path_stat) if len(collector.source_files) == 0: # say so rather than silently report no duplicates messages.warning(\"no files to scan\") corpus = build_corpus(collector.source_files, settings.ignore_patterns, settings.region_rules) output = sys.stdout.buffer found_duplicates = False statistics = [] for length in settings.lengths: # one report per requested length, in command-line order reports = build_reports(corpus, find_repeated_blocks(corpus, length), settings.report_order) shown_reports = select_shown_reports( reports, settings.limit_report_size, settings.max_report_size, settings.reverse_report) write_report(output, corpus, length, shown_reports, settings.verbose) if settings.summary: # gather the totals for the summary statistics.append(LengthStatistics(length, len(reports), count_surplus_lines(corpus, reports))) found_duplicates = found_duplicates or len(reports) > 0 if settings.summary: # totals after all reports write_summary(output, corpus.code_line_count, statistics) output.flush() return EXIT_DUPLICATES_FOUND if found_duplicates else EXIT_NO_DUPLICATESif __name__ == \"__main__\": sys.exit(main(sys.argv[1:]))\n```\n\n### Addressing the complexity\n\nThen it was time to address the biggest issue: complexity. To attack this I first had Claude create my own tool that measures the complexity:\n\n``` bash\n#!/usr/bin/env python3# MIT License## Copyright (c) 2026 Gerben Wierda## Permission is hereby granted, free of charge, to any person obtaining a copy# of this software and associated documentation files (the \"Software\"), to deal# in the Software without restriction, including without limitation the rights# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell# copies of the Software, and to permit persons to whom the Software is# furnished to do so, subject to the following conditions:## The above copyright notice and this permission notice shall be included in all# copies or substantial portions of the Software.## THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE# SOFTWARE.\"\"\"rna-findcompl - find and report structured-programming issues in source files.Usage: rna-findcompl.py [--all] [--CHECK] [--CHECK-threshold=N] [--CHECK-penalty=M] ... [--quality-reference=P] [--override EXTENSION GRAMMAR ...] [--verbose] [--worst-first | --highest-score-first] [--reverse-report] [--max-report=N] [--summary] PATH ... rna-findcompl.py --usage rna-findcompl.py --versionCHECK: too-long-routines, too-long-flows, too-deep or too-complex.Each file is read with the grammar for its extension, which finds itsroutines, the flows (blocks) within them, their nesting depth and their flowcontrol statements. Every excess over a check's threshold is scored with thecheck's penalty; a file's penalty score averages the checks' scores per 1000lines, and penalty scores are also expressed as a quality from 10 (perfect)to 1.\"\"\"import argparseimport enumimport osimport reimport sysfrom collections import namedtuplefrom decimal import ROUND_HALF_UP, Decimal# sourcefiles.py, embedded by install.py: registered as module sourcefiles for the import belowimport sysimport typesdef _register_embedded_module(module_name, source): \"\"\"Run an embedded module's source and register it, so that importing it finds this copy.\"\"\" module = types.ModuleType(module_name) exec(compile(source, f\"\", \"exec\"), module.__dict__) sys.modules[module_name] = module_register_embedded_module('sourcefiles', ( b'\"\"\"sourcefiles - parts shared by rna-finddry and rna-findcompl.\\n' b'\\n' b'Messages, the --usage action, command-line path checks, collecting and reading\\n' b'the source files, and limiting a report. install.py embeds this module in the\\n' b'installed programs, so that they run without it.\\n' b'\"\"\"\\n' b'\\n' b'import argparse\\n' b'import os\\n' b'import signal\\n' b'import stat\\n' b'import sys\\n' b'\\n' b'MODULE_VERSION = 1 # started at 1; increase by one on every change to this file\\n' b'UNRELEASED = \"unreleased\" # a program\\'s RELEASE_DATE when run from the source tree; install.py sets the real date\\n' b'EXIT_ERROR = 2 # also the status argparse uses for command-line errors\\n' b'\\n' b'FILE_COLLECTION_MANUAL = \"\"\"\\\\\\n' b' Directories are scanned recursively, following symlinks. Hidden directories\\n' b' (name starting with \".\") and binary files (containing a NUL byte) are\\n' b' skipped. A file or directory reached by several paths is scanned once.\\n' b'\"\"\"\\n' b'\\n' b'\\n' b'class MessageReporter:\\n' b' \"\"\"Writes warnings and errors on stderr, prefixed by the program name.\"\"\"\\n' b'\\n' b' def __init__(self, program_name):\\n' b' self.program_name = program_name\\n' b'\\n' b' def warning(self, message):\\n' b' \"\"\"Print a warning on stderr.\"\"\"\\n' b' print(f\"{self.program_name}: warning: {message}\", file=sys.stderr)\\n' b'\\n' b' def error(self, message):\\n' b' \"\"\"Print an error on stderr.\"\"\"\\n' b' print(f\"{self.program_name}: error: {message}\", file=sys.stderr)\\n' b'\\n' b'\\n' b'class ManualAction(argparse.Action):\\n' b' \"\"\"argparse action for --usage: print the help followed by the manual (keyword `manual`), then exit.\"\"\"\\n' b'\\n' b' def __init__(self, option_strings, manual, dest=argparse.SUPPRESS, default=argparse.SUPPRESS, help=None):\\n' b' super().__init__(option_strings=option_strings, dest=dest, default=default, nargs=0, help=help)\\n' b' self.manual = manual\\n' b'\\n' b' def __call__(self, parser, namespace, values, option_string=None):\\n' b' \"\"\"Print and exit successfully; runs before required arguments are checked, like -h.\"\"\"\\n' b' sys.stdout.write(parser.format_help() + self.manual)\\n' b' parser.exit()\\n' b'\\n' b'\\n' b'def version_text(program_name, program_version, release_date):\\n' b' \"\"\"Return the --version text; release_date is an installation date (yyyy-mm-dd) or UNRELEASED.\"\"\"\\n' b' if release_date == UNRELEASED: # run from the source tree, not installed\\n' b' return f\"{program_name} version {program_version}, unreleased\"\\n' b' return f\"{program_name} version {program_version}, released on {release_date}\"\\n' b'\\n' b'\\n' b'def install_default_signal_handlers():\\n' b' \"\"\"End quietly on a closed pipe and on Ctrl-C, as a C program does, instead of with a traceback.\"\"\"\\n' b' signal.signal(signal.SIGPIPE, signal.SIG_DFL)\\n' b' signal.signal(signal.SIGINT, signal.SIG_DFL)\\n' b'\\n' b'\\n' b'def check_max_report(parser, max_report):\\n' b' \"\"\"Validate --max-report (argparse\\'s None: not given); return (limit_report_size, max_report_size).\"\"\"\\n' b' limit_report_size = max_report is not None\\n' b' max_report_size = 0 # read only when limit_report_size\\n' b' if limit_report_size: # a limit was given: it must be positive\\n' b' if max_report < 1:\\n' b' parser.error(f\"--max-report must be at least 1, got {max_report}\")\\n' b' max_report_size = max_report\\n' b' return limit_report_size, max_report_size\\n' b'\\n' b'\\n' b'def check_arguments(paths, messages):\\n' b' \"\"\"Stat every command-line path; return (all valid, [(path, stat)]) and report each invalid one.\"\"\"\\n' b' all_valid = True\\n' b' checked_arguments = []\\n' b' for path in paths: # must exist and be a directory or regular file\\n' b' try:\\n' b' path_stat = os.stat(path)\\n' b' except OSError as failure:\\n' b' messages.error(f\"cannot access {path}: {failure.strerror}\")\\n' b' all_valid = False\\n' b' continue\\n' b' if not (stat.S_ISDIR(path_stat.st_mode) or stat.S_ISREG(path_stat.st_mode)):\\n' b' messages.error(f\"{path} is neither a file nor a directory\")\\n' b' all_valid = False\\n' b' continue\\n' b' checked_arguments.append((path, path_stat))\\n' b' return all_valid, checked_arguments\\n' b'\\n' b'\\n' b'class SourceCollector:\\n' b' \"\"\"Collects the files to scan; follows symlinks and visits every directory and file once.\\n' b'\\n' b' is_wanted_file_name(name) selects the files found while scanning a\\n' b' directory; files named on the command line are always collected.\\n' b' \"\"\"\\n' b'\\n' b' def __init__(self, is_wanted_file_name, messages):\\n' b' self.is_wanted_file_name = is_wanted_file_name\\n' b' self.messages = messages\\n' b' self.visited_directories = set() # (device, inode)\\n' b' self.visited_files = set() # (device, inode)\\n' b' self.source_files = [] # (path, named on command line)\\n' b'\\n' b' def add_argument(self, path, path_stat):\\n' b' \"\"\"Add a command-line path; named paths bypass the file-name and hidden-directory filters.\"\"\"\\n' b' if stat.S_ISDIR(path_stat.st_mode): # directory: scan recursively\\n' b' self._walk(path, path_stat)\\n' b' else: # regular file, as ensured by check_arguments\\n' b' self._add_file(path, path_stat, True)\\n' b'\\n' b' def _add_file(self, path, file_stat, is_explicit):\\n' b' \"\"\"Record a file unless it was already reached by another path.\"\"\"\\n' b' identity = (file_stat.st_dev, file_stat.st_ino)\\n' b' if identity in self.visited_files: # hard link, symlink or repeated argument\\n' b' return\\n' b' self.visited_files.add(identity)\\n' b' self.source_files.append((path, is_explicit))\\n' b'\\n' b' def _mark_directory_visited(self, directory_stat):\\n' b' \"\"\"Mark a directory visited; return False if it already was.\"\"\"\\n' b' identity = (directory_stat.st_dev, directory_stat.st_ino)\\n' b' if identity in self.visited_directories: # symlink loop or repeated argument\\n' b' return False\\n' b' self.visited_directories.add(identity)\\n' b' return True\\n' b'\\n' b' def _walk(self, root_path, root_stat):\\n' b' \"\"\"Scan a directory tree depth-first, in name order.\"\"\"\\n' b' if not self._mark_directory_visited(root_stat): # already scanned via another path\\n' b' return\\n' b' pending = [root_path]\\n' b' while pending: # one directory per iteration\\n' b' directory_path = pending.pop()\\n' b' try:\\n' b' with os.scandir(directory_path) as iterator:\\n' b' entries = sorted(iterator, key=lambda entry: entry.name)\\n' b' except OSError as failure:\\n' b' self.messages.warning(f\"cannot read directory {directory_path}: {failure.strerror}\")\\n' b' continue\\n' b' subdirectories = []\\n' b' for entry in entries: # classify each entry, following symlinks\\n' b' try:\\n' b' entry_stat = entry.stat()\\n' b' except OSError as failure:\\n' b' self.messages.warning(f\"cannot access {entry.path}: {failure.strerror}\")\\n' b' continue\\n' b' if stat.S_ISDIR(entry_stat.st_mode): # descend into new, non-hidden directories\\n' b' if not entry.name.startswith(\".\") and self._mark_directory_visited(entry_stat):\\n' b' subdirectories.append(entry.path)\\n' b' elif stat.S_ISREG(entry_stat.st_mode): # regular file: apply the file-name filter\\n' b' if self.is_wanted_file_name(entry.name):\\n' b' self._add_file(entry.path, entry_stat, False)\\n' b' else: # fifo, socket or device: never a source file\\n' b' self.messages.warning(f\"skipping special file {entry.path}\")\\n' b' pending.extend(reversed(subdirectories)) # so that pop() yields them in name order\\n' b'\\n' b'\\n' b'def read_source_files(source_files, messages):\\n' b' \"\"\"Yield (path, content) of each readable, non-binary source file; warn about unreadable ones.\"\"\"\\n' b' for path, is_explicit in source_files: # one file per iteration\\n' b' try:\\n' b' with open(path, \"rb\") as source:\\n' b' content = source.read()\\n' b' except OSError as failure:\\n' b' messages.warning(f\"cannot read {path}: {failure.strerror}\")\\n' b' continue\\n' b' if b\"\\\\0\" in content: # binary file: skipped, noted only when named explicitly\\n' b' if is_explicit:\\n' b' messages.warning(f\"skipping binary file {path}\")\\n' b' continue\\n' b' yield path, content\\n' b'\\n' b'\\n' b'def select_shown_reports(reports, limit_report_size, max_report_size, reverse_report):\\n' b' \"\"\"Return the reports to list: the most important ones if limited, reversed if asked.\"\"\"\\n' b' shown_reports = reports\\n' b' if limit_report_size: # keep only the most important\\n' b' shown_reports = shown_reports[:max_report_size]\\n' b' if reverse_report: # most important last\\n' b' shown_reports = shown_reports[::-1]\\n' b' return shown_reports\\n'))from sourcefiles import (EXIT_ERROR, FILE_COLLECTION_MANUAL, UNRELEASED, ManualAction, MessageReporter, SourceCollector, check_arguments, check_max_report, install_default_signal_handlers, read_source_files, select_shown_reports, version_text)PROGRAM_NAME = \"rna-findcompl\"PROGRAM_VERSION = 1 # started at 1; increase by one on every change to this fileRELEASE_DATE = '2026-10-25' # set by install.pyEXIT_NO_ISSUES = 0EXIT_ISSUES_FOUND = 1THRESHOLD_PATTERN = re.compile(r\"[0-9]{1,9}\")POSITIVE_NUMBER_PATTERN = re.compile(r\"[0-9]+(\\.[0-9]*)?|\\.[0-9]+\")RATE_QUANTUM = Decimal(\"0.1\") # rates, penalty scores, qualities and percentages are shown with one decimalBEST_QUALITY = Decimal(10)QUALITY_RANGE = Decimal(9) # from the best quality, 10, down to the worst, 1NO_ROUTINE_NAME = b\"\" # in classify results that are not a routine, where the name is never readUNNAMED_ROUTINE = b\"(unnamed)\"STRING_TOKEN = b'\"\"' # string and character literals and numbers are reduced to a fixed tokenCHARACTER_TOKEN = b\"''\"NUMBER_TOKEN = b\"0\"BRACED_GROUP_TOKEN = b\"{}\" # stands in a header for a skipped initialiser {...}IDENTIFIER_START_BYTES = frozenset( b\"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_$\" + bytes(range(0x80, 0x100)))TYPE_KEYWORDS = frozenset({b\"class\", b\"struct\", b\"union\", b\"enum\", b\"namespace\", b\"typedef\"})QUALIFIERS_WITH_ARGUMENTS = frozenset({b\"noexcept\", b\"throw\", b\"requires\", b\"alignas\"})INITIALIZER_QUALIFIERS = frozenset({b\"noexcept\", b\"throw\", b\"try\"}) # may stand between ) and : of a constructorATTRIBUTE_KEYWORDS = frozenset({b\"__attribute__\", b\"__declspec\"})ACCESS_SPECIFIERS = frozenset({b\"public\", b\"protected\", b\"private\", b\"signals\", b\"slots\", b\"Q_SIGNALS\", b\"Q_SLOTS\"})BLOCK_OPENING_TOKENS = frozenset({b\")\", b\"]\", b\"^\"}) # a { after these, if no control body: lambda, block, macro blockSTATEMENT_END_TOKENS = frozenset({b\";\", b\"{\", b\"}\", b\":\"}) # a { after these starts a statement: a scope blockTRY_KEYWORDS = frozenset({b\"try\", b\"@try\"})HANDLER_KEYWORDS = frozenset({b\"catch\", b\"@catch\", b\"@finally\"}) # continue a try statementJUMP_KEYWORDS = frozenset({b\"continue\", b\"break\", b\"return\", b\"throw\", b\"goto\", b\"@throw\", b\"co_return\"})OBJECTIVE_C_SECTION_KEYWORDS = frozenset({b\"@interface\", b\"@implementation\", b\"@protocol\", b\"@end\"})messages = MessageReporter(PROGRAM_NAME)class CheckKind(enum.Enum): \"\"\"The structured-programming issues, in report order; the value is the option name without dashes.\"\"\" TOO_LONG_ROUTINES = \"too-long-routines\" TOO_LONG_FLOWS = \"too-long-flows\" TOO_DEEP = \"too-deep\" TOO_COMPLEX = \"too-complex\"class ReportOrder(enum.Enum): \"\"\"Order of the file reports, most important first.\"\"\" PATH = \"by path\" WORST = \"highest penalty score first\" HIGHEST_SCORE = \"highest total score first\"class FrameKind(enum.Enum): \"\"\"What an open {, body or statement belongs to.\"\"\" CONTAINER = \"namespace, class or extern block, which may hold routines\" DATA = \"initialiser or type body outside routines\" ROUTINE = \"routine body\" BLOCK = \"braced flow inside a routine: a control body, lambda, block or macro block\" BRACELESS = \"control body without braces inside a routine: a flow of one statement\" SCOPE = \"block after a label, or bare block, inside a routine: only a scope, no flow\" INITIALIZER = \"initialiser list inside a routine\" CONTROL = \"control statement inside a routine, from its keyword to the end of its last body\"class ControlPhase(enum.Enum): \"\"\"Where a control statement is, i.e. what comes next.\"\"\" HEADER = \"before or in its (...) header\" BODY_EXPECTED = \"the next token starts its body\" IN_BODY = \"in its body\" ELSE_EXPECTED = \"its if body ended and the next token is else\" ELSE_IF_EXPECTED = \"else is followed by if, which continues this statement on its level\" HANDLER_EXPECTED = \"its try or catch body ended and the next token is a catch or @finally\" WHILE_EXPECTED = \"its do body ended: the while of do ... while comes next\" DO_CONDITION = \"before or in the (...) after the while of do ... while\" DO_END_EXPECTED = \"after the condition of do ... while: its ; comes next\"CHECK_ORDER = {kind: index for index, kind in enumerate(CheckKind)}DEFAULT_QUALITY_REFERENCE = Decimal(350) # the penalty score that counts as quality 1DEFAULT_THRESHOLDS = {CheckKind.TOO_LONG_ROUTINES: 60, CheckKind.TOO_LONG_FLOWS: 40, CheckKind.TOO_DEEP: 3, CheckKind.TOO_COMPLEX: 15}DEFAULT_PENALTIES = {CheckKind.TOO_LONG_ROUTINES: Decimal(1), CheckKind.TOO_LONG_FLOWS: Decimal(1), CheckKind.TOO_DEEP: Decimal(10), CheckKind.TOO_COMPLEX: Decimal(3)}EXCESS_UNITS = {CheckKind.TOO_LONG_ROUTINES: \"lines\", CheckKind.TOO_LONG_FLOWS: \"lines\", CheckKind.TOO_DEEP: \"levels\", CheckKind.TOO_COMPLEX: \"statements\"}Check = namedtuple(\"Check\", \"kind threshold penalty\") # threshold: int, penalty: DecimalSettings = namedtuple( \"Settings\", \"checks quality_reference grammar_by_extension verbose report_order reverse_report limit_report_size\" \" max_report_size summary paths\")Token = namedtuple(\"Token\", \"text line starts_line\")UnbalancedBranch = namedtuple(\"UnbalancedBranch\", \"line brace_balance\") # brace_balance: { minus }LexResult = namedtuple( \"LexResult\", \"is_valid problem tokens comment_byte_count unbalanced_branches\") # problem read only when not validFileAnalysis = namedtuple( \"FileAnalysis\", \"is_analysable problem unbalanced_branches routines blocks comment_byte_count\") # problem: when not analysableRoutine = namedtuple(\"Routine\", \"name start_line end_line flow_control_count\")Block = namedtuple(\"Block\", \"routine_name start_line end_line depth\")Issue = namedtuple(\"Issue\", \"line kind routine_name measure excess score\")CheckTotal = namedtuple(\"CheckTotal\", \"issue_count excess score\")FileReport = namedtuple( \"FileReport\", \"path is_analysed line_count check_totals issues problem unbalanced_branches\") # the last two: when not analysedMANUAL = \"\"\"details:grammars The grammar of a file follows from its extension: c .c c++ .cc .cpp .cxx .c++ .hh .hpp .hxx objc .m objc++ .mm .h (a header may be C, C++ or Objective-C: the superset) --override EXTENSION GRAMMAR changes or adds one (e.g. --override .inl c++). c++ and objc extend c; objc++ combines them. Comments (// and /* */), string and character literals and preprocessor lines are ignored for detection. Of each #if, #ifdef or #ifndef ... #endif only one branch is read: the first that is not dead code (condition 0 or false, e.g. #if 0); conditions are not evaluated otherwise. A file whose braces or conditionals do not balance, or with an unterminated comment, fails: it is reported, but not scored.structure A routine is a {} block outside other routines whose header ends in a parameter list (a function, constructor or operator, or a lambda assigned outside routines), or, in objc, a method: - or + at the start of a line. A flow is the body of a control statement inside a routine, with or without braces (without: the one statement that follows): if, else, for, while, do, switch, in c++ try, catch, in objc @try, @catch, @finally, @synchronized, @autoreleasepool. A {} block after ), ] or ^ that is no such body (a lambda, block or macro block) is a flow too. Not flows: a control body that is empty or a single continue, break, return, throw or goto (a guard clause); a block after a label (case 1: {...}) or a bare block, which only scopes variables; initialiser lists and type bodies. The routine body is depth 0, a flow directly inside it depth 1, and so on; the bodies of \"else if\" stay on the level of their if, so \"if (a) if (b) s;\" reaches depth 2 but an else-if chain stays at 1. Flow control statements are if, else, for, while, do, switch, case, default, goto, ?: and in c++ try, catch, in objc @try, @catch; \"else if\" and \"do ... while\" count once.lines Lines are non-blank lines, comment lines included. A routine or flow runs from the line of its { through the line of its }; a flow without braces from the line of its first token through that of its last.checks --all makes all four checks, --CHECK one; --CHECK-threshold N or --CHECK-penalty M also make it. At least one check is required. CHECK measure N M too-long-routines lines of a routine 60 1 too-long-flows lines of a flow 40 1 too-deep depth of a flow 3 10 too-complex flow control statements of a routine 15 3 (N and M: the default threshold and penalty.) A measure up to and including the threshold is perfect. Above it, the excess is the raw number: the lines, levels or statements beyond the threshold. Every flow counts on its own: with threshold 3, a flow at depth 5 has excess 2 and the flow at depth 4 around it excess 1. The score of an issue is its excess times the penalty. N is a whole number (0 or more), M a number above 0, fractions allowed.scores Per file and check, excesses and scores are added up. The penalty score of a file is the average, over the checks made, of their score per 1000 lines of the file. That of all files together, in FINDCOMPL, is the same over all lines: the average of the file penalty scores weighted by file length. A penalty score is also expressed as a quality from 10 (penalty 0: perfect) down to 1 (penalty P or more; P is 350 unless --quality-reference P): quality = 10 - 9 * min(penalty / P, 1)files\"\"\" + FILE_COLLECTION_MANUAL + \"\"\" Only files with a grammar are scanned; a file named on the command line without one is skipped with a warning.output For each file (PENALTY and QUALITY are undefined for a file without lines): PATH: penalty PENALTY, lines LINES, quality QUALITY with --verbose followed by its totals per check that has an excess, and by its issues, in line order: PATH: CHECK: excess EXCESS UNIT, score SCORE, RATE per 1000 lines PATH: LINE: CHECK ROUTINE: MEASURE (threshold N, excess E, score S) LINE is where the routine or flow starts: its { or, without braces, its first token. A file that failed: PATH: failed with --verbose followed by where the failure showed and, if found, the first conditional branch read whose braces do not balance on their own: PATH: LINE: PROBLEM PATH: LINE: likely cause: the braces of this conditional branch do not ... After all files: FINDCOMPL: penalty PENALTY, lines LINES, quality QUALITY Files are ordered by path (default), by penalty score (--worst-first) or by total score, not per lines (--highest-score-first), highest first, then by path; failed files come last. --max-report N lists only the first N files; --reverse-report lists them in reverse order, so the most important come last. FINDCOMPL, --summary and the exit status always count all files; failed files add no score and no lines.summary (--summary) After FINDCOMPL, per check made, over all files: CHECK: issues ISSUES, excess EXCESS UNIT, score SCORE, RATE per 1000 lines the share of comments in the files, after line endings are unified: Comments ignored: BYTES out of TOTAL: PERCENTAGE and the files that failed: failed files: COUNT failed: PATH one per failed fileexit status 0 no excess 1 excess found 2 error: invalid argument, or a path that is missing or neither file nor directory Unreadable files, and binary files named on the command line, are skipped with a warning on stderr.\"\"\"def is_identifier(text): \"\"\"True if a token is a word: an identifier or keyword.\"\"\" return len(text) > 0 and text[0] in IDENTIFIER_START_BYTESdef skip_group(texts, open_index, open_text, close_text): \"\"\"Return the index after the group opened at open_index, or len(texts) if it is not closed.\"\"\" depth = 0 for index in range(open_index, len(texts)): # find the matching close if texts[index] == open_text: depth += 1 elif texts[index] == close_text: depth -= 1 if depth == 0: # matched return index + 1 return len(texts)def skip_group_backwards(texts, close_index, open_text, close_text): \"\"\"Return the index of the opener of the group closed at close_index, or -1 if it is not opened.\"\"\" depth = 0 for index in range(close_index, -1, -1): # find the matching open if texts[index] == close_text: depth += 1 elif texts[index] == open_text: depth -= 1 if depth == 0: # matched return index return -1def strip_attributes(texts): \"\"\"Remove __attribute__((...)), __declspec(...) and [[...]], which may stand anywhere in a header.\"\"\" stripped = [] index = 0 while index < len(texts): # copy all but the attributes text = texts[index] if text in ATTRIBUTE_KEYWORDS and index + 1 < len(texts) and texts[index + 1] == b\"(\": index = skip_group(texts, index + 1, b\"(\", b\")\") elif text == b\"[\" and index + 1 < len(texts) and texts[index + 1] == b\"[\": index = skip_group(texts, index, b\"[\", b\"]\") else: # part of the declaration stripped.append(text) index += 1 return strippeddef top_level_parenthesis_groups(texts): \"\"\"Return (open index, close index) of every outermost (...) group, in order.\"\"\" groups = [] depth = 0 open_index = 0 # set at each outermost ( for index, text in enumerate(texts): # track the nesting if text == b\"(\": if depth == 0: # an outermost group starts open_index = index depth += 1 elif text == b\")\" and depth > 0: depth -= 1 if depth == 0: # an outermost group ends groups.append((open_index, index)) return groupsdef scan_routine_tail(texts, start): \"\"\"Check what follows a parameter list; return (is a routine's, has constructor member initialisers). Allowed: qualifiers and attribute-like macros (const, override, noexcept(...), Q_DECL_OVERRIDE), & and &&, then optionally a trailing return type (->) or, right after the ) and noexcept/throw/try, member initialisers (:). \"\"\" allows_initializers = True index = start while index < len(texts): # one qualifier per iteration text = texts[index] if text == b\"->\": # trailing return type: anything may follow return True, False if text == b\":\": # member initialisers, only right after the parameters return allows_initializers, allows_initializers if text == b\"&\": # reference qualifier index += 1 continue if not is_identifier(text) or text in TYPE_KEYWORDS: # cannot follow a routine's parameters return False, False if text not in INITIALIZER_QUALIFIERS: # e.g. const: no member initialisers after it allows_initializers = False index += 1 if text in QUALIFIERS_WITH_ARGUMENTS and index < len(texts) and texts[index] == b\"(\": index = skip_group(texts, index, b\"(\", b\")\") return True, Falsedef join_name_tokens(texts): \"\"\"Join tokens into a name, with a space only between two words, e.g. b\"operator const char*\".\"\"\" name = b\"\" previous_is_word = False for text in texts: # insert spaces between words only is_word = is_identifier(text) if is_word and previous_is_word: name += b\" \" name += text previous_is_word = is_word return namedef qualify_name(texts, start, name): \"\"\"Prefix name, which starts at texts[start], with its scopes: A::B::name (template arguments dropped).\"\"\" while start >= 2 and texts[start - 1] == b\"::\": # one scope per iteration scope_index = start - 2 if texts[scope_index] == b\">\": # A::name scope_index = skip_group_backwards(texts, scope_index, b\"<\", b\">\") - 1 if scope_index < 0 or not is_identifier(texts[scope_index]): # e.g. a leading :: for the global scope break name = texts[scope_index] + b\"::\" + name start = scope_index return namedef routine_name(texts, open_index): \"\"\"Name a routine from the tokens before its parameter list at open_index.\"\"\" name_end = open_index - 1 if name_end < 0: # nothing before the parameters return UNNAMED_ROUTINE last_text = texts[name_end] if last_text in (b\"]\", b\"^\"): # a lambda or block: named after the variable it is assigned to for index in range(name_end - 1, 0, -1): if texts[index] == b\"=\" and is_identifier(texts[index - 1]): return texts[index - 1] return b\"lambda\" if last_text == b\"]\" else b\"block\" if last_text == b\">\": # a template-id, e.g. a specialisation f arguments_start = skip_group_backwards(texts, name_end, b\"<\", b\">\") if arguments_start > 0 and is_identifier(texts[arguments_start - 1]): return qualify_name(texts, arguments_start - 1, texts[arguments_start - 1] + join_name_tokens(texts[arguments_start:name_end + 1])) for index in range(name_end, max(name_end - 4, 0) - 1, -1): # operator==, operator(), operator bool if texts[index] == b\"operator\": return qualify_name(texts, index, join_name_tokens(texts[index:name_end + 1])) if is_identifier(last_text): # a plain or qualified name, or a destructor if name_end > 0 and texts[name_end - 1] == b\"~\": return qualify_name(texts, name_end - 1, b\"~\" + last_text) return qualify_name(texts, name_end, last_text) return UNNAMED_ROUTINEdef is_container_header(texts): \"\"\"True if a { after this header opens a namespace, class, struct, union or extern \"C\" block.\"\"\" if b\"namespace\" in texts: # namespace return True if texts[-2:] == [b\"extern\", STRING_TOKEN]: # extern \"C\" { return True if b\"enum\" in texts or b\"=\" in texts: # an enum body or initialiser is data return False return any(text in (b\"class\", b\"struct\", b\"union\") for text in texts)def is_constant_false(condition): \"\"\"True if an #if or #elif condition is 0 or false, possibly parenthesised: the branch is dead code.\"\"\" while len(condition) >= 2 and condition[0] == b\"(\" and skip_group(condition, 0, b\"(\", b\")\") == len(condition): condition = condition[1:-1] return condition == [b\"0\"] or condition == [b\"false\"]class ConditionalBranches: \"\"\"Which lines the preprocessor conditionals let through. Of each #if/#ifdef/#ifndef ... #endif only one branch is read: the first that is not dead code (#if 0, #elif false). Its value is not evaluated otherwise, so that braces opened in several branches do not add up. \"\"\" def __init__(self): self.is_active = True # the current line is read self.open_conditionals = [] # OpenConditional, innermost last self.unbalanced_branches = [] # UnbalancedBranch per read branch whose own braces do not balance def count_brace(self, text): \"\"\"Count a { or } read in the innermost conditional branch.\"\"\" if len(self.open_conditionals) > 0: # inside a conditional self.open_conditionals[-1].brace_balance += 1 if text == b\"{\" else -1 def end_branch(self, conditional): \"\"\"Note the branch that ends here if its braces do not balance: a likely cause of a failure.\"\"\" if conditional.brace_balance != 0: self.unbalanced_branches.append(UnbalancedBranch(conditional.branch_line, conditional.brace_balance)) def apply(self, name, arguments, line): \"\"\"Apply a directive; return (is valid, problem), problem read only when not valid.\"\"\" if name in (b\"if\", b\"ifdef\", b\"ifndef\"): # a new conditional: this branch is read unless dead is_dead = name == b\"if\" and is_constant_false(arguments) self.open_conditionals.append(OpenConditional(self.is_active, line, not is_dead)) self.is_active = self.is_active and not is_dead elif name in (b\"elif\", b\"elifdef\", b\"elifndef\", b\"else\"): # read only if no branch was chosen yet if len(self.open_conditionals) == 0: return False, f\"{line}: #{name.decode()} without #if\" conditional = self.open_conditionals[-1] self.end_branch(conditional) conditional.branch_line = line conditional.brace_balance = 0 is_dead = name == b\"elif\" and is_constant_false(arguments) if conditional.has_chosen_branch or is_dead: # an earlier branch is read, or this one is dead self.is_active = False else: # the first branch that is not dead conditional.has_chosen_branch = True self.is_active = conditional.is_active_around elif name == b\"endif\": # back to the state around the conditional if len(self.open_conditionals) == 0: return False, f\"{line}: #endif without #if\" conditional = self.open_conditionals.pop() self.end_branch(conditional) self.is_active = conditional.is_active_around return True, \"\"class OpenConditional: \"\"\"An #if whose #endif has not been seen yet.\"\"\" __slots__ = (\"is_active_around\", \"line\", \"has_chosen_branch\", \"branch_line\", \"brace_balance\") def __init__(self, is_active_around, line, has_chosen_branch): self.is_active_around = is_active_around self.line = line self.has_chosen_branch = has_chosen_branch self.branch_line = line # where the current branch starts self.brace_balance = 0 # { minus } read in the current branch, outside nested conditionalsdef strip_standard_attributes(tokens): \"\"\"Remove C++ [[attributes]], which may stand e.g. between a control header and its body. An attribute's inner [...] is directly followed by ], unlike an Objective-C message [[object message] message]; attributes hold no braces. \"\"\" texts = [token.text for token in tokens] kept = [] index = 0 while index < len(tokens): # copy all but the attributes if texts[index] == b\"[\" and index + 1 < len(texts) and texts[index + 1] == b\"[\": inner_end = skip_group(texts, index + 1, b\"[\", b\"]\") if (inner_end < len(texts) and texts[inner_end] == b\"]\" and b\"{\" not in texts[index + 2:inner_end]): # an attribute: skip it index = inner_end + 1 continue kept.append(tokens[index]) index += 1 return keptclass RoutineFrame: \"\"\"An open brace, control body or control statement inside a routine.\"\"\" __slots__ = (\"kind\", \"line\", \"parenthesis_depth\", \"depth\", \"is_control_body\", \"statement_count\", \"first_text\", \"awaiting_first_token\", \"has_inner_flow\", \"keyword\", \"phase\", \"header_depth\", \"header_opened\", \"in_final_body\") def __init__(self, kind, line, parenthesis_depth, depth): self.kind = kind self.line = line self.parenthesis_depth = parenthesis_depth # where it opened self.depth = depth # flow depth of what is directly inside it self.is_control_body = False # a flow that is the body of the control statement below it self.statement_count = 0 # statements directly inside a routine, block or scope self.first_text = b\"\" # first token inside a flow; read only once set self.awaiting_first_token = False self.has_inner_flow = False self.keyword = b\"\" # these five: of a control statement only self.phase = ControlPhase.HEADER self.header_depth = 0 # parenthesis depth around its (...) header self.header_opened = False self.in_final_body = False # in an else or @finally body, after which nothing can followclass RoutineParser: \"\"\"Follows the statements of one routine body: its flows with their depth, and its flow control statements. A flow is the body of a control statement, with or without braces, or a braced block after ) ] ^ that is not (a lambda, block or macro block). A block after a label, or a bare block, only scopes variables: no flow. A control body that is empty or a single jump statement (a guard clause) is no flow either. The bodies of \"else if\" stay on the level of their if. \"\"\" def __init__(self, grammar, name, start_line): self.grammar = grammar self.name = name self.start_line = start_line self.flow_control_count = 0 self.blocks = [] self.frames = [RoutineFrame(FrameKind.ROUTINE, start_line, 0, 0)] self.parenthesis_depth = 0 self.previous_text = b\"{\" self.previous_line = start_line self.line = start_line # of the token being read def feed(self, token, next_text): \"\"\"Read a token (next_text: of the next token, empty at the end); return True at the routine's closing }.\"\"\" text = token.text self.line = token.line top = self.frames[-1] if top.awaiting_first_token: # the first token inside a braced flow top.first_text = text top.awaiting_first_token = False is_do_while = text == b\"while\" and top.kind == FrameKind.CONTROL and top.phase == ControlPhase.WHILE_EXPECTED if (text in self.grammar.flow_control_keywords and not (text == b\"if\" and self.previous_text == b\"else\") and not is_do_while): # else if and do ... while count once self.flow_control_count += 1 if (top.kind == FrameKind.CONTROL and top.phase == ControlPhase.BODY_EXPECTED and text != b\"{\" and text != b\";\"): # a body without braces: the statement that starts here top.phase = ControlPhase.IN_BODY self.open_flow(FrameKind.BRACELESS, token.line, True) self.frames[-1].first_text = text is_routine_end = False if text == b\"(\": self.open_parenthesis() elif text == b\")\": self.close_parenthesis() elif text == b\";\": self.end_simple_statement(next_text) elif text == b\"{\": self.open_brace(token.line) elif text == b\"}\": is_routine_end = self.close_brace(next_text) elif text in self.grammar.control_keywords: self.control_keyword(text, next_text) self.previous_text = text self.previous_line = token.line return is_routine_end def innermost_open_brace_line(self): \"\"\"The line of the innermost { that is still open, possibly the routine's own.\"\"\" for frame in reversed(self.frames): # skip control statements and bodies without braces if frame.kind not in (FrameKind.CONTROL, FrameKind.BRACELESS): return frame.line return self.start_line def open_flow(self, kind, line, is_control_body): \"\"\"Open a flow, one level deeper than where it stands.\"\"\" for frame in reversed(self.frames): # the enclosing flow, if any, now has a flow inside if frame.kind in (FrameKind.BLOCK, FrameKind.BRACELESS): frame.has_inner_flow = True break flow = RoutineFrame(kind, line, self.parenthesis_depth, self.frames[-1].depth + 1) flow.is_control_body = is_control_body flow.awaiting_first_token = kind == FrameKind.BLOCK self.frames.append(flow) def open_parenthesis(self): \"\"\"Count a (; it may start the header of the control statement on top.\"\"\" top = self.frames[-1] if (top.kind == FrameKind.CONTROL and top.phase in (ControlPhase.HEADER, ControlPhase.DO_CONDITION) and not top.header_opened): # the header starts top.header_opened = True top.header_depth = self.parenthesis_depth self.parenthesis_depth += 1 def close_parenthesis(self): \"\"\"Count a ); it may end the header of the control statement on top.\"\"\" self.parenthesis_depth -= 1 top = self.frames[-1] if (top.kind == FrameKind.CONTROL and top.header_opened and self.parenthesis_depth == top.header_depth): # the header ends if top.phase == ControlPhase.HEADER: top.phase = ControlPhase.BODY_EXPECTED elif top.phase == ControlPhase.DO_CONDITION: top.phase = ControlPhase.DO_END_EXPECTED def end_simple_statement(self, next_text): \"\"\"Handle a ;, which ends a statement unless it stands in parentheses or a control header.\"\"\" top = self.frames[-1] if top.kind == FrameKind.CONTROL: # a ; in a control statement if top.phase == ControlPhase.BODY_EXPECTED: # an empty body: no flow top.phase = ControlPhase.IN_BODY self.end_body(next_text) elif top.phase == ControlPhase.DO_END_EXPECTED: # do ... while (...); is complete self.frames.pop() self.end_statement(next_text) return # otherwise in a header: for (;;), if (init; condition) if self.parenthesis_depth != top.parenthesis_depth: # inside parentheses return self.end_statement(next_text) def end_statement(self, next_text): \"\"\"A statement directly inside the top frame has ended.\"\"\" top = self.frames[-1] if top.kind == FrameKind.BRACELESS: # it was the whole body self.frames.pop() self.record_flow(top, self.line) self.end_body(next_text) elif top.kind in (FrameKind.ROUTINE, FrameKind.BLOCK, FrameKind.SCOPE): # one more statement in it top.statement_count += 1 def end_body(self, next_text): \"\"\"The body of the control statement on top has ended: else, a handler or while follows, or it ends.\"\"\" control = self.frames[-1] if control.keyword == b\"do\": # the while of do ... while comes next control.phase = ControlPhase.WHILE_EXPECTED return if not control.in_final_body: # it may go on if control.keyword == b\"if\" and next_text == b\"else\": control.phase = ControlPhase.ELSE_EXPECTED return if control.keyword in TRY_KEYWORDS and next_text in HANDLER_KEYWORDS: control.phase = ControlPhase.HANDLER_EXPECTED return self.frames.pop() self.end_statement(next_text) def open_brace(self, line): \"\"\"Open a control body, lambda, block or macro block (flows), a scope, or an initialiser list.\"\"\" top = self.frames[-1] if top.kind == FrameKind.CONTROL and ( top.phase == ControlPhase.BODY_EXPECTED or (top.phase == ControlPhase.HEADER and not top.header_opened)): # its body top.phase = ControlPhase.IN_BODY self.open_flow(FrameKind.BLOCK, line, True) elif (self.previous_text in BLOCK_OPENING_TOKENS or self.previous_text in self.grammar.lambda_qualifiers): # a lambda, block or macro block self.open_flow(FrameKind.BLOCK, line, False) elif self.parenthesis_depth > top.parenthesis_depth: # an argument: f({1, 2}) self.frames.append(RoutineFrame(FrameKind.INITIALIZER, line, self.parenthesis_depth, top.depth)) elif self.previous_text in STATEMENT_END_TOKENS and top.kind != FrameKind.INITIALIZER: # case 1: { or { { self.frames.append(RoutineFrame(FrameKind.SCOPE, line, self.parenthesis_depth, top.depth)) else: # after =, return or a type name, or inside an initialiser list self.frames.append(RoutineFrame(FrameKind.INITIALIZER, line, self.parenthesis_depth, top.depth)) def close_brace(self, next_text): \"\"\"Close the innermost {; return True if it is the routine's own.\"\"\" while self.frames[-1].kind in (FrameKind.CONTROL, FrameKind.BRACELESS): # a statement left without ; frame = self.frames.pop() if frame.kind == FrameKind.BRACELESS: # its body ends before this brace self.record_flow(frame, self.previous_line) if len(self.frames) == 1: # only the routine is left return True frame = self.frames.pop() self.parenthesis_depth = frame.parenthesis_depth if frame.kind == FrameKind.BLOCK: # a braced flow ends self.record_flow(frame, self.line) if frame.is_control_body: self.end_body(next_text) elif frame.kind == FrameKind.SCOPE: # a block statement ends self.end_statement(next_text) return False def control_keyword(self, text, next_text): \"\"\"Continue the control statement on top (else, else if, a handler, the while of do), or start one.\"\"\" top = self.frames[-1] if top.kind == FrameKind.CONTROL: # possibly a continuation if text == b\"else\" and top.phase == ControlPhase.ELSE_EXPECTED: if next_text == b\"if\": # else if: one statement, one level top.phase = ControlPhase.ELSE_IF_EXPECTED else: # the final else top.phase = ControlPhase.BODY_EXPECTED top.in_final_body = True return if text == b\"if\" and top.phase == ControlPhase.ELSE_IF_EXPECTED: top.phase = ControlPhase.HEADER top.header_opened = False return if text == b\"while\" and top.phase == ControlPhase.WHILE_EXPECTED: top.phase = ControlPhase.DO_CONDITION top.header_opened = False return if text in HANDLER_KEYWORDS and top.phase == ControlPhase.HANDLER_EXPECTED: top.header_opened = False if text == b\"@finally\": # the final handler, without header top.phase = ControlPhase.BODY_EXPECTED top.in_final_body = True else: # catch (...) top.phase = ControlPhase.HEADER return control = RoutineFrame(FrameKind.CONTROL, self.line, self.parenthesis_depth, top.depth) control.keyword = text if text in self.grammar.control_keywords_with_header: # if (...), for (...), ... control.phase = ControlPhase.HEADER else: # else, do, try, ... control.phase = ControlPhase.BODY_EXPECTED self.frames.append(control) def record_flow(self, frame, end_line): \"\"\"Record a flow, unless it is a control body that is empty or a single jump statement (a guard clause).\"\"\" if frame.is_control_body and not frame.has_inner_flow: # possibly a guard clause if frame.kind == FrameKind.BRACELESS: is_guard = frame.first_text in JUMP_KEYWORDS else: # FrameKind.BLOCK is_guard = frame.statement_count == 0 or ( frame.statement_count == 1 and frame.first_text in JUMP_KEYWORDS) if is_guard: return self.blocks.append(Block(self.name, frame.line, end_line, frame.depth))class CGrammar: \"\"\"C: structure from {}-pairs outside comments, literals and preprocessor lines. Also understands the C++ forms found in C headers and shared by the derived grammars (namespaces, classes, extern \"C\", qualified names). A grammar has a name and analyse(path, text) -> FileAnalysis; derived grammars add to the detection of the grammar they extend. \"\"\" name = \"c\" flow_control_keywords = frozenset({b\"if\", b\"else\", b\"for\", b\"while\", b\"do\", b\"switch\", b\"case\", b\"default\", b\"goto\", b\"?\"}) control_keywords_with_header = frozenset({b\"if\", b\"for\", b\"while\", b\"switch\"}) # followed by (...) control_keywords_without_header = frozenset({b\"else\", b\"do\"}) lambda_qualifiers = frozenset() # a { right after one of these opens a lambda body has_raw_strings = False has_at_keywords = False def __init__(self): self.token_pattern = re.compile(self.token_regex(), re.DOTALL) self.control_keywords = self.control_keywords_with_header | self.control_keywords_without_header def token_regex(self): \"\"\"Return the tokenizer's regular expression; every byte of the text is part of some token.\"\"\" alternatives = [ rb\"(?P[ \\t\\f\\v]+)\", rb\"(?P\\n)\", rb\"(?P\\\\\\n)\", rb\"(?P//(?:\\\\\\n|[^\\n])*)\", rb\"(?P/\\*.*?\\*/)\", rb\"(?P/\\*.*)\", ] if self.has_raw_strings: # C++ R\"delimiter(...)delimiter\" alternatives += [ rb'(?P(?:u8|[uUL])?R\"(?P[^ ()\\\\\\t\\v\\f\\n\"]{0,16})\\(.*?\\)(?P=delimiter)\")', rb'(?P(?:u8|[uUL])?R\"[^ ()\\\\\\t\\v\\f\\n\"]{0,16}\\()', ] alternatives += [ rb'(?P\"(?:\\\\.|[^\"\\\\\\n])*\")', rb'(?P\"(?:\\\\.|[^\"\\\\\\n])*)', rb\"(?P'(?:\\\\.|[^'\\\\\\n])*')\", rb\"(?P'(?:\\\\.|[^'\\\\\\n])*)\", rb\"(?P\\.?[0-9](?:[eEpP][+-]|'[0-9A-Za-z_]|[0-9A-Za-z_.])*)\", ] if self.has_at_keywords: # Objective-C @interface, @try, ... alternatives.append(rb\"(?P@[A-Za-z_][A-Za-z0-9_]*)\") alternatives += [ rb\"(?P[A-Za-z_$\\x80-\\xff][A-Za-z0-9_$\\x80-\\xff]*)\", rb\"(?P::|->|[^\\n])\", ] return b\"|\".join(alternatives) def analyse(self, path, text): \"\"\"Find the routines and flows of a file (line endings unified to newline).\"\"\" lexed = self.tokenize(path, text) if not lexed.is_valid: # the structure cannot be trusted return FileAnalysis(False, lexed.problem, [], [], [], 0) analysis = self.find_structure(lexed.tokens, lexed.comment_byte_count) return analysis._replace(unbalanced_branches=lexed.unbalanced_branches) def tokenize(self, path, text): \"\"\"Split text into tokens, dropping comments, preprocessor lines and the conditional branches not read.\"\"\" tokens = [] comment_byte_count = 0 line = 1 at_line_start = True # no token yet on this line: a # here starts a directive in_directive = False expecting_directive_name = False directive_name = b\"\" # of the directive being read; stays empty for a null directive (#) directive_line = 0 directive_arguments = [] # texts of the tokens after the name, e.g. the condition of an #if conditionals = ConditionalBranches() for match in self.token_pattern.finditer(text): # one token per iteration kind = match.lastgroup if kind == \"space\": continue if kind == \"newline\": # ends the line and any directive line += 1 at_line_start = True if in_directive: # a complete directive: conditionals decide which lines are read in_directive = False is_valid, problem = conditionals.apply(directive_name, directive_arguments, directive_line) if not is_valid: return LexResult(False, problem, [], 0, []) continue if kind == \"splice\": # backslash-newline: the line continues line += 1 continue start, end = match.span() if kind == \"line_comment\" or kind == \"block_comment\": # counted, otherwise ignored comment_byte_count += end - start line += text.count(b\"\\n\", start, end) continue if kind == \"unterminated_comment\": return LexResult(False, f\"{line}: unterminated /* comment\", [], 0, []) if kind == \"unterminated_raw_string\": return LexResult(False, f\"{line}: unterminated raw string literal\", [], 0, []) token_line = line if kind in (\"string\", \"raw_string\", \"character\", \"unterminated_string\", \"unterminated_character\"): line += text.count(b\"\\n\", start, end) # line splices inside a literal is_first_on_line = at_line_start at_line_start = False if in_directive: # collected, applied at the end of the directive if expecting_directive_name: expecting_directive_name = False directive_name = match.group() else: # an argument directive_arguments.append(match.group()) continue if kind == \"punctuator\" and is_first_on_line and match.group() == b\"#\": # a directive starts in_directive = True expecting_directive_name = True directive_name = b\"\" directive_line = token_line directive_arguments = [] continue if not conditionals.is_active: # in a conditional branch that is not read continue if kind == \"identifier\" or kind == \"punctuator\" or kind == \"at_keyword\": token_text = match.group() elif kind == \"number\": token_text = NUMBER_TOKEN elif kind == \"string\" or kind == \"raw_string\": token_text = STRING_TOKEN elif kind == \"character\": token_text = CHARACTER_TOKEN else: # unterminated string or character literal: taken to end at the end of the line literal = \"string\" if kind == \"unterminated_string\" else \"character\" messages.warning(f\"{path}: {token_line}: unterminated {literal} literal; taken to end at the end of\" f\" the line\") token_text = STRING_TOKEN if kind == \"unterminated_string\" else CHARACTER_TOKEN tokens.append(Token(token_text, token_line, is_first_on_line)) if token_text == b\"{\" or token_text == b\"}\": # per branch, to point at a likely cause of failure conditionals.count_brace(token_text) if in_directive: # a directive on the last line, without newline is_valid, problem = conditionals.apply(directive_name, directive_arguments, directive_line) if not is_valid: return LexResult(False, problem, [], 0, []) if len(conditionals.open_conditionals) > 0: # an #if is still open return LexResult(False, f\"{conditionals.open_conditionals[-1].line}: #if without #endif\", [], 0, []) return LexResult(True, \"\", tokens, comment_byte_count, conditionals.unbalanced_branches) def find_structure(self, tokens, comment_byte_count): \"\"\"Find the routines, and the flows within them, from the tokens.\"\"\" tokens = strip_standard_attributes(tokens) routines = [] blocks = [] stack = [] # per open { outside routines, and the { of the current routine: (FrameKind, line) header = [] # tokens of the declaration being read outside routines header_parenthesis_depth = 0 in_routine = False routine_parser = RoutineParser(self, UNNAMED_ROUTINE, 0) # replaced at each routine; read only in one previous_text = b\"\" # empty at the start of the file for index, token in enumerate(tokens): # one token per iteration text = token.text if in_routine: # its parser follows the statements next_text = tokens[index + 1].text if index + 1 < len(tokens) else b\"\" if routine_parser.feed(token, next_text): # the routine's closing }: back outside routines stack.pop() routines.append(Routine(routine_parser.name, routine_parser.start_line, token.line, routine_parser.flow_control_count)) blocks.extend(routine_parser.blocks) in_routine = False header = [] header_parenthesis_depth = 0 previous_text = text continue if len(stack) > 0 and stack[-1][0] == FrameKind.DATA: # only pair the braces if text == b\"{\": stack.append((FrameKind.DATA, token.line)) elif text == b\"}\": stack.pop() if len(stack) == 0 or stack[-1][0] == FrameKind.CONTAINER: # the header goes on header.append(Token(BRACED_GROUP_TOKEN, token.line, False)) previous_text = text continue if text == b\"{\": # outside routines: classify by the header kind, name = self.classify_container_brace(header, header_parenthesis_depth, previous_text) stack.append((kind, token.line)) if kind == FrameKind.ROUTINE: # enter it in_routine = True routine_parser = RoutineParser(self, name, token.line) elif kind == FrameKind.CONTAINER: # its contents start new headers header = [] header_parenthesis_depth = 0 elif text == b\"}\": # closes a container if len(stack) == 0: return FileAnalysis(False, f\"{token.line}: unmatched }}\", [], [], [], 0) stack.pop() header = [] header_parenthesis_depth = 0 elif text == b\";\" or (text == b\":\" and header_parenthesis_depth == 0 and len(header) > 0 and header[-1].text in ACCESS_SPECIFIERS): # a declaration or public: ends header = [] header_parenthesis_depth = 0 else: # part of the header if text == b\"(\": header_parenthesis_depth += 1 elif text == b\")\" and header_parenthesis_depth > 0: header_parenthesis_depth -= 1 header.append(token) previous_text = text if in_routine: # a { in the routine, or the routine's own, is never closed return FileAnalysis(False, f\"{routine_parser.innermost_open_brace_line()}: unclosed {{\", [], [], [], 0) if len(stack) > 0: # a { is never closed return FileAnalysis(False, f\"{stack[-1][1]}: unclosed {{\", [], [], [], 0) return FileAnalysis(True, \"\", [], routines, blocks, comment_byte_count) def classify_container_brace(self, header, header_parenthesis_depth, previous_text): \"\"\"Classify a { outside routines by its header; return (FrameKind, routine name).\"\"\" if header_parenthesis_depth > 0: # inside parentheses: e.g. a default argument {} return FrameKind.DATA, NO_ROUTINE_NAME is_routine, name, has_member_initializers = self.detect_routine(header) if is_routine: # a routine body, or a member initialiser x{...} before it if has_member_initializers and (is_identifier(previous_text) or previous_text == b\">\"): return FrameKind.DATA, NO_ROUTINE_NAME return FrameKind.ROUTINE, name if len(header) > 0 and is_container_header([token.text for token in header]): return FrameKind.CONTAINER, NO_ROUTINE_NAME return FrameKind.DATA, NO_ROUTINE_NAME def detect_routine(self, header): \"\"\"Recognise a routine header; return (is routine, name, has constructor member initialisers). The parameter list is the first outermost (...) followed by nothing but qualifiers, a trailing return type or member initialisers. \"\"\" texts = strip_attributes([token.text for token in header]) for open_index, close_index in top_level_parenthesis_groups(texts): # the first that fits is_routine_tail, has_member_initializers = scan_routine_tail(texts, close_index + 1) if is_routine_tail: # found the parameter list return True, routine_name(texts, open_index), has_member_initializers return False, NO_ROUTINE_NAME, Falseclass CppGrammar(CGrammar): \"\"\"C++: C plus try and catch, raw string literals and lambda qualifiers.\"\"\" name = \"c++\" flow_control_keywords = CGrammar.flow_control_keywords | {b\"try\", b\"catch\"} control_keywords_with_header = CGrammar.control_keywords_with_header | {b\"catch\"} control_keywords_without_header = CGrammar.control_keywords_without_header | {b\"try\"} lambda_qualifiers = frozenset({b\"mutable\", b\"noexcept\"}) has_raw_strings = Trueclass ObjectiveCGrammar(CGrammar): \"\"\"Objective-C: C plus methods, @try and @catch, and @interface/@implementation sections.\"\"\" name = \"objc\" flow_control_keywords = CGrammar.flow_control_keywords | {b\"@try\", b\"@catch\"} control_keywords_with_header = CGrammar.control_keywords_with_header | {b\"@catch\", b\"@synchronized\"} control_keywords_without_header = CGrammar.control_keywords_without_header | {b\"@try\", b\"@finally\", b\"@autoreleasepool\"} has_at_keywords = True def classify_container_brace(self, header, header_parenthesis_depth, previous_text): \"\"\"Recognise instance-variable blocks; strip section openings and @end from the header.\"\"\" if header_parenthesis_depth == 0: # look for the last @interface, @implementation, @protocol or @end texts = [token.text for token in header] for index in range(len(texts) - 1, -1, -1): if texts[index] in OBJECTIVE_C_SECTION_KEYWORDS: # the header proper starts after it if texts[index] == b\"@end\": header = header[index + 1:] break section_head_end = objective_c_section_head_end(texts, index) if section_head_end >= len(texts): # @interface Name ... { : instance variables return FrameKind.DATA, NO_ROUTINE_NAME header = header[section_head_end:] break return super().classify_container_brace(header, header_parenthesis_depth, previous_text) def detect_routine(self, header): \"\"\"A method starts with - or + at the start of a line, followed by its return type in parentheses.\"\"\" for index in range(len(header) - 2, -1, -1): # the last method start if header[index].starts_line and header[index].text in (b\"-\", b\"+\") and header[index + 1].text == b\"(\": return True, objective_c_method_name([token.text for token in header[index:]]), False return super().detect_routine(header)class ObjectiveCppGrammar(CppGrammar, ObjectiveCGrammar): \"\"\"Objective-C++: C++ and Objective-C together.\"\"\" name = \"objc++\" flow_control_keywords = CppGrammar.flow_control_keywords | ObjectiveCGrammar.flow_control_keywords control_keywords_with_header = (CppGrammar.control_keywords_with_header | ObjectiveCGrammar.control_keywords_with_header) control_keywords_without_header = (CppGrammar.control_keywords_without_header | ObjectiveCGrammar.control_keywords_without_header)def objective_c_section_head_end(texts, index): \"\"\"Return the index after `@interface Name (Category) : Super ` starting at index.\"\"\" position = index + 1 if position < len(texts) and is_identifier(texts[position]): # the class or protocol name position += 1 while position < len(texts): # categories, superclass, protocols, generic parameters text = texts[position] if text == b\"<\": position = skip_group(texts, position, b\"<\", b\">\") elif text == b\"(\": position = skip_group(texts, position, b\"(\", b\")\") elif text == b\":\" and position + 1 < len(texts) and is_identifier(texts[position + 1]): position += 2 else: # the section head ends break return positiondef objective_c_method_name(texts): \"\"\"Name a method from its header, which starts with - or +: e.g. b\"-initWithFrame:style:\".\"\"\" sign = texts[0] position = 1 if position < len(texts) and texts[position] == b\"(\": # the return type position = skip_group(texts, position, b\"(\", b\")\") selector_parts = [] plain_names = [] while position < len(texts): # keywords (name:), parameter types and names text = texts[position] if text == b\"(\": position = skip_group(texts, position, b\"(\", b\")\") elif is_identifier(text) and position + 1 < len(texts) and texts[position + 1] == b\":\": selector_parts.append(text + b\":\") position += 2 elif text == b\":\": # an unnamed keyword selector_parts.append(b\":\") position += 1 else: # a parameter name, or the name of a method without parameters if is_identifier(text): plain_names.append(text) position += 1 if len(selector_parts) > 0: # a method with parameters return sign + b\"\".join(selector_parts) if len(plain_names) > 0: # a method without parameters return sign + plain_names[0] return UNNAMED_ROUTINEGRAMMARS = {grammar.name: grammar for grammar in (CGrammar(), CppGrammar(), ObjectiveCGrammar(), ObjectiveCppGrammar())}DEFAULT_GRAMMAR_NAMES = { \".c\": \"c\", \".cc\": \"c++\", \".cpp\": \"c++\", \".cxx\": \"c++\", \".c++\": \"c++\", \".hh\": \"c++\", \".hpp\": \"c++\", \".hxx\": \"c++\", \".m\": \"objc\", \".mm\": \"objc++\", \".h\": \"objc++\", # a header may be C, C++ or Objective-C: the superset}def file_extension(path): \"\"\"Return the extension of a path, e.g. \".cpp\".\"\"\" return os.path.splitext(path)[1]def single_value(parser, option, values): \"\"\"Return (is given, text) of an option that may be given once (argparse's None: not given).\"\"\" if values is None: # not given; the text is never read return False, \"\" if len(values) > 1: parser.error(f\"{option} given more than once\") return True, values[0]def parse_threshold(parser, option, text): \"\"\"Validate a threshold: a whole number.\"\"\" if THRESHOLD_PATTERN.fullmatch(text) is None: parser.error(f\"{option} must be a whole number from 0 to 999999999, got {text!r}\") return int(text)def parse_positive_number(parser, option, text): \"\"\"Validate a penalty or quality reference: a number above 0, fractions allowed.\"\"\" if POSITIVE_NUMBER_PATTERN.fullmatch(text) is None or Decimal(text) == 0: parser.error(f\"{option} must be a number above 0, got {text!r}\") return Decimal(text)def parse_arguments(argv): \"\"\"Parse and validate the command line; exits with EXIT_ERROR on invalid input.\"\"\" parser = argparse.ArgumentParser( prog=PROGRAM_NAME, allow_abbrev=False, description=\"Find and report structured-programming issues: too long routines and flows, too deep\" \" nesting, too complex routines.\") check_help = { CheckKind.TOO_LONG_ROUTINES: \"check routine length: lines beyond the threshold\", CheckKind.TOO_LONG_FLOWS: \"check flow length (a flow is a block inside a routine): lines beyond the threshold\", CheckKind.TOO_DEEP: \"check flow depth: levels beyond the threshold, for every flow\", CheckKind.TOO_COMPLEX: \"check routine complexity: flow control statements beyond the threshold\", } parser.add_argument(\"--all\", action=\"store_true\", help=\"make all four checks\") for kind in CheckKind: # three options per check parser.add_argument(f\"--{kind.value}\", dest=f\"{kind.name}_requested\", action=\"store_true\", help=check_help[kind]) parser.add_argument( f\"--{kind.value}-threshold\", dest=f\"{kind.name}_threshold\", action=\"append\", metavar=\"N\", help=f\"threshold of --{kind.value} (default {DEFAULT_THRESHOLDS[kind]}); implies --{kind.value}\") parser.add_argument( f\"--{kind.value}-penalty\", dest=f\"{kind.name}_penalty\", action=\"append\", metavar=\"M\", help=f\"penalty of --{kind.value}, the score per unit of excess (default {DEFAULT_PENALTIES[kind]});\" f\" implies --{kind.value}\") parser.add_argument( \"--quality-reference\", action=\"append\", metavar=\"P\", help=f\"the penalty score from which on quality is 1, the worst; penalty score 0 is quality 10\" f\" (default {DEFAULT_QUALITY_REFERENCE})\") parser.add_argument( \"--override\", action=\"append\", nargs=2, default=[], metavar=(\"EXTENSION\", \"GRAMMAR\"), help=f\"read files ending in EXTENSION (e.g. .inl) with GRAMMAR, one of {', '.join(GRAMMARS)};\" f\" may be repeated\") parser.add_argument( \"--verbose\", action=\"store_true\", help=\"also list every issue of each file, and why a failed file failed\") order_group = parser.add_mutually_exclusive_group() order_group.add_argument( \"--worst-first\", dest=\"report_order\", action=\"store_const\", const=ReportOrder.WORST, default=ReportOrder.PATH, help=\"order the files by penalty score, highest first (default: by path)\") order_group.add_argument( \"--highest-score-first\", dest=\"report_order\", action=\"store_const\", const=ReportOrder.HIGHEST_SCORE, help=\"order the files by total score (not per lines), highest first\") parser.add_argument( \"--reverse-report\", action=\"store_true\", help=\"list the files in reverse order, so the most important come last\") parser.add_argument( \"--max-report\", type=int, metavar=\"N\", help=\"list at most N files, the most important ones; FINDCOMPL, --summary and exit status still count all\") parser.add_argument( \"--summary\", action=\"store_true\", help=\"end with the issues, excess and score per check, the share of comments, and the failed files\") parser.add_argument( \"--usage\", action=ManualAction, manual=MANUAL, help=\"show this help plus a description of behaviour, output and exit status, and exit\") parser.add_argument( \"--version\", action=\"version\", version=version_text(\"%(prog)s\", PROGRAM_VERSION, RELEASE_DATE), help=\"show the version and release date, and exit\") parser.add_argument(\"paths\", nargs=\"+\", metavar=\"PATH\", help=\"file, or directory to scan recursively\") arguments = parser.parse_args(argv) checks = {} for kind in CheckKind: # in report order option = f\"--{kind.value}\" has_threshold, threshold_text = single_value( parser, f\"{option}-threshold\", getattr(arguments, f\"{kind.name}_threshold\")) has_penalty, penalty_text = single_value( parser, f\"{option}-penalty\", getattr(arguments, f\"{kind.name}_penalty\")) if arguments.all or getattr(arguments, f\"{kind.name}_requested\") or has_threshold or has_penalty: threshold = DEFAULT_THRESHOLDS[kind] if has_threshold: threshold = parse_threshold(parser, f\"{option}-threshold\", threshold_text) penalty = DEFAULT_PENALTIES[kind] if has_penalty: penalty = parse_positive_number(parser, f\"{option}-penalty\", penalty_text) checks[kind] = Check(kind, threshold, penalty) if len(checks) == 0: parser.error(\"at least one check is required: --all, or one of \" + \", \".join(f\"--{kind.value}\" for kind in CheckKind) + \" or their -threshold or -penalty\") has_quality_reference, quality_reference_text = single_value( parser, \"--quality-reference\", arguments.quality_reference) quality_reference = DEFAULT_QUALITY_REFERENCE if has_quality_reference: quality_reference = parse_positive_number(parser, \"--quality-reference\", quality_reference_text) grammar_by_extension = {extension: GRAMMARS[name] for extension, name in DEFAULT_GRAMMAR_NAMES.items()} overridden_extensions = set() for extension, grammar_name in arguments.override: # each must be valid and given once if not extension.startswith(\".\") or len(extension) < 2 or os.sep in extension: parser.error(f\"--override: EXTENSION must be a dot followed by at least one character, got {extension!r}\") if extension in overridden_extensions: parser.error(f\"--override {extension} given more than once\") if grammar_name not in GRAMMARS: parser.error(f\"--override: unknown grammar {grammar_name!r}; choose from {', '.join(GRAMMARS)}\") overridden_extensions.add(extension) grammar_by_extension[extension] = GRAMMARS[grammar_name] limit_report_size, max_report_size = check_max_report(parser, arguments.max_report) return Settings(checks, quality_reference, grammar_by_extension, arguments.verbose, arguments.report_order, arguments.reverse_report, limit_report_size, max_report_size, arguments.summary, arguments.paths)def count_non_blank_lines(text): \"\"\"Return a list whose entry i is the number of non-blank lines among lines 1 to i.\"\"\" counts = [0] for line in text.split(b\"\\n\"): # running total counts.append(counts[-1] + (1 if line.strip() else 0)) return countsdef score_file(path, analysis, non_blank_counts, checks): \"\"\"Apply the checks to a file's routines and flows; return its FileReport.\"\"\" issues = [] def add_issue_if_over(kind, line, name, measure): \"\"\"Record an issue when the check is made and the measure exceeds its threshold.\"\"\" if kind in checks and measure > checks[kind].threshold: excess = measure - checks[kind].threshold issues.append(Issue(line, kind, name, measure, excess, excess * checks[kind].penalty)) for routine in analysis.routines: # length and complexity length = non_blank_counts[routine.end_line] - non_blank_counts[routine.start_line - 1] add_issue_if_over(CheckKind.TOO_LONG_ROUTINES, routine.start_line, routine.name, length) add_issue_if_over(CheckKind.TOO_COMPLEX, routine.start_line, routine.name, routine.flow_control_count) for block in analysis.blocks: # length and depth length = non_blank_counts[block.end_line] - non_blank_counts[block.start_line - 1] add_issue_if_over(CheckKind.TOO_LONG_FLOWS, block.start_line, block.routine_name, length) add_issue_if_over(CheckKind.TOO_DEEP, block.start_line, block.routine_name, block.depth) issues.sort(key=lambda issue: (issue.line, CHECK_ORDER[issue.kind])) return FileReport(path, True, non_blank_counts[-1], total_by_check(issues, checks), issues, \"\", [])def total_by_check(issues, checks): \"\"\"Add up issues per check made, also for checks without issues; return {CheckKind: CheckTotal}.\"\"\" totals = {} for kind in checks: # in report order kind_issues = [issue for issue in issues if issue.kind == kind] totals[kind] = CheckTotal(len(kind_issues), sum(issue.excess for issue in kind_issues), sum((issue.score for issue in kind_issues), Decimal(0))) return totalsdef combine_check_totals(reports, checks): \"\"\"Add up the per-check totals of all files.\"\"\" totals = {} for kind in checks: # in report order file_totals = [report.check_totals[kind] for report in reports] totals[kind] = CheckTotal(sum(total.issue_count for total in file_totals), sum(total.excess for total in file_totals), sum((total.score for total in file_totals), Decimal(0))) return totalsdef failed_file_report(path, analysis, checks): \"\"\"Return the FileReport of a file that could not be analysed: no score, no lines.\"\"\" return FileReport(path, False, 0, total_by_check([], checks), [], analysis.problem, analysis.unbalanced_branches)def total_score(check_totals): \"\"\"The scores of all checks added up.\"\"\" return sum((total.score for total in check_totals.values()), Decimal(0))def per_thousand_lines(score, line_count): \"\"\"Score per 1000 lines; only for line_count > 0.\"\"\" return score * 1000 / line_countdef penalty_score(check_totals, line_count): \"\"\"The average over the checks made of their score per 1000 lines; only for line_count > 0.\"\"\" return per_thousand_lines(total_score(check_totals), line_count) / len(check_totals)def quality(penalty, quality_reference): \"\"\"From 10 for penalty score 0 down to 1 for the reference penalty score or more.\"\"\" return BEST_QUALITY - QUALITY_RANGE * min(penalty / quality_reference, Decimal(1))def format_number(value): \"\"\"Show a Decimal without exponent or trailing zeros: 37, 55.5.\"\"\" return format(value.normalize(), \"f\")def format_one_decimal(value): \"\"\"Show a Decimal rounded to one decimal: 412.0.\"\"\" return format(value.quantize(RATE_QUANTUM, ROUND_HALF_UP), \"f\")def format_rate(score, line_count): \"\"\"Show a score per 1000 lines with one decimal, or \"undefined\" without lines.\"\"\" if line_count == 0: # nothing to relate to return \"undefined\" return format_one_decimal(per_thousand_lines(score, line_count))def describe_scores(check_totals, line_count, settings): \"\"\"Describe the penalty score, lines and quality of a file or of all files.\"\"\" if line_count > 0: # both are defined penalty = penalty_score(check_totals, line_count) penalty_text = format_one_decimal(penalty) quality_text = format_one_decimal(quality(penalty, settings.quality_reference)) else: # nothing to relate to penalty_text = \"undefined\" quality_text = \"undefined\" return f\"penalty {penalty_text}, lines {line_count}, quality {quality_text}\"def sort_reports(reports, report_order): \"\"\"Order the file reports, most important first, then by path; failed files come last.\"\"\" if report_order == ReportOrder.WORST: # highest penalty score first; files without lines last return sorted(reports, key=lambda report: ( not report.is_analysed, report.line_count == 0, -penalty_score(report.check_totals, report.line_count) if report.line_count > 0 else 0, report.path)) if report_order == ReportOrder.HIGHEST_SCORE: # highest total score first return sorted(reports, key=lambda report: (not report.is_analysed, -total_score(report.check_totals), report.path)) return sorted(reports, key=lambda report: (not report.is_analysed, report.path)) # ReportOrder.PATHdef describe_measure(kind, measure): \"\"\"Describe the measured value of an issue, e.g. \"87 lines\".\"\"\" if kind == CheckKind.TOO_DEEP: return f\"depth {measure}\" if kind == CheckKind.TOO_COMPLEX: return f\"{measure} flow control statements\" return f\"{measure} lines\" # CheckKind.TOO_LONG_ROUTINES or CheckKind.TOO_LONG_FLOWSdef describe_check_total(kind, total, line_count): \"\"\"Describe the issues, excess and score of one check, e.g. for a file or for all files.\"\"\" return (f\"{kind.value}: issues {total.issue_count}, excess {total.excess} {EXCESS_UNITS[kind]},\" f\" score {format_number(total.score)}, {format_rate(total.score, line_count)} per 1000 lines\")def describe_unbalanced_branch(branch): \"\"\"Describe a conditional branch whose braces do not balance.\"\"\" surplus_brace = \"{\" if branch.brace_balance > 0 else \"}\" return (f\"{branch.line}: likely cause: the braces of this conditional branch do not balance\" f\" ({abs(branch.brace_balance)} {surplus_brace} too many)\")def write_file_report(output, report, settings): \"\"\"Write a file's scores and, if verbose, its totals and issues; or that it failed and, if verbose, why.\"\"\" encoded_path = os.fsencode(report.path) if not report.is_analysed: # failed: no score output.write(encoded_path + b\": failed\\n\") if settings.verbose: # where the failure showed, and the first branch that may have caused it output.write(encoded_path + f\": {report.problem}\\n\".encode()) if len(report.unbalanced_branches) > 0: first_branch = min(report.unbalanced_branches, key=lambda branch: branch.line) output.write(encoded_path + f\": {describe_unbalanced_branch(first_branch)}\\n\".encode()) return output.write(encoded_path + f\": {describe_scores(report.check_totals, report.line_count, settings)}\\n\".encode()) if settings.verbose: # totals of the checks with excess, then one line per issue for kind, total in report.check_totals.items(): if total.excess > 0: output.write(encoded_path + f\": {describe_check_total(kind, total, report.line_count)}\\n\".encode()) for issue in report.issues: output.write(encoded_path + f\": {issue.line}: {issue.kind.value} \".encode() + issue.routine_name + (f\": {describe_measure(issue.kind, issue.measure)}\" f\" (threshold {settings.checks[issue.kind].threshold}, excess {issue.excess},\" f\" score {format_number(issue.score)})\\n\").encode())def write_summary(output, check_totals, reports, line_count, comment_byte_count, total_byte_count): \"\"\"Write per check the issues, excess and score, the share of comments, and the failed files.\"\"\" for kind, total in check_totals.items(): # in report order output.write(f\"{describe_check_total(kind, total, line_count)}\\n\".encode()) if total_byte_count > 0: # the share is defined percentage = format_one_decimal(Decimal(comment_byte_count * 100) / total_byte_count) + \"%\" else: # nothing scanned percentage = \"undefined\" output.write(f\"Comments ignored: {comment_byte_count} out of {total_byte_count}: {percentage}\\n\".encode()) failed_paths = sorted(report.path for report in reports if not report.is_analysed) output.write(f\"failed files: {len(failed_paths)}\\n\".encode()) for path in failed_paths: output.write(b\"failed: \" + os.fsencode(path) + b\"\\n\")def main(argv): \"\"\"Scan the given paths and report structured-programming issues; return the exit status.\"\"\" install_default_signal_handlers() settings = parse_arguments(argv) all_valid, checked_arguments = check_arguments(settings.paths, messages) if not all_valid: # a command-line path is missing or unusable return EXIT_ERROR collector = SourceCollector(lambda name: file_extension(name) in settings.grammar_by_extension, messages) for path, path_stat in checked_arguments: collector.add_argument(path, path_stat) source_files = [] for path, is_explicit in collector.source_files: # files found in directories all have a grammar if file_extension(path) in settings.grammar_by_extension: source_files.append((path, is_explicit)) else: # named on the command line messages.warning(f\"no grammar for {path}; skipped (see --override)\") if len(source_files) == 0: # say so rather than silently report no issues messages.warning(\"no files to scan\") reports = [] comment_byte_count = 0 total_byte_count = 0 for path, content in read_source_files(source_files, messages): # analyse and score each file text = content.replace(b\"\\r\\n\", b\"\\n\").replace(b\"\\r\", b\"\\n\") analysis = settings.grammar_by_extension[file_extension(path)].analyse(path, text) if not analysis.is_analysable: # its structure cannot be trusted: reported as failed reports.append(failed_file_report(path, analysis, settings.checks)) continue reports.append(score_file(path, analysis, count_non_blank_lines(text), settings.checks)) comment_byte_count += analysis.comment_byte_count total_byte_count += len(text) output = sys.stdout.buffer shown_reports = select_shown_reports(sort_reports(reports, settings.report_order), settings.limit_report_size, settings.max_report_size, settings.reverse_report) for report in shown_reports: write_file_report(output, report, settings) check_totals = combine_check_totals(reports, settings.checks) line_count = sum(report.line_count for report in reports) output.write(f\"FINDCOMPL: {describe_scores(check_totals, line_count, settings)}\\n\".encode()) if settings.summary: # totals after FINDCOMPL write_summary(output, check_totals, reports, line_count, comment_byte_count, total_byte_count) output.flush() return EXIT_ISSUES_FOUND if total_score(check_totals) > 0 else EXIT_NO_ISSUESif __name__ == \"__main__\": sys.exit(main(sys.argv[1:]))\n```\n\n[By the way, this part of developing with an LLM is great. A frontier model generates a ‘small’ Python script like this easily, and that improves my power to work with a large code base]\n\nAs it turned out, Claude had generated code with extremely long and complicated logic. Here is that previous example from smack in the middle of a routine of several hundred lines — already some levels deep — again, where Claude has defined routines inside routines.\n\n```\n// The container's own subtree and its ancestors are never candidates.std::unordered_set containerSubtree;std::function collectSubtree = [&](const EACMP_Uuid& parentUUID) { // depth-first over the nesting forest below parentUUID for (const auto& childPtr : view->viewElements()) { // each possible child if (childPtr->parentViewElement() != parentUUID) { continue; } // not a child here if (!containerSubtree.insert(childPtr->uuid()).second) { continue; } // already collected collectSubtree(childPtr->uuid()); } };collectSubtree(containerViewElementUUID);\n```\n\nAfter the complete rewrite this became a separate small named routine at the top level:\n\n```\n// Depth-first over the nesting forest below parentUUID: every depiction nested in it, transitively.static void EACMP_collectNestingSubtree(const EACMP_View* view, const EACMP_Uuid& parentUUID,std::unordered_set& subtree){ // depth-first over the nesting forest below parentUUID for (const auto& childPtr : view->viewElements()) { // each possible child if (childPtr->parentViewElement() != parentUUID) { continue; } // not a child here if (!subtree.insert(childPtr->uuid()).second) { continue; } // already collected EACMP_collectNestingSubtree(view, childPtr->uuid(), subtree); }}\n```\n\nAnd when it is called in the routine that needs it, it is now a single line:\n\n```\nstd::unordered_set containerSubtree;EACMP_collectNestingSubtree(view, containerViewElementUUID, containerSubtree);\n```\n\nAs a result we get more code (about ~10%), but it is easier to read and understand. We lose local complexity, we gain readability. Every function has a clear job and its job is not written as part of another job.\n\n### IfSQ: Cause of Concern (CfC)\n\nCfC is not part of a Level-2 IfSQ inspection. It requires much more in depth work for an auditor. It is a pure manual human thing and it is hard. Take hundreds of lines of code in a product and try to find missing functionality.\n\nBut here we find that a review *by an LLM* actually improves on IfSQ here. Because, the pattern matching of an LLM is capable of catching WIP based on code alone. I’ve had the LLM create reviews that came up with “the implementation of X is incomplete, it is missing from Y and Z”. To be honest, this is a mix of WIP and a — ‘missing functionality’ — ‘Cause for Concern’. IfSQ CfC’s are hard to spot for humans, the auditor’s trained eye is better at detecting it than ordinary software engineers, but it is still a matter of chance. I’ll get to that below. \n\nNow, the LLM created all that code itself and this leads to an important observation:\n\nFor an LLM, generating code and reviewing code are quite different things\n\nI’ve seen this before. LLMs do not understand code, because if they would, they would not produce code they themselves flag as very bad, and that is what I (and others) have seen.\n\n## The current code quality document\n\nAs promised: This is my current code quality document that is part of my ‘stable project documentation’ (the ‘context harness’), which is the material Claude can draw from for its context:\n\n# EACMP Code Quality — robustness, naming, comments and structure\n\nAuthoritative, self-contained reference for how EACMP code is written: how it treats programming errors and input it did not produce (§2–§10), how it names and documents itself (§11–§16), how its routines are structured (§17), and how the audits and the on-demand measurements check them (§18–§23). It covers eacmplib, its bridges, qtgui, tools, the language bindings and mechanical_tests, plus the shape SVGs’ own variable names (§16); the one exemption is named in §1.1. This document is independent — other documents may cite it; it cites none. Read it before writing, naming, commenting or reviewing code.\n\n## 1. Scope\n\nThe conventions apply to every line the project writes in **eacmplib**, its **bridges** (`*_qt`,\n`*_ogdf`, `*_lua`), the **language bindings** (` bindings/`), **qtgui**, **tools**,\n**mechanical_tests**, and the **shape SVGs’ `data-vars` names** (§16). Tests and tools are not a\nlower tier: they are read by the same people, and a test that cannot be read cannot be trusted to\nsay what it asserts.\n\n### 1.1 The frozen-migration exemption\n\n**Exempt: the per-version migration ledgers.** `tools/migrate_landscapes.cpp` and\n`tools/migrate_preferences.cpp` are write-once records. Each `migrate_N_to_M` step describes a\nformat transition that has already shipped and is never edited again — the tool grows a new step\nper version bump and leaves the older ones untouched. Holding them to conventions that evolve\nwould mean rewriting frozen history for no reader’s benefit and risking a silent change to a\nmigration that can no longer be re-tested against the data it was written for. They are therefore\nskipped in full by the audits, which print them as `EXEMPT` rather than silently passing them.\nThe exemption is a property of *frozen content*, not of the `tools/` directory: every living tool\n(the validators, dumpers, table builders, dev scripts) is fully in scope.\n\nTwo narrower carve-outs, for the same “the literal IS the subject” reason:\n\n- The `[token]` rule (§18 rule 7) does not apply inside`mechanical_tests/` . A test that asserts\nthe scheme-referral wire vocabulary must spell`\"@@view\"` ; using the constant would only compare\nthe constant with itself.\n- W3C/SVG specification notation kept verbatim so the code reads against the specification — the\narc-algorithm locals in `grammar_svg.cpp` (`x1p` ,`x1p2` , …) — is §11’s exception: commented at\nits declaration, recorded in`tools/name_audit_accept.txt` .\n\n## 2. The principle\n\nBe hard on programming errors. Data that defines the application’s behaviour is fully correct or refused. What other producers made may get a best effort — never a silent one.\n\n- **A programming error** is a value the program itself produced that contradicts what the\nprogram guarantees: a broken invariant, an enumerator a function cannot take, a caller breaking\na function’s contract. It is never stepped around (§3).\n- **Outside input** is everything the running program reads that it did not compute itself:\ngrammar, scheme, landscape, preference and script files, imports, the clipboard, OS and Qt\nevents, typed text. An unexpected value there is not a programming error; it is checked where\nit enters (§4).\n- **Data that defines behaviour** — grammars and schemes — is fully correct or refused. A best\neffort is for content: what a landscape holds, what another tool’s file says.\n- **Nothing is silent.** What is refused or adapted is logged, and named wherever the user sees a\nreport.\n\nWhere a value came from decides which rules apply, not where it is used: a switch over an enum the program set is internal; the same enum converted from a file is boundary input up to its checked conversion (§5.2, §5.3).\n\n## 3. Programming errors\n\n### 3.1 Invariants\n\nAn invariant is a property the code may rely on, guaranteed by a named layer. Invariants are\nnumbered **I-n**, each stated where its layer is documented; code that relies on one or asserts it\ncites its number. A condition that can legitimately be false at runtime is not an invariant: it is\na guard with a guard reason (§4.1).\n\n### 3.2 The invariant checks\n\nInvariant violations are programmer-contract breaches, not runtime guards. The macros are in\n`eacmplib/include/eacmplib/development.h`, Qt-free and used by every layer.\n\nThe macro is chosen by the **consequence of continuing past the violation at that call site** — a\nproperty of the site, not of the invariant. The same invariant may be dangerous to ignore at one\nsite (I-1 while writing the grammars registry → a corrupt save) and harmless at another (I-1 while\nassigning type IDs → skip).\n\n- \n`EACMP_BREAKING_UNRECOVERABLE_INVARIANT(cond, msg)` —**dangerous or no safe recovery** :\ncontinuing would corrupt data or invoke undefined behaviour. Fails hard in either mode (Panic =\nlog + abort). This is the default; “always fail hard” still governs anything with dangerous\nconsequences.\n- \n`EACMP_BREAKING_RECOVERABLE_INVARIANT(cond, msg)` —**non-dangerous and recoverable** :\ncontinuing cannot corrupt data and the site has a defined local recovery. Development mode →\nPanic (caught in testing); production mode → logged at`Error` , then the caller’s explicit\nrecovery runs. Write the recovery right after the check:\n\n```\nEACMP_BREAKING_RECOVERABLE_INVARIANT(modelElement != nullptr, \"I-11: row UUID no longer resolves\");\nif (!modelElement) { continue; } // production recovery; development mode aborted above\n```\n\n Both modes evaluate the condition; only the level differs, and it is fetched once a check has already failed. A deliberate, bounded exception to “always fail hard”, justified only because the consequence is harmless — a general pattern, applied wherever it fits, not a one-off. Cite the `I-n` in`msg` .\n- \n`EACMP_DEV_LOG(level, msg)` — a permanent-in-source Debug/Info diagnostic that runs only in\ndevelopment mode; its arguments are not evaluated in production.`Warning` /`Error` /`Panic` are never wrapped (they are not development-only).\n\nNeither invariant macro is a guard: a condition that can legitimately be false at runtime stays an ordinary guard (§4.1).\n\n### 3.3 The development and production modes\n\nIn **development** mode a programmer error fails hard, so it blocks the work until it is fixed; in\n**production** mode the same violation is reported and the caller’s recovery runs. The mode is one\nsetting of the whole process (`EACMP_developmentMode()`), and one binary runs in either:\n\n- **Every process starts in development mode.** A tool, a test and the Lua module never leave it,\nso none can run lenient by omission.\n- **The application sets its mode at its start** : production, unless it is launched with`--development` , which the development launches and the mechanical suite’s plays pass.\n- **Developer ▸ Development Mode** turns the mode on and off while the application runs. Turned\non, a recoverable programmer error aborts the application, unsaved work included.\n- **What the launch alone decides.** The permission for logic shipped with a grammar to change a\nlandscape follows how the application was launched, never the menu.\n\nNo source compiles differently between the modes.\n\n### 3.4 No silent guard against a programming error\n\nA condition that can only be false when an invariant is violated — a pointer that I-2/I-4\nguarantees non-null, a lookup that a stale UUID cannot reach — must not be quietly stepped around\nwith `if (ptr) { … }` that does nothing when `ptr` is null: that hides the bug behind a no-op.\nChoose one:\n\n- **Trust the invariant** and use the value directly, with a comment naming the invariant. Prefer\nthis when the invariant genuinely holds at that point.\n- **Assert it** with an invariant check (§3.2), so a violation is loud, not swallowed.\n\nThe same holds for a value no caller may pass: a function that cannot take an enumerator asserts that it did not get it — or better, takes a type that cannot hold it (§7).\n\n## 4. Outside input\n\n### 4.1 Guard reasons\n\nA guard whose condition can legitimately be false is **not** an invariant violation: it stays a\nruntime guard (refuse / skip / return), commented with its `GR-n`. These are the only sanctioned\nreasons. A guard that fits none of them is guarding an invariant and belongs to §3, not a silent\nskip.\n\n| Code | Reason | Typical sites | \n|---|---|---|\n| **GR-1** | Boundary input not yet validated — data from disk, import, paste, clipboard | loader, import, paste | \n| **GR-2** | Resource availability — a grammar (or other named resource) referenced by name may be uninstalled | open, paste, `reloadGrammars` ,`addModel` | \n| **GR-3** | Stale lookup — a UUID held across an operation/refresh may no longer resolve | selection after delete, browser refresh | \n| **GR-4** | Optional absence / user cancel — an optional field is absent or a dialog was cancelled | optional JSON fields, dialogs | \n| **GR-5** | Transient construction order — an object observed before it is fully wired during two-phase construction | two-phase load / undo restore | \n\nCode-comment form: `// GR-2: grammar may be uninstalled`.\n\n### 4.2 Classes of input\n\n| Class | Examples | Treatment | \n|---|---|---|\n| 1. Data that defines behaviour | grammars with their shapes, languages, constraints, logic and external-format specs; colour, style and appearance schemes; bridge grammars | fully correct or refused: each defect is named at Error and the grammar or scheme does not load; nothing is ignored or defaulted | \n| 2. The frame of our own files | landscape, preference, session and clipboard files: version, parse, structure, identity | refused, with an Error naming the defect; never repaired by a guess such as a minted stand-in identity | \n| 3. Landscape content | an item of a type the grammar no longer knows, a depiction whose model item is gone, a dangling parent, an invalid `@@` value, an unknown value of a closed vocabulary, a value of the wrong type | by the strictness (§4.4): refused whole, every defect listed; or invalid content dropped where a valid remainder is possible, at Warning, named in the load’s warnings, the landscape marked changed so that saving it is a deliberate act | \n| 4. Files from other tools | ArchiMate exchange, BPMN, LMTX: import and update | by the strictness (§4.4): refused whole and nothing changed, every defect listed; or invalid content dropped where a valid remainder is possible, at Warning, each case an entry in the import report. Ordinary format translation and its notices are not best-effort recovery | \n| 5. Preference entries | a key no build knows any more, an invalid value | dropped or reset to the default, at Warning: one preferences file serves every build on the machine, so a key another build abandoned is normal there | \n| 6. Gestures | typed values, drops, a paste the destination accepts only in part | refused, or done in part, and shown to the user at the gesture: leaving out what the destination does not allow is what a paste means | \n| 7. Scripts and logic at run time | a Lua error in a demo script or in a grammar’s logic | the script stops; the action is refused | \n| 8. Environment | a missing file, no permission, an uninstalled resource (GR-2), a Qt or OS enum value | the operation is refused and reported; a third-party enum’s uninteresting rest is a commented `default:` (§5.2) | \n| 9. Derived caches | the compiled grammar cache | stale or unreadable — a value of the wrong type included: rebuilt exactly, which is not a best effort | \n\nA grammar’s ordinary import mappings define how the source format is represented. Best effort does not invent or select a replacement when that representation fails: it drops the unsupported content, or refuses when dropping cannot leave a valid result.\n\n### 4.3 Refusing and dropping\n\n- **A refusal** names what it refused and why, at Error, and changes nothing: a read that refuses\nleaves no half-loaded result, an import that refuses rolls back its action.\nA failure of behavior supplied by a grammar, such as a transform that cannot evaluate, refuses\neither mode through`EACMP_reportInputRefusal` ; it is not disguised as a content drop.\n- **A drop** goes through`EACMP_reportInputDrop` , which logs it and records it with the open\nreading (§4.4), and through the one helper for its kind, which adds the report entry and count,\nas an import’s skips do (`EACMP_skipElement` ,`EACMP_skipRelation` ). Drop the invalid entry,\noptional field or item only when its removal leaves valid content. An optional field’s absence\nhas exactly its format-defined meaning; no guessed value, coercion, substitute type, reversed\nrelation or other repair is best effort. A required frame or identity is refused, never invented.\n- **A retained unresolved reference** is reported through`EACMP_reportUnresolvedInput` . It is\na well-formed reference whose resource is unavailable, kept unchanged. It does not mark content\nchanged. A malformed reference is invalid content, not an unresolved resource.\n- **The result carries the report.** A reader includes issues raised by deep codecs in its result,\nnot just its own top-level notices. Dropped content marks the loaded landscape changed; a\nretained unresolved reference does not. Logs alone do not replace the caller’s report.\n- **One statement of a format’s structure.** A grammar file and its scheme files are validated\nagainst their schemas at load (`format_schema.h` ); the schema states every key, kind and closed\nvocabulary, and the loader repeats none of it — its own checks are of meaning, and of what the\nschema cannot state readably (a key one section takes and another does not). A schema change is a\nformat change, enforced by the loader the moment it is built.\n- **Entries of the wrong kind.** A reader takes a collection’s entries through the checked readers\nof`json_read.h` (`EACMP_jsonEntryIsObject` ,`EACMP_jsonStringArray` ,`EACMP_jsonReadStringMap` ):\nan entry of the wrong kind is a reported drop like an optional field of the wrong type, never a bare`continue` . A grammar’s load and a cache’s read are Strict readings of their own, so there it\nrefuses the grammar, or makes the cache stale.\n- **Log levels.** A drop or unresolved reference in outside input logs a Warning. A refusal, and\na recovered programming error (§3.2), log an Error. A drop made while no reading is open is of\nthe program’s own data — a history replay, a value it wrote itself — and so a defect: it logs\nan Error. Best effort never excuses a programming error.\n- **Checked once, where it enters.** After its checked conversion a value is internal, and the\nrules of §3 apply to it.\n\n### 4.4 Strictness\n\nA reader of landscape content and of other tools’ files takes the **input strictness** its caller\nstates (`EACMP_InputStrictness`), and opens a reading with it (` EACMP_InputReading`) for as long as\nit reads:\n\n- **Strict** refuses a reading that needed any drop or has an unresolved reference. The reader checks before it publishes\nanything, so a refused load hands out nothing and a refused import changes no landscape; the\nrefusal lists every issue it found.\n- **BestEffort** permits only the reported drops described in §4.3. It refuses when no valid\nremainder can be obtained by dropping. Retained unresolved references remain visible as warnings.\n\nEvery issue reported while a reading is open belongs to it, however deep in a codec it was found, and readings nest: an issue belongs to the innermost. Data that defines behaviour is always read strictly, whatever its caller’s strictness: a grammar’s load is a strict reading of its own, so a grammar loaded on the way through a landscape load is judged by itself, and a compiled cache that cannot be read exactly is stale.\n\nWho states what: the application reads with the strictness of its session — Strict in development\nmode at its start, BestEffort in production — which **Developer ▸ Strict Input** changes. A test\nstates it per input: Strict for an input known to be correct, BestEffort where the drops are\nwhat it tests. A tool states its own: one that checks or rewrites a file reads it strictly, one\nthat imports another tool’s file with a best effort.\n\n## 5. Every branch is deliberate\n\n### 5.1 `if`\n\nAn `if` has an `else`, **unless** it is genuinely correct to act in the true case and do nothing in\nthe false case — and then the absence of the `else` is stated in the `if`’s **opening comment**\n(the flow-control comment right after `{`), which explains why and contains the `no 'else'`\nmarker. “Do nothing” is a decision; make it visible where the branch is stated:\n\n```\nif (viewElement->isSelected()) { // only a selected item has handles; an unselected one needs none — no 'else' branch\n addResizeHandles(viewElement);\n}\n```\n\nA bare `if` with no `else` and no such note reads as an unfinished thought (a TODO in disguise).\n\n- **A branch that ends in a jump** —`return` ,`continue` ,`break` — has its`else` : the rest of\nthe block. It needs no marker.\n- **`else if` is not `else`.** A chain` if … else if …` without a final`else` is an`if` without\nan`else` : either it ends in`else` , or the*last* branch’s opening comment carries the marker.\n- **A one-line body** (`if (…) { …; } // comment` ) follows the same rule; its marker sits in the\ntrailing comment.\n\nReserve an elseless `if` for conditions that are legitimately optional at runtime (no view is\nopen, an optional field is absent) — never for a programming error (§3.4).\n\n### 5.2 `switch` and `default:`\n\nWhether reaching a `default:` is a programming error is a property of where the switched value came\nfrom (§2), not of `default:` itself. A deliberately-partial switch and a forgotten case look\nidentical in source, so the source says which it is.\n\n- \n**A closed internal enum with every enumerator handled** : write no`default:` . A`default:` disables`-Wswitch` , which the build makes an error, and trades a compile-time check for a\nruntime one. The switch’s opening comment contains the word`exhaustive` :\n\n```\nswitch (placement) { // exhaustive: every box placement has its offset\n```\n\n When every case returns, the line after the switch is reached only by a value that is no enumerator — an invariant violation. It is an invariant check (§3.2) followed by the recovery, never a plausible-looking return value presented as ordinary code.\n- \n**A closed internal enum of which only some enumerators matter** : name the others as grouped\ncases sharing one body, so an enumerator added later still reaches the compiler. No`default:` .\n- \n**Boundary input** (a grammar token, a landscape field, an imported attribute): switch on it only\nafter its checked conversion (§5.3, §7), which is where an unknown value is refused or adapted\n(§4). A switch over a raw boundary value — an int or a character from a file — has a`default:` that refuses or adapts, by its class.\n- \n**A third-party enum** (Qt, the OS, a library): a`default:` is legitimate “not interested”, and\nits comment says so:\n\n```\nswitch (event->type()) {\ndefault: // every other event type is uninteresting here — not a missing case\n break;\n}\n```\n\n- \n**A `default:` in a switch that already covers every enumerator** never runs and only switches`-Wswitch` off: there is none.\n\nEvery `default:` carries a comment saying which kind it is (§18 rule 6).\n\n### 5.3 Ints that stand for enumerators\n\nAn int becomes a project enum only through a checked conversion that yields no value for an int\nthat is no enumerator (an `std::optional`). The places such ints come from are boundaries: Qt item\ndata, a compiled cache read back, a Lua argument. A combo stores the enumerator as its item data,\nnever its position, so the enum’s order is no contract with the GUI. After the conversion, the\nswitches downstream are internal again.\n\n## 6. Explicit values\n\n- **Empty means empty.** An empty string or container never stands for something else (“if empty,\nassume X”). Where “not given” must be told apart, the value is an`std::optional` , read by the\noptional reader of the JSON read layer (`EACMP_jsonOptionalString` ); where a default applies,\nthe default is stored as an explicit value. A stated value that may not be empty has the\nschema’s`minLength` . An inclusion filter, which restricts nothing when absent and admits\nnothing when stated empty, is an`EACMP_NameFilter` or`EACMP_DomainFilter` (`types.h` ).\n- **A display text is never a value.** A combo’s “(default)” entry is shown, never stored or\ncompared; the state it stands for is an explicit value — a referral token, an enumerator, an\noptional.\n- **Stored data carries explicit values.** A referral token such as`@@view` is a value stored in\ndata, not a hidden sentinel.\n\n## 7. Closed vocabularies\n\nA **closed vocabulary** is a field whose value is one of a fixed set the application defines: a\nlabel placement, an alignment, a rule phase, a log level. Open vocabularies — type, shape, colour\nand scheme names — are defined by grammars and stay strings.\n\n- **An `enum class`** , never an unscoped enum: an unscoped enum converts to int implicitly and its\nenumerators leak into the enclosing scope.\n- **One descriptor per vocabulary** : the pairs`{enumerator, wire string}` , beside the enum, as an\narray of`EACMP_VocabularyEntry` (`vocabulary.h` ). The conversions both ways are derived from it\nby`EACMP_vocabularyFromString` and`EACMP_vocabularyToString` , never written out again as an`if` chain or a switch per direction;`EACMP_vocabularyWireStrings` lists the spellings for a refusal.\n- **The from-string conversion reports an unknown value and never substitutes one.** The reader\ndecides by the input’s class (§4.2): a grammar or scheme reader refuses it, a landscape or import\nreader adapts it through its helper (§4.3).\n- **Parsed where it enters, written where it leaves.** Inside the program the value is the\nenumerator, never its string.\n- **No number stands for an enumerator** — no 0/1/−1 for an axis, no bits for a set of events:\nthe enumerators, or a set of them, say it. A set of enumerators is an`EACMP_EnumSet` (`enum_set.h` ) over an enum of plain enumerators, never an enum whose values are bits.\n- **The schemas list the vocabulary as an `enum`** , so a misspelt value fails schema validation.` vocabulary_parity_test` holds every descriptor against the schema sites that state it, and\nevery schema`enum` against the descriptor that claims it; a vocabulary no schema states (a\npreference, a cache field) says why. The audit (§18) requires every descriptor in`eacmplib/include` to be registered there, so a new descriptor cannot escape the test.\n- **A field takes its own vocabulary.** Where two fields share an enum but not its whole range —\nthe legs and the result of a derivation — each field has its descriptor, and a value valid only\nfor the other field is unknown here.\n- **A type cannot hold a value its user cannot take.** Where one enumerator is a different mode\nrather than one more choice, it is a different type, so no function has an impossible case.\n- **The wire string is not display text.** What a person reads for an enumerator — a combo’s\nentry, its tooltip, the`\"(default)\"` entry of a field that follows its default — comes from the\npresentation layer,`eacmplib_presentation` (`EACMP_presentation` , one exhaustive switch per\nvocabulary), and qtgui translates it through one hook. A combo stores the enumerator as its item\ndata, never an index or a text, and reads it back as the enumerator.\n\n## 8. Named constants\n\n- **Numbers.** A literal other than 0, 1 and 2, halving, and angle arithmetic over 90/180/360 is a\nnamed constant:`inline constexpr` , typed and scoped, in the header of its subject — a domain\nvalue with its subject in eacmplib, a GUI metric in`gui_constants.h` , tolerances together\n(`tolerances.h` , one name per question), the scene’s z-layers and an item’s z-parts as one\nordered table each (`EACMP_QT_ZLayers` ,`EACMP_QT_ZParts` ), a colour channel’s bound as`EACMP_COLOR_CHANNEL_MAX` . Where the named constant exists, the literal is not written. A local\nname for a tolerance takes its value from the header.`#define` only for what must be a macro.\n- **The standard’s own values first.** Bits per byte are`CHAR_BIT` , a field’s width`sizeof(T) * CHAR_BIT` , an all-ones mask`std::numeric_limits::max()` , a collection’s size its\nown`size()` . An extreme is`std::numeric_limits` (`infinity()` ,`lowest()` ), or`std::optional` for “no value yet”. What the standards lack is named once with its source: the units\n(`units.h` ), the FNV-1a and golden-ratio hash constants with the one hash combiner\n(`EACMP_hashCombine` ,`types.h` ).\n- **Text is built as `std::string`** , never in a hand-sized character buffer. Where a C API needs a\ncaller’s buffer, its size is a named constant citing the API’s bound, the buffer a`std::array` ,\nand truncation is reported. Prefix and suffix tests go through`EACMP_startsWith` /`EACMP_endsWith` (`text.h` ), so no length stands beside its text; a fixed text format’s length\ncomes from its definition and is checked in its one parser. Character classes and case go through`EACMP_classicCharacters()` , never by arithmetic and never through “, whose answers for a\nbyte above 0x7f follow the process’s locale.\n- **Defaults are named once.** A default is a`const` in the header of the type or function it\ndefaults, and the member or parameter takes its value from it. A preference’s default is\ndeclared with its key (`EACMP_PreferenceKey` ,`preference_keys.h` ); a read passes none and the\nstore answers the declared one. Shared choice lists and ranges are named once beside what they\nconstrain.\n- **Structures have names.** A fixed collection is indexed by an`enum class` naming its members\nand ending in`Count` , held in an`EACMP_EnumArray` (`enum_set.h` ); a loop runs over a collection\nor over`EACMP_enumerators()` , never up to a number. A positional layout — a JSON array of fixed\nshape, a function’s arguments, a tool’s command line, a table’s columns — is an enum of places,\nand its arity check and its indexing use it; a strict JSON object’s key count comes from its key\ntable (`EACMP_jsonHasExactlyKeys` ). A combo entry is found by the value it holds, never by its\nposition.\n- **The GUI’s look is named.** An icon’s geometry is data: an SVG glyph in EACMPBase drawn by the\nshape renderer (`EACMP_QT_paintBaseShape` ), never coordinates in code. A colour is a named\nfunction in`gui_constants.h` (eacmplib’s own, the placeholder grey, in`types.h` ); a GUI metric,\na duration and a font size are named constants there, a duration and a font size with the unit\nin the name (`_MS` ,`_PT` ), an animation’s key values beside its duration. A file’s own timing\nconstants stay with it where it has them.\n- **Parameters and formulas.** An algorithm’s or a feature’s parameter — a search budget, a\nnumber of halvings, a bound, a step, a kept count — is a named constant stating why it has its\nvalue, in the header of what it tunes. Every JSON text written takes one indent\n(`EACMP_JSON_INDENT` ). A formula’s coefficients stay literal in the one function that names the\nformula and writes it out in its comment, and go on the accept list citing it.\n- **Tables and specification constants** — a key-code table, a UUID’s bit layout, fixture\ncoordinates, a generator’s recipe — are data and stay literal, with their source named.\n- **Protocol tokens** — strings that are compared or composed as protocol: a wildcard, a prefix, a\nfile suffix, a referral token — are named constants. A typo in a literal compiles and silently\nmisses.\n- **JSON keys.** A codec takes its keys from the format’s field descriptor, so a key is spelled\nonce per format.\n- **Text** for people is not a constant: messages are prose and stay where they are written;\nqtgui marks user-visible text with`tr()` .\n\n## 9. Derived coverage, not listed coverage\n\nWhere several places must range over one set — every field of a struct, every enumerator of an\nenum, every item a container owns — the coverage is derived from **one** enumeration, never\nrestated as a list per place. The test a mechanism must pass: **adding a member must not require\nediting anything else.** An array sized from a `Count` sentinel passes it; a container with a\nhand-written `clear()` naming its members does not — that reduces the places to forget from N to\none, which is a mitigation, not a fix. Where the last edit genuinely cannot be removed (a visitor\nstill names the members it visits), the omission is made **loud** rather than silent: a coverage\nassertion, an exhaustive switch the build checks, or an audit rule.\n\nIn force at: the field registries of every document record driving storage, clipboard and history\n(`document_field.h`, `Record::fieldRegistry()` + `field_parity_test`), the bus-topic registry and\nits panic guard, `referral_chains.h` + `referral_chain_test`, `EACMP_TypeVariantKey`, the\nlandscape-variable registry, the exposure of fields to logic (read and written through the field\nregistries, the open set held against them by `logic_test`), the canvas’s overlay array and\nscene-group visitors, and the closed vocabularies’ descriptors (§7).\n\n## 10. Errors without exceptions\n\n- **Errors are reported through logging and return values.** Project code throws no C++\nexception. A library that throws is caught at the call that can throw, and the failure becomes\na return value there.\n- **Exceptions where an external library needs them.** A`try` /`catch` is acceptable where an\nexternal library is concerned or where it cannot be avoided, and nowhere else. Lua, built as C++,\nraises its errors as exceptions that its own protected call catches; they pass through the Lua\nbridge’s functions, which neither throw nor catch them.\n- **A hardcoded fallback value is a recovery from our own error** : it follows the invariant check\n(§3.2) or an Error-level log, with a comment saying why the fallback is safe. Adapting outside\ninput is not a fallback: it goes through its helper at Warning (§4.3).\n- **Never crash on outside input.** A defect in what the program reads is refused or adapted by its\nclass (§4.2); only a programming error may abort, and only through an invariant check.\n\n## 11. The naming principle\n\nThe goal is self-documenting code: a reader understands a line from its names alone, without a comment, a lookup or a trip to the declaration. A name therefore documents its purpose, role and behaviour, and is optimised for the reader, not for typing.\n\nA name, or a part of one, that is not an accepted short name — a universally recognised idiom, a\nliteral Qt/STL API term, a clear short word (§12, §18 rule 3) — is spelled out. A name that needs a\n`// xy = meaning` comment to be understood is too short: the name is expanded and the comment not\nwritten.\n\nThe one exception is a name kept as a specification, an algorithm or a published artefact spells\nit, so that the code reads against its source: the W3C arc algorithm’s `x1p`, the BD relations\ntable’s `bd`. Such a name is accepted at its site, with its reason, in the accept list of the audit\nthat would flag it (`tools/var_audit_accept.txt` for §18 rule 3, `tools/name_audit_accept.txt` for\n§19), and its declaration carries `// xy = meaning` — the name or its short part on the left, and\na meaning that says more than the name does (§15).\n\nDo not weigh “minimise typing”, compile speed, or debugger convenience against clarity. Clarity wins.\n\n## 12. Spelled out or kept short\n\nA name is spelled out — written as the descriptive name — when it is one of these:\n\n- A domain abbreviation: `ve` ,`vr` ,`me` ,`mr` ,`ls` ,`ctx` ,`prefs` ,`lp` , … (§13.1).\n- A cryptic prefix or a packing of stubs: `fit` ,`dblVr` ,`szVe` .\n- A single letter or stub that stands for a specific thing: a loop index (§13.3), `idx` (§13.4),`v` /`cur` for a view or a current value, a parametric`t` (§13.6), a rotation`QTransform t` .\n- A math single letter, written as a descriptive local (§13.6).\n\nA short name is kept when it is one of these:\n\n- **Qt/STL API and full English words** :`combo` ,`key` ,`item` ,`row` ,`data` ,`ok` ,`pen` . They\nare already human-understandable and are largely the API itself (`->item(row)` ,`->data(Qt::UserRole)` ,`setData(...)` ,`rowCount()` ,`pathItem->pen()` , the`&ok` dialog idiom).\n- **Unit suffixes** :`…Pt` (points),`…Px` (pixels).\n- **The `m_` member prefix** , public Qt class names, signals, slots. Only an abbreviated suffix\nafter`m_` is spelled out (§13.2).\n- **Unchangeables** :`tmp` for an actual temp-file path; any letter inside a string literal, which\nis text, not a name.\n\nA name kept as a specification or a published artefact spells it is neither: it is §11’s exception, accepted at its site.\n\nSide by side:\n\n- `viewElement` , not`ve` ;`eacmpViewUUID` , not`lu` (the`EACMP_Uuid` of a Qt`viewUUID` ).\n- `for (int groupIndex = 0; groupIndex pen();` ,`bool ok = false;` ,`item->data(Qt::UserRole)` ,`for (auto* combo : combos)` .\n\n## 13. Naming conventions\n\n### 13.1 Domain abbreviations\n\nSpell these out (write the right column):\n\n| write | not | | write | not | \n|---|---|---|---|---|\n| `viewElement` | `ve` | | `modelElement` | `me` | \n| `viewRelation` | `vr` | | `modelRelation` | `mr` | \n| `landscape` | `ls` | | `genericGrammar` /`grammar` | `gg` | \n| `context` | `ctx` | | `appManager` | `mgr` | \n| `preferences` | `prefs` | | `labelProperties` | `lp` | \n| `elementType` /`relationType` | `et` /`rt` | | `elementTypeInfo` /`relationTypeInfo` | `eti` /`rti` | \n| `colorScheme` /`styleScheme` | `cs` /`ss` | | `languageTag` /`language` | `lang` | \n| `modelUUID` | `mu` | | `elementUUID` /`relationUUID` | `eUuid` /`rUuid` | \n| `undoStack` | `us` | | `description` | `desc` | \n| `position` (local only — never`event->pos()` ) | `pos` | | `button` | `btn` | \n\nA token that means different things at different sites — `w` a window or a warning, `c` a canvas\nor a colour — has no row: its expansion is read from the site (§14).\n\n### 13.2 Structural conventions\n\n- **Qt ↔ eacmplib UUID locals.** Name a local that holds the`EACMP_Uuid` form of a Qt`QUuid` (a`fromQUuid(...)` result)`eacmpUUID` . Write`eacmpViewUUID` instead of`lu` .\n- **Template type parameters** carry the`_EACMPType` suffix: write`Item_EACMPType` ,`Value_EACMPType` ,`Scheme_EACMPType` , not bare`T` /`ItemT` .\n- **nlohmann::json.** Name variables`…J` :`cacheJObject` ,`rulesJArray` . Name the grammar\ninternal helper functions`jsonString` /`jsonObject` /`jsonDouble` /`jsonArray` /`jsonBool` /`jsonInt` .\n- **pugixml handles.** Prefix a variable that holds a pugixml node, document, or path with`xml` :`xmlNode` ,`xmlChild` ,`xmlDocument` ,`xmlRoot` ,`xmlPath` ,`xmlSegment` — not bare`node` /`doc` /`seg` .\n- **Members.** The`m_` prefix stays and the suffix is spelled out:`m_preferences` , not`m_prefs` ;`m_pointerButton` , not`m_btnPointer` . A member whose suffix is already a clear word\n(`m_view` ,`m_scene` ,`m_labels` ) is as it should be.\n\n### 13.3 Loop indices\n\nName a loop index for what it indexes — `Index` from the iterated or subscripted\ncontainer:\n\n```\nfor (size_t pointIndex = 1; pointIndex elements())`. Read\n `conflict.kind`, not `it.kind`.\n- **An index** — a number used as `container[x]` — takes `Index` (§13.3, §13.4).\n- **A held STL cursor** — a `find()` result tested against `end()`, or a `begin()/end()` cursor\n where range-for will not serve — takes `Entry`: `fillItemEntry` for\n `m_elementFillItems.find(…)`, `relationLabelEntry`, `overlayStrokeEntry`. Two cursors over one\n map in one function scope are told apart by their role: `relationEndEntry` in the pass that\n hides, `oldEndEntry` in the pass that rebuilds. Where only the value is needed, the cursor is\n dereferenced into a named object.\n\n### 13.6 Math idioms\n\nThe math stays local (the compiler folds it), named descriptively, so that no comment has to\nexplain the letters:\n\n``` cpp\ndouble angleInRad = qDegreesToRadians(degrees);\ndouble cosineOfAngle = std::cos(angleInRad), sineOfAngle = std::sin(angleInRad);\nreturn QPointF(point.x()*cosineOfAngle - point.y()*sineOfAngle,\n point.x()*sineOfAngle + point.y()*cosineOfAngle);\n```\n\nWrite `cosineOfAngle` instead of `c`, `sineOfAngle` instead of `s`, `angleInRad` instead of `rad`,\n`fraction` instead of a parametric `t` (`fractionA`/` fractionB`, `fractionStart`/` fractionEnd`),\n`rotationTransform` instead of a `QTransform t`, and `lhs`/` rhs` instead of sort-comparator\n`a`/` b`.\n\n## 14. Renaming\n\nA name that falls short of §11–§13 — found by the audits (§18 rule 3, §19) or in review — is\nrenamed. The new name comes from what the variable means at its occurrence, never from the short\ntoken: one token stands for different things at different sites (`w` a window or a warning; `c` a\ncanvas, a conflict or a colour; `s` a scene position, a string, a scale or a sine), so a token is\nnever swapped project-wide. A rename is a word-boundary token swap within one binding, and guards\nagainst:\n\n- **Reaching past the binding.** A loop index, lambda parameter or block local is renamed only\nwithin its scope; reaching past it silently mis-names a different same-letter variable. Two\nscopes that share a function get distinct names, or the compiler reports a redefinition.\n- **Param-vs-local collision.** Renaming both a`…Pt` parameter and a local to`…Point` self-assigns (`QPointF insidePoint = insidePoint;` ); the local gets a distinct name\n(`insideBound` ).\n- **Method calls that look like variables.**`pos → position` does not touch`event->pos()` (`QContextMenuEvent` has no`position()` ;`QMouseEvent::position()` returns a different type).\n- **String literals.** A bare token can sit inside a string — the article`a` in`tr(\"select a landscape\")` , the unit in`tr(\"%1 pt\")` , test labels`\"le true eq\"` ,`\"re-set\"` .\nA token that occurs in a string is never renamed blanket; the rename scopes around it.\n- **The build verifies.** Under-reach surfaces as “use of undeclared identifier”; a collision as\n“redefinition”; the rest as type errors.\n\n## 15. Comments\n\nA comment says what the code cannot: the why, a unit, an invariant — never a restatement of the code. Each is terse, a line as a rule, and together they stay at about one comment line per ten lines of code at most; fewer is fine. The code carries these:\n\n- **A function’s purpose** , on the line after its opening`{` (§18 rule 1).\n- **A branch’s reason** , on every`if` ,`else if` ,`else` ,`for` and`while` : between the keyword’s\nline and the`{` ‘s line, or on the comment lines right after the`{` ; a one-line body\n(`if (…) { …; } // comment` ) carries it as its trailing comment (§18 rule 2). It says why the\nbranch is there, and where an`if` has no`else` , why none is needed, with the`no 'else'` marker (§5.1). A`switch` without`default:` carries`exhaustive` in the same place (§5.2).\n- **A case’s meaning** , after a`case` or`default` label whose body is more than a bare jump or\nthe next label; every`default:` says which kind of default it is (§5.2, §18 rules 2b and 6).\n- **A type’s description** , on the line before a`struct` or`class` definition (§18 rule 4).\n- **A guard’s reason** (`// GR-2: grammar may be uninstalled` , §4.1) and**an invariant’s number** where code relies on one or asserts it (§3.1).\n- **What a name cannot carry** : a unit, a range, the formula whose coefficients a function keeps\nliteral (§8), the reason a value or a form was chosen.\n\nA `// name = meaning` comment whose right-hand side only restates the name\n(`// pointIndex = point index`) says nothing the name does not; it is not written (§11, §18 rule 8).\nA name kept as a specification spells it (§11’s exception) carries exactly this comment at its\ndeclaration, with a meaning the name cannot give: `// x1p = x1' of the W3C arc notes`.\n\n**Cite only durable artifacts.** A comment may point at a reference doc (`docs/EACMP-*`), an\ninvariant/rule ID (I-12, GR-3), or a symbol — never an **ephemeral** document: a dated working doc,\na post-mortem, a `*-changes.md` ledger, a dated `YYYYMMDD-*.md` working document, `queries.txt`,\n`goal.txt`, `todo-later.txt`. Those get archived or deleted and leave the citation dangling. State\nthe fact inline instead. This is the reference documents’ own citation rule — they cite only each\nother — extended to code, a comment being a durable artifact too. Nor does a comment carry\nhistory: it states the timeless reason, never the date, the former behaviour or the process that\nproduced the code.\n\n## 16. Shape SVGs\n\nA shape SVG carries two kinds of identifier, and only one of them is ours:\n\n- **The SVG specification’s own vocabulary** — element and attribute names (`viewBox` ,`cx` ,`rx` ,\nthe path-data command letters`M` /`L` /`C` /`A` ) — is the file format talking. Never rename it,\nand never rename our own code’s mirrors of it (§12, §1.1).\n- **The `data-vars` identifiers** are declared by the shape author, read only by the constraint\nevaluator, and mean whatever that shape decides. They are ours, and §11–§13 apply:`foldSize` ,`curlControlOffset` ,`plugWidth` ,`bodyTop` — never`f` ,`d` ,`b` ,`bt` . A`data-vars` name is\nreferenced from`data-constraints` in the same file and from nowhere else, so a rename is\nfile-local; grammars recompile on their own because the cache’s input manifest sees the change.\n\nReserved names are the exception in the other direction: `targetWidth`, `targetHeight`, `scaleX`,\n`scaleY`, `labelWidth`, `labelHeight`, `labelVisible` are put into scope by the evaluator, and\n`cornerRadius`, `cornerInset`, `minimumWidth`, `minimumHeight` are read back out of it. Those four\noutputs are a contract with the engine: keep the spelling exactly, in every shape that uses them.\n\n## 17. Structure\n\nA routine is read top to bottom by someone who did not write it. Its length, its depth and the number of its decisions are what that reader has to hold at once, so each has a limit.\n\n### 17.1 The limits\n\n| Measure | Limit | \n|---|---|\n| Lines of a routine, comment lines included, blank lines not | 60 | \n| Lines of a block inside a routine | 40 | \n| Depth of a block: the routine’s body is depth 0, a block directly inside it depth 1 | 3 | \n| Flow-control statements of a routine: `if` ,`else` ,`for` ,`while` ,`do` ,`switch` ,`case` ,`default` ,`goto` ,`?:` ,`try` ,`catch` (`else if` and`do … while` count once) | 15 | \n\nA block is every `{}` inside a routine: the body of a branch, a loop, a `switch`, a braced `case`,\na bare block, and the body of a lambda. A lambda is therefore a level, and its lines and decisions\ncount against the routine that defines it. The limits are those of the structure measurement,\n`findcompl.py --all` at its default thresholds.\n\n### 17.2 The forms that keep a routine within them\n\n- **A phase is a function.** A routine whose comments divide it into phases is a sequence of\ncalls, one function per phase; the routine that remains reads as the outline its comments were.\n- **Shared state is a named struct.** What several functions share travels as one context struct\npassed by reference, every field named for what it holds (§6: no field means “absent” by being\nempty). A field one function writes and another reads is then a line in the struct, not a\ncapture.\n- **A lambda is short.** A lambda longer than a few lines is a function. A family of lambdas that\ncapture the same locals and each other is a class: the locals become its members, the lambdas\nits member functions. No capture can then be missed — the compiler checks every member — and\nthe original function constructs the class and runs it.\n- **A recursive lambda is a static function.** Recursion through`std::function` becomes a\nfunction taking its context, defined once rather than on every pass of the loop around it.\n- **A body is not wrapped.** Where a function takes a callable — an undo action, the\napplication’s action wrapper, a signal connection, a subscription — a body of more than a few\nlines is a function, and the callable calls it.\n- **A dispatch calls handlers.** A`switch` whose cases hold more than a few lines calls one\nfunction per case; the switch stays as the list of cases.\n- **A guard returns early only in tail position.**`if (!condition) { return; }` (or`continue` ,`break` ) replaces`if (condition) { … }` only when the block is the last statement of its\nroutine, loop body or case: the jump’s`else` is the rest of the block (§5.1), and an early exit\nbefore further work would skip it. A guarded block followed by more work is extracted first, so\nthat its guard is last in a function of its own, or it stays as it is.\n- **A list is split along its own groups.** A declarative list — menus, actions, registrations, a\nfield registry, the rows of a panel, the docks of a window — stays a list, split per menu, per\ngroup, per row or per dock; an entry whose body holds logic calls a function.\n- **Twins split as twins.** The element and the relation version of a piece of code are split\ntogether and stay textually parallel; a split shares along the dimension axis (colour, style,\ngeometry), never across the two kinds.\n- **A per-item path stays as fast.** A split on a path that runs per item — scene rebuild, hit\ntesting, the properties refresh, drag, move and resize, the data layer under them — passes its\ncontext by reference, adds no`std::function` and no allocation per item, and is measured\nbefore and after.\n- **A split moves code, it does not change it.** No condition, value or message changes in the\nsame change as a move; a defect found while moving is fixed in a change of its own.\n\n### 17.3 Over a limit\n\nA unit stays over a limit only where every form above would make it worse: where a split would move code whose silent change corrupts data and no test would notice, where the split reads worse than the whole, or where it moves the complexity to another place instead of removing it. Such a unit is recorded where it is documented, with its measurement and the argument — “not here, because …” — stated so that it can be checked. Length alone is no argument, and neither is the unit’s age.\n\nOne such hold applies everywhere: **a braced `case` body is a level.** The braces carry the brace\nrule and the case’s opening comment, so the level is accepted; a case that nests further than its\nown body calls a handler.\n\nThe frozen migration ledgers (§1.1) are exempt, as they are from the audits.\n\n## 18. The audit (`make audit`)\n\n`tools/audit_comments.py` mechanically checks the conventions above and writes reports to\n`audits/`; the rules that judge statements rather than lines (2, 5, 9, 10, 11 and rule 7’s protocol\ntokens) are in `tools/audit_structure.py`, which reads the code with comments, literal bodies and\npreprocessor lines blanked. This document is the authoritative description of what it checks — the\ntools cite this section rather than repeating the descriptions.\n\n`make check` and `make check-full` run `make audit-check`. A finding of a **blocking** rule fails\nthe check; the findings of the other rules are **advisory**, printed as a warning. A rule becomes\nblocking once the code holds at zero findings of it, so what is fixed stays fixed: which rules block\nis the tool’s `BLOCKING_CATEGORIES`, printed on every run. Every rule below blocks. The standalone\ntargets below retain their reports under `audits/`.\n\n- `make audit` — one report per file under`audits/` .\n- `make combined-audit` — a single combined report.\n- `make audit-check` — a summary using temporary output, with no retained report.\n- `make audit-lib` /`make audit-gui` /`make audit-tools` /`make audit-tests` — scope to`eacmplib` /`qtgui` /`tools` /`mechanical_tests` .\n\nEach finding carries an issue-type slug in brackets (e.g. `[else]`); cite a rule by that slug or by\nits number (“audit rule 5” = the `[else]` rule).\n\n1. \n**`[func]`** — a function definition’s opening`{` is followed by a comment stating the\nfunction’s purpose (§15).\n2. \n**`[flow]`** — every` if` /`else if` /`else` /`for` /`while` branch has an opening comment\n(§15): a comment from its keyword’s line to its`{` ‘s line, or on the comment lines right after\nthe`{` . That includes`} else if (…) {` and`} else {` , a one-line body (`if (…) { …; } // comment` ), whose comment trails the line, and a brace-less body, whose comment may sit on any of\nits lines. An empty block, a do-while’s condition, and a nested loop already covered by the outer\nloop’s comment are exempt.\n - **2b. `[case]`** — a` case` /`default` label is followed by a comment, unless the body is a\nbare`break;` /`return;` /`{` or the next label.\n3. \n**`[var]`** — flags a declaration whose name is a cryptic short name. Three checks:\n - the whole name is ≤ 2 characters or an abbreviation-map entry (§13.1) — `gg` ,`eti` ,`ve` ;\n - a structured-binding name (`auto [a, b]` ) is cryptic-short —`[k, v]` ;\n - a longer name **embeds** a cryptic 2–3 char part when split on camelCase (`hitVr` →`hit` +`vr` ).\nThe embedded check is an**allowlist** : a 2–3 char part is cryptic unless it is an accepted\nshort part (units`pt` /`px` , the §12 keep-words, clear short English words, domain\nacronyms). Extend`ACCEPT_SHORT_PARTS` in`tools/audit_comments.py` — not the code — when a\ngenuinely clear short part is flagged. This catches novel abbreviations without enumerating\nevery bad one.\n Rename per §11–§13.5, or accept: a genuinely clear short part on `ACCEPT_SHORT_PARTS` ; a name\nkept as a specification or a published artefact spells it (§11) on`tools/var_audit_accept.txt` , with its reason, which accepts it only on a declaration carrying its`// name = meaning` comment (the name or its cryptic part on the left) — a listed name without\nthe comment, and an entry no cryptic declaration of its file uses, are findings.`i` /`j` /`k` /`it` are**not** exempt (§13.3, §13.5 forbid them); the only kept short names are the clear English\nwords / API idioms of §12 (`key` ,`item` ,`row` ,`data` ,`ok` ,`pen` ,`combo` ).\n4. the whole name is ≤ 2 characters or an abbreviation-map entry (§13.1) — \n5. \n**`[type]`** — a` struct` /`class` definition is preceded by a short descriptive comment (§15).\n6. \n**`[else]`** — an` if` chain ends in an`else` , or its**last** branch ends in a jump (`return` ,`continue` ,`break` ,`goto` ,`throw` , or an exit — also a last statement that is itself a chain\nevery branch of which jumps), or the last branch’s**opening comment** (rule 2) notes the no-else\nreason and contains the marker`no 'else'` (matched case- and quote-insensitively as “no else”).\nAn`else if` is a branch, never the chain’s`else` . See §5.1 “Every branch is deliberate”.\n7. \n**`[default]`** — a` default:` label carries a comment saying WHICH kind of default it is\n(§5.2): a genuinely uniform remainder, boundary input being refused, or a third-party enum’s\nuninteresting rest. The comment may sit inline, on the line above, or as the first line of the\nbody (the ordinary flow-comment position). Unlike`[case]` , a bare`break;` /`return;` body is**not** exempt — a silent remainder is exactly the shape the rule exists to catch. A switch over\na closed internal enum has no`default:` at all, which rule 9 checks (see §5.2).\n8. \n**`[token]`** — a scheme referral level is written as its named constant, never as a raw`\"@@…\"` string literal (`EACMP_SCHEME_VIEW_DEFAULT` , not`\"@@view\"` ; see`scheme_tokens.h` ).\nThese tokens are wire vocabulary — a typo (`\"@@veiw\"` ) compiles and silently misses. Exempt:\nthe`scheme_tokens.h` definitions themselves, and any`EACMP_…` constant-definition line (the single sanctioned home for such a literal). The same holds for the\nprotocol tokens of §8 — the wildcard`\"*\"` , the`\"pcre:\"` prefix, the file suffixes`\".eacmpgrammar.json\"` ,`\".eacmpgrammar.compiled.json\"` and`\".eacmpl.json\"` , and a boolean’s\nspellings`\"true\"` and`\"false\"` (`EACMP_TRUE_TEXT` ,`EACMP_FALSE_TEXT` , read by`EACMP_booleanFromText` ), matched as whole literals (`PROTOCOL_TOKENS` in`tools/audit_structure.py` ). Prose in comments is not flagged; the\nmigration tools freeze their own copies and are not scanned;`mechanical_tests/` is exempt because\nthere the literal is the subject under test (§1.1).\n9. \n**`[restate]`** — a`// name = meaning` (or`// name: meaning` ) comment whose right-hand side\nonly spells out the words the name already carries (`// pointIndex = point index` ,`// signalByte = signal byte` ). A descriptive name makes that comment redundant, and §11 forbids\nthe crutch, so the comment goes — the name stays. Filler words (“the”, “each”, “current”) do\nnot count as saying something new; a comment that adds a unit, a range, an invariant or a why\ndoes, and is not flagged. The name must occur in the code on the same line, or on the next line\nwhen the comment stands above the declaration.\n10. \n**`[switch]`** — three checks per` switch` (§5.2):\n - a switch with no `default:` carries the word`exhaustive` in its opening comment (the comments\nfrom its keyword’s line to the line before its first label);\n - a switch whose labels all name enumerators of one project enum has no `default:` — the\nenumerators it does not handle are grouped cases (an enum the project does not declare, or a\nQt,`std` , nlohmann, pugixml, OGDF or Lua qualifier, is third-party and may keep its`default:` );\n - after a switch with no `default:` whose every case returns, the next statement is an invariant\ncheck (`EACMP_BREAKING_…` ); a switch that ends its block that way is flagged too.\n11. a switch with no \n12. \n**`[vocab]`** — two checks (§7):\n\n- a string literal that a schema lists in an `\"enum\"` (`schemas/*.schema.json` , read at each\nrun, so the vocabularies are stated once) is compared with`==` or`!=` : the value is a closed\nvocabulary’s wire string, to be converted to its enumerator where it enters. A literal that\nonly shares a spelling with a schema value and belongs to no closed vocabulary goes on`tools/vocab_audit_accept.txt` — — with its reason;\n- a descriptor declared in `eacmplib/include/` (an array of`EACMP_VocabularyEntry` ) is named in`mechanical_tests/vocabulary_parity_test.cpp` , which holds it against its schema sites — the\nderived coverage that keeps a new vocabulary from escaping the parity test.\n\n`mechanical_tests/` is exempt from the first check. Not to be confused with `make audit-vocab`,\nwhich holds the grammar loader’s accepted *keys* against the schemas.\n11. **`[number]`** — a numeric literal outside a named definition (§8). Exempt: the values 0, 1, 2,\n0.5, 90, 180, 270 and 360 in any spelling; a literal in a `const` or `constexpr` declaration’s\ninitializer, an enumerator’s value, or the one `return` of a parameterless function, which names\nit; preprocessor lines; `mechanical_tests/`, whose literals are test data. A table, a\nspecification’s constants or a formula’s coefficients go on `tools/number_audit_accept.txt` —\n , or `*` for a whole table file — each with its reason.\n\n`make audit-lib` / `audit-gui` / `audit-tools` / `audit-tests` scope a run to one area; with no\nfile arguments the audit covers all of them.\n\nEach run also prints an advisory **long-function count** (definitions over 150 lines, longest\nfirst). It is not an issue category and never fails anything: it exists so a standing inventory of\ndeliberately-long functions can be reconciled against measurement instead of memory.\n\nWhat the audit cannot judge about a switch — whether it names every enumerator, whether a\n`default:` can still run — the compiler can. `make survey-switches` (macOS, on demand) compiles\nevery project translation unit syntax-only with clang’s `-Wswitch-enum` and\n`-Wcovered-switch-default` and lists what they report. The flags are no part of the build:\n`-Wswitch-enum` also fires on a legitimate `default:` over a third-party enum, so the survey is read,\nnot enforced; the build’s `-Werror=switch` and rule 9 are the enforced half.\n\n## 19. Cooperative short-name audit (`make name-audit`)\n\nThe `[var]` rule catches short names *structurally*: ≤ 2 chars, abbreviation-map entries (§13.1),\nand camelCase names embedding a cryptic 1–3 char component (rule 3c, `hitVr` → `hit`+` vr`). A\n**whole-name-opaque** short identifier with no camelCase seam — `clvl`, `csch`, `svis` — gives\nrule 3c nothing to split, so it slips through. `tools/name_audit.py` (run via `make name-audit`;\n**advisory**, never part of `make check`) closes that gap with a dictionary check: it flags\nall-lowercase 4–5 letter declared names that are not a dictionary word, not a simple plural of one,\nand not on the accept-list `tools/name_audit_accept.txt`, writing\n`audits/-semiautomaticaudit.md` (vscode links, a per-name `fix`/` keep` field). A dictionary\nis a better discriminator than a vowel count at this length — a vowel rule flags real words like\n`child`/` width` yet misses vowel-bearing cryptic names like `svis`.\n\nIt is **cooperative and on-demand**: the owner reviews the report and marks each name `fix` or\n`keep`; a `keep` is appended to the accept-list so it never resurfaces, growing the list across\nruns. CamelCase cryptic names (`cType`, `rLang`) stay rule 3c’s domain and are out of this tool’s\nscope.\n\n## 20. Shape-variable audit (`make audit-svg-vars`)\n\n`tools/audit_svg_vars.py` (advisory, never part of `make check`) reads every\n`grammars/*/shapes/*.svg`, collects the `data-vars` declarations, and flags a declared name that is\nthree characters or shorter, or four-to-five characters with no camelCase seam. The engine’s own\nvariables (both the inputs and the four reserved outputs of §16) are never flagged. The tool prints\nonly the files with findings, so a clean run is one summary line.\n\n## 21. Documentation citation audit (`make audit-doc-refs`)\n\nThe reference documents cite code constantly, and a citation that no longer resolves sends the\nreader looking for something that does not exist. `tools/audit_doc_references.py` (advisory,\non-demand, never part of `make check`) extracts every backticked token in `docs/EACMP-*.md` that\nlooks like an identifier — an `EACMP_` name, an `m_` member, a camelCase name, a file name — and\nreports the ones that occur nowhere in the source tree. Prose that happens to look like code, an\nexternal API cited by name, and this document’s own examples-of-what-not-to-write go on\n`tools/doc_reference_accept.txt` once and never resurface. Advisory by design: an unresolved\ncitation may be documentation drift or a rename in flight, and only a human knows which.\n\n## 22. Structure ratchet (`make audit-complexity`)\n\nThe limits of §17 are measured by `findcompl.py`, which the repository does not carry; it gives\nevery file a penalty and a quality from 10 (no excess) down to 1. `tools/audit_complexity.py`\n(advisory, on-demand, never part of `make check`) holds what has been reached: it runs the\nmeasurement over every directory §1 names and compares each file with its line in the committed\nbaseline `tools/complexity_baseline.txt`. A finding is a file whose quality fell, a file whose\npenalty rose, a file the baseline does not hold whose quality is below 10, and a file the\nmeasurement can no longer read. A new version of the tool may weigh an issue differently while its\nquality scale stays the same, so penalties are compared only when the baseline’s first line names\nthe version that runs (`findcompl.py --version`); across a version change only the qualities are\ncompared, and the change is reported. A file no longer scanned is reported, and the frozen ledgers\n(§1.1) are printed as `EXEMPT`.\n\nIt is run when a piece of work is complete, before it is committed. `make audit-complexity-baseline` rewrites the baseline once the findings are dealt with: the issue\nremoved, or the unit held in §17.3’s form with its argument recorded.\n\n## 23. Duplication (`make audit-dry`)\n\nLogic written twice is a defect: a copy within one kind, or across the element and relation kinds\nwhere nothing of the kind is involved, is one function, or one template instantiated per kind. The\ntwins of §17.2 — the element and the relation version of code that is of its kind — stay textually\nparallel and are no defect. The copies are found by `finddry.py`, which the repository does not\ncarry (`FINDDRY` names it, as `FINDCOMPL` names the structure measurement): `make audit-dry`\n(advisory, on-demand, never part of `make check`) scans the `.cpp` and `.h` files of every\ndirectory §1 names, the frozen ledgers (§1.1) left out, and reports every block of at least 20, 15\nand 10 lines that occurs more than once — lines compared after whitespace normalisation, with\nblank lines, preprocessor lines and lines holding only a closing brace invisible to the search. It\nruns twice: once with the comments compared, and once with them cut and the string and character\nliterals kept, which finds the copy that differently worded comments hide and loses the one whose\nlength was its comments. The counts are a floor: a copy with one variable renamed is not the same\ntext. Each run ends with the tool’s summary — per length, the number of blocks and of surplus\nlines per 1000 lines of code — and the target exits 1 when either run found a block. A reported\nblock is read in the code, with its surroundings, before it is judged.\n\nIt is run when a piece of work is complete, before it is committed, beside the ratchet (§22).\n\nLast modified: 2026-10-24T14:17:26Z\n\n## So what are the key observations?\n\n- Claude generates code based on the patterns it was trained with. But — with the large context I have had it create regarding architecture, code quality and more — it can be constrained. Codex less so, that is a major reason why I dropped it.\n- I am running against the limit of the 1M context. A 1M context has its disadvantages anyway, anything that is further back has less impact on the LLM’s generation and your context can collect crud for the current task. We are working with a pure statistical pattern-based machine after all. It is amazing what it can do, but it is also fundamentally constrained in what it can do, as these patterns do not equate what a human would call understanding. And this shows up particularly clearly when we compare the LLM’s *generation* with the LLM’s*review* . LLM’s create code which an LLM — when asked to do a review — can make mince meat out of.\n- Even with all the context and audit tools I already had put in place, by itself, Claude doesn’t create code that is geared to being ‘good’ overall. And given that I experienced Claude itself falling for the risks of DRY, the conclusion must be that the context limits of Claude (1M tokens) and the way its statistics work have their limits. The obvious next step is to use swarms of LLM agents each having its own 1M context, each doing its own stuff, but it is clear that kind of kludge-like fragmentation will be hideously inefficient. Somehow I am reminded by earlier ‘combinatorial explosion’ bottlenecks that turned earlier AI booms to their busts (in the 1960’s and 1990’s). Nothing here is free, nor in cost, nor in quality.\n- It helps when — next to having enough volume in your context to create a ‘context harness’ — you create ‘symbolic logic’ tools to support the ‘constrained randomness’ of LLM code generation, but the (symbolic) constraining of non-understanding doesn’t create understanding, it just improves the quality of the approximations that the statistical generation brings. There is, therefore, no way that this approach is going to bring ‘AGI’ or ‘recursive self-improvement’, it is going to end in its own version of the ‘combinatorial explosion’. And no, LLMs aren’t ‘good at coding’ at their own. For small tools: yes. For serious code bases: no.\n\n*Next time I will report on how much better the code got after Claude reworked it based on the inspection results.*\n\n## Odds & Ends\n\nA few observations related to (future) cost and models (these are lab notes after all):\n\n- Anthropic (and the others) are looking at their cost as I noted in [the previous lab note](https://ea.rna.nl/2026/10/21/serious-coding-with-llms-lab-notes-2026-10-21-the-throttling-of-subscriptions-by-openai-and-anthropic-is-really-happening/) . Another sign: Claude has gotten an ‘idle compaction’ setting. If you’re not doing anything for a while, it will (unless you turn it off) compact your context (i.e.[summarise it](https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/) ). This is, I estimate, a cost issue. The less there is in your context, the less generating a next token costs them. But the summary can be less informative than the original. I expect that when the party is over (i.e. after the IPO), using larger contexts will also be priced. And smaller contexts really affect the usability on larger subjects like mine.\n - (Side note: if Anthropic succeeds with its massive IPO, OpenAI’s investors are probably cooked).\n- The old ‘thinking’ (NOT!) models were in fact multiple runs of the LLM exchanging text (tokens). They are a [‘recursive use’ of models](https://ea.rna.nl/2025/02/28/generative-ai-reasoning-models-dont-reason-even-if-it-seems-they-do/) as I wrote a while back. I suspect that the labs are moving this kind of LLM-LLM interaction to the*inside* of the models (directly, instead of via the roundabout way of exchanged texts between independent runs), hence that the ‘thinking’ (which can better be labeled the ‘tries’) can no longer be inspected in the most recent frontier models.\n\n## Previously on the LLM-coding experiment:\n\n• First report: [Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them](https://ea.rna.nl/2026/06/07/anthropic-openai-may-be-spending-more-than-1000-for-every-100-you-pay-them/) (7 June 2026)\n\n• Lab Note #1: [Serious coding with LLMs. Lab notes 2026-09-25: Codex vs Claude Code, Usage Limit Resets](https://ea.rna.nl/2026/09/26/serious-coding-with-llms-lab-notes-2026-09-25-codex-vs-claude-code-usage-limit-resets/) (25 September 2026)\n\n• Lab Note #2: [Serious coding with LLMs. Lab notes 2026-09-27: Codex succeeds, Claude reviews the result.](https://ea.rna.nl/2026/09/27/serious-coding-with-llms-lab-notes-2026-09-27-codex-succeeds-claude-reviews-the-result/) (27 September 2026)\n\n• Lab Note #3: [Serious coding with LLMs. Lab notes 2026-10-01: Codex and Claude generate a shared analysis](https://ea.rna.nl/2026/10/01/serious-coding-with-llms-lab-notes-2026-10-01-codex-and-claude-generate-a-shared-analysis/) (1 October 2026)\n\n• Lab Note #4: [Serious coding with LLMs. Lab notes 2026-10-13: Codex vs Claude on the same coding task](https://ea.rna.nl/2026/10/14/serious-coding-with-llms-lab-notes-2026-10-13-codex-vs-claude-on-the-same-coding-task/). (13 October 2026)\n\n• Lab Note #5: [Serious coding with LLMs. Lab notes 2026-10-21: The throttling of subscriptions by OpenAI and Anthropic is really happening](https://ea.rna.nl/2026/10/21/serious-coding-with-llms-lab-notes-2026-10-21-the-throttling-of-subscriptions-by-openai-and-anthropic-is-really-happening/)\n\n**Subscribe?** I don’t know if I am going to publish the appearance of new notes every time everywhere (it is a drag), so best to keep up to date with my observations and analysis on the Information Revolution is to subscribe (top right). You’ll get an email whenever I publish. These Lab Note are on a pretty high frequency schedule so far, but normally I haven’t been doing more than 1-2 a month.\n\n(I also like to be read. **Share?**)\n\n[You do not have my permission to use any content on this site for training a Generative AI (or any comparable use), unless you can guarantee your system never misrepresents my content and provides a proper reference (URL) to the original in its output. If you want to use it in any other way, you need my explicit permission]", "url": "https://wpnews.pro/news/a-code-inspection-of-serious-llm-generated-code", "canonical_source": "https://ea.rna.nl/2026/10/27/serious-coding-with-llms-lab-notes-2026-10-27-a-code-audit-of-llm-generated-code-improving-from-sludge-to-vakwerk/", "published_at": "2026-10-27 13:25:41+00:00", "updated_at": "2026-10-27 13:41:42.876420+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "generative-ai", "large-language-models", "ai-agents"], "entities": ["Claude Code", "Anthropic", "IfSQ", "ea.rna.nl", "Michael Larabel"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-code-inspection-of-serious-llm-generated-code", "markdown": "https://wpnews.pro/news/a-code-inspection-of-serious-llm-generated-code.md", "text": "https://wpnews.pro/news/a-code-inspection-of-serious-llm-generated-code.txt", "jsonld": "https://wpnews.pro/news/a-code-inspection-of-serious-llm-generated-code.jsonld"}}