{"slug": "working-to-make-python-lazy", "title": "Working to Make Python Lazy", "summary": "Python 3.15a7 introduces lazy imports, a feature proposed in PEP 810 that defers module loading until first use, potentially speeding up CLI applications and large codebases. The author has developed a helper tool, flake8-lazy, to automate converting existing code to use lazy imports, and reports that it was the first library they developed with heavy AI assistance. The feature supports both a new 'lazy import' syntax and a backward-compatible '__lazy_modules__' list, with flags like '-X lazy_imports=all' for testing.", "body_md": "Python 3.15a7, which is now just a `uv python install 3.15`\n\naway on all major\nplatforms, has lazy imports! This exciting feature, proposed in [PEP 810](https://peps.python.org/pep-0810),\npromises to make CLI applications faster (especially when using flags like\n`--help`\n\n), and could make a lot of large code with lots of imports that don’t\nalways get used faster too. Unlike the earlier, failed attempt, this requires\nlibraries to put in some work. I’ve developed a helper tool to make it easy; I’d\nlike to cover what lazy imports are and how to use my tool. Since this is the\nfirst library that I used AI heavily in developing, the second half of the post\nwill cover how my experience with AI for a task like this went.\n\nTL;DR: run `uvx flake8-lazy --apply=list`\n\nto make your code magically faster on\nPython 3.15!\n\n## What is a lazy import?\n\nImagine you have a file like this, with a standard Python argparse CLI:\n\n``` python\nimport argparse\nimport numpy\n\ndef main():\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"--foo\", action=\"store_true\")\n    args = parser.parse_args()\n    if args.foo:\n        print(numpy.array([1, 2, 3]))\n```\n\nWhat happens if you run this with `--help`\n\n? The `numpy`\n\nlibrary will be\nimported, even though it is never used. If you are using modern `uv`\n\ntooling,\nthis can be even worse, since `uv`\n\ndoesn’t pre-compile bytecode unless you ask\nit to; that makes the install faster, but imports are slower the first time.\n\nThe above is just one example; this can also happen when you have this common pattern:\n\n``` python\n# __init__.py\nfrom . import a\nfrom . import b\n\n__all__ = [\"a\", \"b\"]\n```\n\nThe idea behind this is that a user can just use `lib.a.stuff`\n\nwith just\n`import lib`\n\n, rather than `import lib.a`\n\n, but you pay the cost of import even if\nthey never use all the imports. Some libraries, like `rich`\n\n, are careful to\navoid this and ask users to import explicitly, but many older libraries did\nthis.\n\nAnd there are also libraries that can do multiple things (like CLI libraries with subcommands), but you don’t need the dependencies for every subcommand.\n\n## How to use Python 3.15’s lazy imports\n\nTake the first example. In Python 3.15, you can now write:\n\n``` python\nlazy import argparse\nlazy import numpy\n\ndef main():\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"--foo\", action=\"store_true\")\n    args = parser.parse_args()\n    if args.foo:\n        print(numpy.array([1, 2, 3]))\n```\n\nNow, both imports are “lazy”, meaning nothing happens at all when you import\nthem. They might not even be installed. The first time you try to use the\nobject, though, it becomes a real, imported object. So if you do `--help`\n\n,\n`numpy`\n\nis never accessed and never imported.\n\nThere is also a backward-compatible syntax:\n\n``` python\n__lazy_modules__ = [\"argparse\", \"numpy\"]\n\nimport argparse\nimport numpy\n```\n\nThis works on older Pythons (it’s just not lazy), and you can also dynamically generate or manipulate that list if you want. Linters like Ruff have already updated to allow this to be placed above your imports without triggering a lint violation.\n\nI should mention there’s a flag and a variable to make Python treat all imports\nas lazy, `-X lazy_imports=all`\n\nand `PYTHON_LAZY_IMPORTS=all`\n\n(also `normal`\n\nand\n`none`\n\n). That’s mostly for testing.\n\n`none`\n\ndoesn’t disable\n`__lazy_modules__`\n\n; it only disables the syntax version!\n[Will be fixed](https://github.com/python/cpython/pull/146371).\n\n`none`\n\nshould *only*disable the back-compat\n\n`__lazy_modules__`\n\nvariation; `lazy import`\n\nshould be a guaranteed lazy import.\nThe existence of this option as it stands is currently blocking use in the\nstandard library.## Why not lazy?\n\nShouldn’t you just mark everything as lazy? You don’t have to. There are some modules that have side effects when you import them; if those side effects need to happen at the import site, then those can’t be lazy. This pattern, for example, can’t be lazy:\n\n``` python\ntry:\n    import numpy\nexcept ModuleNotFoundError:\n    ...\n```\n\nThe error here will move to the first usage of something from `numpy`\n\n. There is\na semi-lazy alternative:\n\n``` python\nimport importlib.util\n\nif importlib.util.find_spec(\"numpy\") is None:\n  ... # whatever you wanted to do if numpy is missing\n\nlazy import numpy\n```\n\nThis is slightly more expensive than doing nothing at all (which is why lazy\nimporting doesn’t do it), will import packages to get to subpackages (`a.b`\n\nimports `a`\n\n), and some types of import errors won’t trigger when just finding\nthe spec (for the above example, `numpy._core`\n\ncould be missing/broken if\nsomeone didn’t compile numpy correctly - this is rare, though). Regardless, this\nis a pretty good way to check to see if a package is installed.\n\nThe other case you don’t need lazy is if you use something at top level. For example:\n\n``` python\nlazy import re\n\nREGEX = re.compile(...) # not lazy here\n```\n\nHere, the lazy import is not needed, since you can’t process the file without importing this anyway. You can work around this by caching:\n\n``` php\nimport functools\nlazy import re\n\n@functools.cache\ndef regex() -> re.Pattern:\n  return re.compile(...)\npython\nfrom __future__ import annotations\n\n__lazy_modules__ = [\"re\"]\n\nimport functools\nimport re\n\n@functools.cache\ndef regex() -> re.Pattern:\n    return re.compile(...)\n```\n\nNotice I don’t need `from __future__ import annotations`\n\nto make this work; the\nannotation doesn’t cause the `re`\n\nmodule to be loaded because in Python 3.14\nannotations became lazy by default in that version.\n\nYou *can* make these sorts of imports lazy, but you are just moving the import\nerrors for no good reason, so it’s a bit better not to.\n\nIf you want to make everything in a file lazy, you can do it like this:\n\n``` php\nclass AllLazy:\n    @staticmethod\n    def __contains__(_: str) -> bool:\n        return True\n\n__lazy_modules__ = AllLazy()\n```\n\nThis simply is used by testing with `in`\n\non full module names, and you can put\nyour own object in here. (The static tool below doesn’t look for this yet.)\n\n## A tool to help\n\nSo libraries ideally should start adding these `__lazy_modules__`\n\n, but it’s a\nlittle more complex than just putting all modules into it. So I wrote a tool,\n[flake8-lazy](https://github.com/henryiii/flake8-lazy), to help with figuring out exactly what to add, and with keeping\nit tidy. This is the first library I’ve used AI tools heavily in developing\n(I’ve started using them to help maintain plumbum, but that’s not from scratch),\nso I’ll end with a section about how that went (very well). I’ve developed\n[flake8-errmsg](https://github.com/henryiii/flake8-errmsg) in the past, so it’s not my first flake8 plugin. Like that\nproject, there’s also a built-in standalone runner; early in the 3.15 lifecycle,\nI rather expect that to be the main way to use it.\n\nTo use it:\n\n```\n# Show flake8-style errors\nuvx flake8-lazy <filenames>\n# Show the lines you need to add\nuvx flake8-lazy --format=lazy-modules\n# Just add it!\nuvx flake8-lazy --apply=list <filenames>\n# Show flake8-style errors\npipx run flake8-lazy <filenames>\n# Show the lines you need to add\npipx run flake8-lazy --format=lazy-modules\n# Just add it!\npipx run flake8-lazy --apply=list <filenames>\n```\n\nThis will report the errors (`noqa`\n\ndoesn’t work with the simple runner).\n\nHere are the errors currently implemented (0.6.0):\n\n| Code | 1xx: Missing lazy declarations |\n|---|---|\n`LZY101` | stdlib module should be listed in `__lazy_modules__` |\n`LZY102` | third-party or local module should be listed in `__lazy_modules__` |\n\nThese try to find things that are not used at top level, and suggest they be\nadded to your `__lazy_modules__`\n\n(the `lazy`\n\nsyntax works too). Currently, they\nassume annotations do not trigger an import (since flake8, unlike Ruff, doesn’t\nknow the minimum Python version you are targeting, it can’t tell if it’s 3.14+\nor not).\n\n| Code | 2xx: `__lazy_modules__` validation |\n|---|---|\n`LZY201` | `__lazy_modules__` is not sorted |\n`LZY202` | module listed in `__lazy_modules__` is never imported |\n`LZY203` | module listed in `__lazy_modules__` is duplicated |\n`LZY204` | `__lazy_modules__` is assigned after importing modules it names |\n`LZY205` | module listed in `__lazy_modules__` must be an absolute name |\n\nThese look for general problems specifically with `__lazy_modules__`\n\n.\n\n| Code | 3xx: Native `lazy` keyword (Python 3.15+) |\n|---|---|\n`LZY301` | lazy import inside `suppress(ImportError)` is misleading |\n`LZY302` | module declared lazy by both `lazy` keyword and `__lazy_modules__` |\n`LZY303` | module imported both eagerly and lazily |\n\nThese look for issues specific to Python 3.15+’s new syntax. These only work on\n3.15+ as the host Python, as well. You can tell uv to use it already with\n`--python=3.15`\n\n.\n\n| Code | 4xx: Lazy import safety and semantics |\n|---|---|\n`LZY401` | module is declared lazy but accessed at the top level |\n`LZY402` | module is an enclosing package for this file and should not be lazy |\n\n`LZY401`\n\nis the opposite of the `LZY101`\n\n/`LZY102`\n\nchecks, basically; if you\naccess something at top level, you might as well not make it lazy. This might\nget moved to a 9xx check, as it’s not problematic to do it, and the check system\ncould be wrong.\n\n## Tips\n\nDon’t apply this to test suites.\n\nLook for opportunities to make things lazy if they are not listed here. The `re`\n\nexample above is an example of this. But also check the *actual* imported\nlibraries, too - one library may import another anyway (quite a few libraries\nimport `re`\n\n, including `typing`\n\n, making that one really hard to avoid! `re`\n\nis\npretty slow, too, sadly). You can do this with `-X importtime`\n\n. Anything that is\nlazy and never gets imported will not show up here anymore. You can force lazy\nimports off to see the difference. You can also force lazy imports on to see how\nmuch time you might save before starting.\n\nType checkers always treat `TYPE_CHECKING`\n\nas `True`\n\n, so you can avoid importing\ntyping with this trick:\n\n```\nTYPE_CHECKING = False\nif TYPE_CHECKING:\n    ...\n```\n\nWith Ruff, you can even enforce this with the `TID251`\n\ncheck:\n\n```\n[tool.ruff.lint.flake8-tidy-imports.banned-api]\n\"typing.TYPE_CHECKING\".msg = \"Use TYPE_CHECKING=False instead\"\n```\n\nThe `__lazy_modules__`\n\nsystem is completely dynamic (just needs a `__contains__`\n\nmethod for absolute module names); the checks don’t handle anything dynamic\nhere. The most common use case, relative imports, can be left static:\n\n``` python\n__lazy_modules__ = [f\"{__spec__.parent}.thing\"]\nfrom . import thing\n```\n\nNote that `__package__`\n\nis the older form of `__spec__.parent`\n\n. Don’t use this\non `__main__.py`\n\n, use absolute imports on that one (mypy will notice that\n`__spec__`\n\ncan be `None`\n\non this file).\n\n## Results\n\nI tried running this tool on its own source code, and managed to get the\n`--help`\n\nflag 2x faster on Python 3.15. On cibuildwheel,\n[this managed](https://github.com/pypa/cibuildwheel/pull/2797) a 3-4x speedup\nfor things like `--help`\n\nand `--print-build-identifiers`\n\n. Hugo, the release\nmanager for Python 3.14, was able to get a bit more speed\n[in a PR to my PR](https://github.com/henryiii/cibuildwheel/pull/17). Here are\nsome of the results I’ve gotten so far; for each case, I’m checking `--help`\n\n,\nthough other things can get faster too. Due to the bug mentioned in 3.15a7, the\nbefore timing sometimes uses 3.14.\n\n| package | Before | After | Speedup | Notes |\n|---|---|---|---|---|\n| flake8-lazy | 100+ ms | 50 ms | 2x | Original speedup (current version is a little faster than original) |\n| repo-review | 113 ms | 35 ms | 3x |\n|\n\n[PR](https://github.com/pypa/cibuildwheel/pull/2797)[PR](https://github.com/pypa/packaging/pull/1129), no`--help`\n\nto test[PR](https://github.com/henryiii/check-sdist/pull/152)100+ ms is noticeable, getting under that makes your app feel snappier. Python itself takes about 15 ms (on my M1), so you can’t get faster than that (and you likely at least need a few things, like argparse).\n\nYou can see the impact of each library, and your success/failure to reduce\nimports, with `-X importtime`\n\n.\n\nIf you want to time this yourself, use hyperfine:\n\n```\nhyperfine --warmup 10 \\\n     -n \"main\" --prepare \"git checkout main\"        \"python3.15 -m <pkg> --help\" \\\n     -n \"PR\"   --prepare \"git checkout some-branch\" \"python3.15 -m <pkg> --help\"\n```\n\nYou can do just one run, and pass `-X lazy_modules=none`\n\nor `all`\n\nas well.\n\nKeep in mind, `uv`\n\nand some other tools don’t compile bytecode by default, which\nmeans you might be saving a lot more for a first-run cost than the measurements\nabove. Some of the above results could get better if third-party libraries or\nthe standard library add lazy imports.\n\nThere’s still a ways to go - there are lots of edge cases in trying to detect if\nsomething is being resolved. For example, dataclasses resolve type hints to see\nif `typing.ClassVar`\n\nis used, which breaks laziness. It’s better to put too much\ninto lazy than too little.\n\nThere’s also a big problem with this syntax:\n\n``` python\nfrom a import b\n```\n\nIs `a.b`\n\na module or not? Only a type checker knows (if it’s typed). This is the\nsame thing again:\n\n``` python\nfrom . import b\n```\n\nI had to assume the right hand side is not a module, but if it is, it will be\nmissed. You can use `as`\n\nto avoid this ambiguous syntax.\n\nAlso, maybe it’s obvious, but most of the big, slow imports like numpy aren’t being built for CPython 3.15 yet (around the first RC is when compiled wheels can be published), so some of the most exciting improvements in time can’t be tested yet.\n\n## Developing the tool with AI\n\nThis was a really interesting project to try AI on, partially because this has\nnever been done before. Lazy imports were added quite recently, were just\nreleased about a week ago for the first time in an alpha build of CPython, and\nhave only been easily available in uv for three or so *days*. The AI can’t be\njust grabbing some existing code because it doesn’t exist (I know that’s not how\nmodel training and validation works). It has to take my input, run tests, and\nread the PEP, and “reason” from that. And it *does*. I used it on over 40 tasks,\nand it never failed to understand what I asked it to do. It didn’t “outsmart” me\nand do something smarter than I would have done, but it followed directions\nperfectly. Not only did I not hand write more than about 5% of the code (mostly\ntweaks and configuration), but I haven’t followed through all the implementation\ndetails. It took less than a day for the initial version (I was doing other\nthings too while the AI worked), and getting it into a usable form (by using it\non libraries) happened over the next couple of days (again, off and on). This is\nprobably 5-7x faster than I could have done it by hand. Code is a bit longer\nthan a hand written solution, mostly due to duplication (I could iterate to make\nsure it was clean/readable, it still looks mostly like my code). Check\n[repo-review](https://github.com/scientific-python/repo-review) to see what my hand designed code looks like.\n\nI started with my\n[Scientific Python Development Guide](https://learn.scientific-python.org/development/)’s\ntemplate, which has strong linting, formatting, and testing setup already,\nperfect for AI usage. I tried a few options I haven’t used before, like\n`uv_build`\n\nfor the backend, and the new `Zensical`\n\ndocumentation engine. I also\nended up finding a few things that could be improved, and put them back into the\ntemplate. I also increased the linting checks to `ALL`\n\nthen used\n`uvx --from sp-repo-review[cli] sp-ruff-checks .`\n\nto get a list of checks that\nare always good to ignore. I think a *lot* of the success of the AI came down to\njust how good this setup is.\n\nFor AI, I’m using [GitHub Copilot](https://github.com/features/copilot), with\nauto model selection, primarily in VSCode (though I also later used the GitHub\nagent feature too to develop features in parallel). The model seemed to mostly\nbe GPT-5.3-codex, though Claude Sonnet 4.6 was auto-selected sometimes too.\n\nI didn’t add configuration at first, but once I filled up my first context\nwindow and wanted to start a new chat, I added a handwritten `AGENTS.md`\n\nand a\ncopilot CI configuration (`.github/workflows/copilot-setup-steps.yml`\n\n). The\nfocus of these was to make `uv`\n\n, `prek`\n\n, and `nox`\n\navailable and instruct the\ntools to use them. This reduced my need to manually run these or tell the agent\nabout them.\n\nI ended up doing very little manual coding - most of my manual edits were setup or configuration. If I didn’t like the model output, I just would ask for it to make changes. I was quite explicit though in instructions; I’ve written a plugin for flake8 with a manual runner before, so I knew what I wanted. And I iterated a lot. For example, when adding better error messages for broken files, the model thought about adding Python 3.11 exception notes, but then did it a different way, due to Python 3.10 being the minimum. I asked it to instead do the notes, but gate it for 3.11.\n\nThe docs were initially written by the model too, though there I did quite a bit of editing as well. I don’t love the repetition between the docs and README, but the agent is pretty good at keeping them in check, even if I edit one, it can fix the other to match. I don’t see a way to include one in the other with Zensical yet.\n\nI even did things like ask to rebase and solve the merge conflicts. It didn’t\nfail at anything, really. The worst it did was not always run the style checks,\nmeaning I had to do one more thing once the CI caught the failing checks. But\neven that was pretty rare. I asked it to refactor the really long `__init__.py`\n\nfile eventually, and it did that perfectly too; the only thing it didn’t do was\nre-apply `__lazy_modules__`\n\n.\n\nI really couldn’t be much happier with the *results*. The agent was great at\nwriting tests for everything it added, even without prompting. It would work\nthrough errors and warnings - I didn’t have to do any of that, which was\nfantastic. I ran on the CPython source code, and found an issue (the encoding\nwasn’t handled), so I just told the agent how to run it, and it found the issue\nand applied the correct fix (use the tokenizer rather than manually opening the\nfile, which I would have taken much longer to find). I asked it to handle\nrelative imports correctly (`.`\n\n), and it generalized for `..`\n\n, etc.\n\nI also asked it to read [PEP 810](https://peps.python.org/pep-0810), and look for possible checks based on the\ntext, which it did a great job with, and some of the checks are actually from\nthose suggestions. I also asked it to come up with a better numbering scheme,\nwhich it also did.\n\nRefactors were amazing. I could just make big changes, like reorganizing the numbering scheme, and then it would just tinker for a while and then it was done. I tried making this multithreaded on free-threaded Python (the CLI runner, that is), but got pretty poor results; something is creating a single-threaded bottleneck, and wasn’t able to find it quickly. But being able to try big things like that very easily was great. And this still wasn’t a “mistake”, it did exactly what I wanted.\n\nIteration on stuff that takes times was a strong point. If you have a tool that outputs something, rather than fixing it itself, the agent is really good at applying a fix (including complex things like typing) and rerunning. This continues to make linting tools even more valuable.\n\nThe code *quality* is not terrible, but hand written would be better, I think.\nI’ve generally seen that - in the past, I’ve used AI for a quick first draft to\nsee if something is performant, etc, but will do a hand written implementation\nfor the actual PR. AI also can do cleanup if you ask it to, and telling it the\nminimum Python version, that you value modern readable code, etc all helps. But\nthat begs the question; if the tests and linting are strong, and if you use AI\nto edit it in the future, does human readability matter as much now? Also, does\nthat lock you into using AI tools? (I made sure the quality wasn’t that bad, but\ninteresting philosophical questions nonetheless.)\n\nIf you’d like to see what the work looked like, you can see the commit history\nand the GitHub Agent PRs. Overall, it’s quite incredible, comparing my attempts\nat AI early last year (just slop), last advent of code in TypeScript (great for\nlearning a language, and actually pretty good at refactoring and helping), and\nnow just *3 months* later, where it’s really, really good. I’ve tried to get it\nto do pattern matching before; it was terrible, and now it actually gets it\ncorrect (still has to be asked, though). It doesn’t always write ideal patterns,\nbut that’s easy to clean up if it’s correct. It’s still a tool that does what\nit’s told, but it’s gotten *good* at doing what it’s told. Combined with proper\nlinting and testing setups (critical!), it’s a *very* good helper.\n\nThe skill set required to work with it, I believe, is the same. I am still doing the sort of high level things I’d do when designing a library. When I added generic typing to boost-histogram, I did one by hand, then told AI to do the rest following my example. I’m making the decisions, it’s just now a lot faster (as in, less of my time, I am doing other things while it’s working) to see the result of those decisions.\n\nBy the way, the 0x token models (I tried GPT 5 mini) work fine at taking the\noutput of `flake8-lazy --format=lazy-modules`\n\nand applying them to a non-trivial\ncodebase automatically. It’s a bit slow, but it works, I used that on\n`cibuildwheel`\n\ninitially. So I added a `--apply`\n\nfeature to the CLI to inject\nthe lines in 0.4.0. Now (in 0.6.0) it supports several formats; `list`\n\n, `set`\n\n,\n`native`\n\n, and `dynamic`\n\n.\n\n[Python](/categories/python/)", "url": "https://wpnews.pro/news/working-to-make-python-lazy", "canonical_source": "https://iscinumpy.dev/post/flake8-lazy/", "published_at": "2026-09-01 18:44:28+00:00", "updated_at": "2026-09-01 18:52:22.157010+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools"], "entities": ["Python", "PEP 810", "flake8-lazy", "uv", "Ruff"], "alternates": {"html": "https://wpnews.pro/news/working-to-make-python-lazy", "markdown": "https://wpnews.pro/news/working-to-make-python-lazy.md", "text": "https://wpnews.pro/news/working-to-make-python-lazy.txt", "jsonld": "https://wpnews.pro/news/working-to-make-python-lazy.jsonld"}}