{"slug": "cutting-context-window-pollution-with-silent-test-runners", "title": "Cutting context window pollution with silent test runners", "summary": "A developer published a Makefile macro called RUN_TEST that suppresses passing test output so AI coding agents do not burn context window tokens on every successful assertion, printing a single confirmation line on success and the full captured output only on failure. The macro buffers stdout and stderr to a mktemp file, prints a labeled pass message such as \"✓ Go tests passed\" on exit code 0, and on non-zero exit cats the buffer and re-exits with the original code; the author also recommends adding a rule to agent instruction files like AGENTS.md or GEMINI.md telling agents to prefer Make targets over calling pytest or go test directly. The targets drop -v and use go test -race -p 1 so failure logs do not interleave.", "body_md": "# Cutting context window pollution with silent test runners\n\nMost AI coding agents send terminal output straight to the LLM context window. While human developers enjoy watching green checkmarks scroll past, an agent consumes every passing assertion as input tokens, which degrades performance and increases costs. To keep the context window clean without losing failure details, I wrap my test commands so they print a single confirmation line on success and show the full output only when a test fails.\n\n## Writing the test runner macro\n\nI implemented this with a reusable macro in my project's `Makefile`:\n\n```\n# Macro to run test commands quietly.\n# Suppresses stdout and stderr on success to save LLM context window tokens.\n# On failure, prints the captured output and exits with the original exit code.\ndefine RUN_TEST\n\t@TMPFILE=$$(mktemp); \\\n\tif $(1) > \"$$TMPFILE\" 2>&1; then \\\n\t\techo \"✓ $(2) passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly.\"; \\\n\t\trm -f \"$$TMPFILE\"; \\\n\telse \\\n\t\tEXIT_CODE=$$?; \\\n\t\tcat \"$$TMPFILE\"; \\\n\t\trm -f \"$$TMPFILE\"; \\\n\t\texit $$EXIT_CODE; \\\n\tfi\nendef\n```\n\nHere's how the shell commands in the macro work:\n\n- \n`TMPFILE=$$(mktemp)` creates a temporary file in`/tmp` to\n              buffer the output.\n- \n`$(1) > \"$$TMPFILE\" 2>&1` runs the test command passed as the\n              first argument (`$(1)` ) and redirects both standard output (file descriptor\n              1) and standard error (file descriptor 2) into the temporary file.\n- `if ... then` checks the exit code of the test command.\n- \n              If the command exits with `0` , the shell prints the confirmation line using\n              the suite label passed as`$(2)` and runs`rm -f \"$$TMPFILE\"` (using`-f` to force removal without\n              prompting) to delete the temporary file.\n- \n              If the command exits with a non-zero status, `EXIT_CODE=$$?` saves the exit\n              code,`cat \"$$TMPFILE\"` prints the captured output,`rm -f \"$$TMPFILE\"` removes the temporary file, and`exit $$EXIT_CODE` exits Make with the original failure code.\n\n            The `echo` message is written directly for the agent. If a test target prints\n            nothing at all (or only prints `✓ passed`), coding agents sometimes assume\n            the tests didn't actually run and bypass `make` to run `pytest` or\n            `go test` directly. Telling the agent in the output itself why the logs were\n            suppressed, and reminding it to use Make targets, stops it from working around the\n            wrapper.\n          \n\n            It also helps to add the same rule to your project's agent instructions (such as\n            `GEMINI.md` or `AGENTS.md`):\n          \n\n```\nWhen running tests, prefer Make targets (such as `make test`) over calling test runners directly.\n```\n\n## Using the macro in your Makefile\n\nHere's how I wrap Go and Python test commands with `RUN_TEST`:\n\n```\n.PHONY: test test-unit test-python\n\n# Run all test suites quietly\ntest:\n\t$(call RUN_TEST,go test -race -p 1 ./...,Go tests)\n\t$(call RUN_TEST,uv run pytest,Python tests)\n\n# Unit test suite\ntest-unit:\n\t$(call RUN_TEST,go test -race ./pkg/...,Go unit tests)\n\n# Python test suite\ntest-python:\n\t$(call RUN_TEST,uv run pytest,Python tests)\n```\n\n            The targets omit `-v` so a failure only outputs the failing test diagnostics\n            rather than every passing test in the suite. For Go tests, `-race` enables\n            the race detector, and `-p 1` runs package tests sequentially so failure logs\n            don't interleave.\n          \n\n## Verifying the output\n\nRun the test suite with `make`:\n\n```\nmake test\n```\n\nWhen all tests pass, `make test` prints two confirmation lines:\n\n```\n✓ Go tests passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly.\n✓ Python tests passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly.\n```\n\nAnd if a test fails, you get the full failure output and exit code:\n\n```\n--- FAIL: TestFlightComputer (0.01s)\n    flight_test.go:42: expected altitude 10000, got 9850\nFAIL\nFAIL\texample.com/flight\t0.015s\nmake: *** [test] Error 1\n```\n\nIf you want to try this out in your project, you can paste this URL into your agent directly.", "url": "https://wpnews.pro/news/cutting-context-window-pollution-with-silent-test-runners", "canonical_source": "https://wietsevenema.eu/blog/2026/cutting-context-window-pollution-with-silent-makefiles/", "published_at": "2026-10-22 00:00:00+00:00", "updated_at": "2026-10-22 12:43:13.281320+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models"], "entities": ["Make", "pytest", "go test", "uv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cutting-context-window-pollution-with-silent-test-runners", "markdown": "https://wpnews.pro/news/cutting-context-window-pollution-with-silent-test-runners.md", "text": "https://wpnews.pro/news/cutting-context-window-pollution-with-silent-test-runners.txt", "jsonld": "https://wpnews.pro/news/cutting-context-window-pollution-with-silent-test-runners.jsonld"}}