Stop Timing the Happy Path

An engineer from MonkeyCode warns that timing only the happy path in cache lookups hides the real cost of exception-based miss handling, which can be over 16 times slower and use more memory than a quiet dict.get. The developer provides a benchmark harness that mixes 20% misses and urges teams to measure both time and peak memory, noting that AI-generated code often defaults to try/except patterns that look clean but are inefficient.

The happy path was never the bottleneck. I was timing successes and shipping a miss. Production traffic is full of misses. Would you trust a bench that never fails? An AI rewrite loves the clean try. It wraps a lookup in except KeyError. It logs the miss "for observability." It looks professional. It is also a tiny furnace. Exceptions are not cheap branches. Log formatters are not free either. I learned that the loud way. Cheap generation makes the trap faster. A model will emit a polite miss path before you blink. Technical debt used to wait for a human. Now it arrives as a helpful patch tonight. The debt is not the lookup. The debt is a story about speed with no miss mix in the graph. I needed variants, not vibes. I used MonkeyCode's free model access and free server option to draft those variants. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The model proposes shapes. It does not know your miss rate. If the graph disagrees, the patch dies. This is a pocket harness. It is not a production claim. Steal the file. Change the mix. Keep your own picture. I am not posting a trophy chart from a machine you cannot see. miss bench.py Lab harness. Treat printed rows as local output, not a benchmark paper. from future import annotations import logging import time import tracemalloc from typing import Callable logging.basicConfig level=logging.DEBUG log = logging.getLogger "hot" HITS = {f"user:{i}": i for i in range 800 } KEYS = f"user:{i}" for i in range 1000 20% misses on purpose def lookup except key: str - int | None: try: return HITS key except KeyError: log.debug "cache miss key=%s", key return None def lookup get quiet key: str - int | None: return HITS.get key def lookup get log key: str - int | None: value = HITS.get key if value is None: log.debug "cache miss key=%s", key return value def run mix fn: Callable str , int | None , rounds: int = 30 - tuple float, int : Warm once so import noise does not sit in the graph. for key in KEYS: fn key tracemalloc.start t0 = time.perf counter for in range rounds : for key in KEYS: fn key elapsed ms = time.perf counter - t0 1000 current, peak = tracemalloc.get traced memory tracemalloc.stop return elapsed ms, peak def main - None: variants = "except+debug", lookup except , "get+quiet", lookup get quiet , "get+debug", lookup get log , print "name,ms,peak kib" for name, fn in variants: ms, peak = run mix fn print f"{name},{ms:.1f},{peak / 1024:.1f}" if name == " main ": main Run it like a skeptic. Do not narrate the first number. Narrate the pair. python miss bench.py LOGLEVEL=WARNING python miss bench.py I keep two columns. Milliseconds next to peak KiB. A time without allocations is a rumor. A time without a miss mix is a bedtime story. Does your staging traffic really hit 100 percent? Be honest. I paste the naive dict lookup and a contract. Same return type. No new dependencies. Three numbered functions. Then I stop chatting. Chat is not a profile. Here is a hot lookup over a dict of user ids. Keep the same return type: int | None. Give me three drop-in functions: 1 try/except KeyError on miss 2 dict.get with no logging 3 dict.get with a debug log on miss Do not change the hit table. Do not add threads. After the functions, list what you did not measure. The last line matters. Models forget the unmeasured part. I make them say it. They still cannot plot it. That is my job. After one local run I paste a table into the PR comment. Your numbers will move. The shape is the lesson. This block is a template, not a trophy. illustrative local shape, miss mix = 20%, debug logging on name,ms,peak kib except+debug,184.0,420.2 get+quiet,11.3,38.4 get+debug,96.7,401.8 Look at that middle row. Quiet get is boring. Boring won. The except variant looks "correct" in a code review. It also throws on every miss. Python exception machinery is a convoy of trucks. Logging is another convoy. I almost merged the polite version. The table said no. Then I flipped the log level. Same functions. Same keys. The except line dropped, but not to quiet. The formatter was gone. The throw was not. That second picture is the one I keep. Algorithm talk without a miss mix is costume jewelry. Why does the model keep doing this? It has read a million style guides. "Ask forgiveness." Cute in a parser you run twice. Ugly in a loop you run with production misses. The model has never paid your allocator. It has never sat in your p99. You have. Act like it. I copy variants by hand. I do not pipe them straight into main. Then I time the mix I actually fear. python -m pip install py-spy optional, local profiler python miss bench.py /tmp/mix20.csv Same bench, fewer fireworks. python -c "import logging; logging.disable logging.CRITICAL ; import miss bench as m; m.main " py-spy record -o /tmp/miss.svg -- python miss bench.py py-spy top -- python miss bench.py The SVG is another graph I keep. I do not keep the model's paragraph about Big-O. Big-O never met logging. Big-O never met a 20 percent miss. Pictures of time still argue when the chat is gone. If the process is longer than this toy, I sample under load. A microbench can flatter CPU tricks. A live request log cannot. When the two pictures disagree, the live one wins. Always. I use the free models as a variant factory. I use the free server as a scratchpad for those three shapes. I still run the harness on my laptop. A remote prompt cannot see tracemalloc. It cannot see your log level. It cannot see the miss mix from last Tuesday's traffic. Treating it as a profiler is how you get a confident slowdown. The useful bit is speed to a killable idea. Three functions in one paste. One CSV. One no. That loop is the whole method. Free access just shortens the wait before the no. It does not replace the no. I also ask the model a mean follow-up. "What workload makes variant 1 win?" Sometimes it admits none. Sometimes it invents a tiny table. I do not take that bait. I change KEYS in the file and rerun. Reality is a Python file you can execute. Reality is not a paragraph. This harness is a microbench. Microbenches lie when the real cost is I/O. If the lookup sits behind a network, stop here. Go trace the call. If the function runs once at boot, do not dress it as a hot path. If the change is correctness, do not sell it as speed. A safer miss path can be worth a slower graph. Say that out loud. Do not hide it under milliseconds. Do not paste secrets into any prompt. Do not paste customer keys. Do not paste proprietary protocol guts to get a nicer except block. If you cannot run the file locally, you cannot keep a graph. Skip the model. Fix the test fixture first. Who should not use this. If you do not own the runtime, stop. If you cannot change log levels in staging, stop. If you need distributed traces, this laptop lab will not save you. If your language has cheap exceptions, do not cargo-cult my Python table. Measure your own miss path. This pocket knife is for dict-shaped lookups with a known mix. It is not a platform. One more refusal. Do not average away the miss. A 1 percent miss can still dominate if the miss does disk. My 20 percent mix is a teaching knob. It is not your traffic. Copy the knob. Do not copy the percentage as folklore. I keep the command. I keep the CSV. I keep the miss mix. I keep the log level. I throw away the model's confidence. The picture that argues with me is the review. Everything else is narration. If you want a scratchpad for those three variants, MonkeyCode's free models and free server are enough for this lab. Then close the chat. Open the CSV. Keep the row that makes you uncomfortable.