{"slug": "geography-is-the-latency-floor-you-can-t-move", "title": "Geography is the latency floor you can't move", "summary": "An engineer at lyr.jp measured round-trip times from Tokyo to AWS regions worldwide, finding a 5ms local latency versus 373ms to Cape Town—a 70x spread. The 166ms round trip to the US East Coast consumes 55–92% of the translation budget, making geographic placement the dominant factor. Moving the inference server from the US to Tokyo cut latency by ~160ms, more than any model optimization.", "body_md": "**What measuring round-trip times from Tokyo to the rest of the world taught me — physics is a wall**\n\nMake the model faster, quantize it — **distance is the one thing that never shrinks**. When I measured round-trip times from Tokyo to points around the world, they lined up from 5ms in Tokyo to 373ms in Cape Town: a spread of more than 70x.\n\nThis is a **floor** set by physics rather than performance, and no amount of code moves it. Worse, the 166ms round trip to the US East Coast eats 55–92% of the budget translation is allowed (~180–300ms) — a remote server misses the deadline even with generation at 0ms.\n\nSo the biggest lever wasn't performance, it was **placement**. Just moving the inference server from the US to the Tokyo area erased ~160ms — more than any model optimization I had done.\n\nIn [Part 2](https://lyr.jp/en/articles/02-the-bottleneck-moves/) I wrote that the dominant term in the slowness had moved to \"distance\" (a 275ms round trip). But distance is a different kind of thing from queueing or generation. **No amount of polishing the code shrinks it.** Packets have to physically travel.\n\nSo how big is that floor, exactly? I measured it.\n\nThe cleanest way to measure this is to take fixed points spread evenly around the globe as your reference. Cloud provider regions (AWS here) are exactly that — scattered across the world's major cities at known locations. Here is the **raw round-trip time (RTT)** from Tokyo to each region.\n\n| Region | Round-trip time from Tokyo (RTT) |\n|---|---|\n| Tokyo | 5 ms |\n| Seoul | 31 ms |\n| Hong Kong | 57 ms |\n| Singapore | 70 ms |\n| Sydney | 104 ms |\n| Mumbai | 121 ms |\n| US East Coast | 166 ms |\n| Frankfurt | 224 ms |\n| London | 233 ms |\n| São Paulo | 273 ms |\n| Cape Town (South Africa) | 373 ms |\n\nInside Tokyo, 5ms. But cross the Pacific to the US East Coast and it's 166ms; Europe is over 220ms; and South Africa (Cape Town), on the far side of the planet, is 373ms — **more than 70x** local Tokyo. None of this has anything to do with server performance. It's the time a packet takes to physically make the round trip through optical fiber.\n\n(The \"275ms of distance\" I quoted in [Part 2](https://lyr.jp/en/articles/02-the-bottleneck-moves/) was the **effective value on the production path**: this raw 166ms of distance, plus the extra round trip through a proxy layer sitting in between. Pull out pure geographic distance on its own and you get 166ms.)\n\nHere's the heart of it. That 166ms is a **floor no optimization can get under**.\n\nAs long as the server is far away, you've already spent 166ms before translation even starts. For the network portion, **where you put it matters more than how much performance you add**.\n\nAnd this floor is **stable**. Keep measuring the Tokyo↔US round trip and it stays around 166ms (in the 170s under continuous measurement) — the transpacific hop itself barely wobbles. The jitter I used to file under \"the network is unstable\" turned out to be the proxy layer in between, not geography. Distance is predictable, and it doesn't move. Which is exactly why optimization can't make it go away.\n\nThink back to the deep dive on [\"the cliff\"](https://lyr.jp/en/articles/07-the-cliff/) — the deadline by which a subtitle still lands in time. Once OCR is done, the **budget available for translation** was roughly 180–300ms.\n\nNow drop the 166ms US East Coast round trip into that. Against a translation budget of 180–300ms, **the round trip alone burns 55–92% of it before a single character has been translated**. Which means a distant server **can't get inside the cliff, no matter how fast the model is**. It's physically impossible.\n\nPut the other way round, there is exactly one way to lower the floor: **put the server closer**. When I actually moved the inference server from the US to the Tokyo area, I cut about 160ms. That was bigger than any model optimization I had done up to that point.\n\nThis is the manifesto's second principle — [\"it can still be cheaper and faster\"](https://lyr.jp/en/articles/00-manifesto/#2-hunt-the-constraint) — in its most physical form. The biggest win wasn't inside the model; it was on the map.\n\n| Item | Measured | Conditions & caveats |\n|---|---|---|\n| Raw RTT from Tokyo | Tokyo 5 / Seoul 31 / Hong Kong 57 / Singapore 70 / Sydney 104 / Mumbai 121 / US East 166 / Frankfurt 224 / London 233 / São Paulo 273 / Cape Town 373 ms | TCP connect time to AWS region endpoints, min of 4 |\n| Transpacific stability | Tokyo↔US stable at ≈ 170ms | the jitter was the shared proxy layer, not geography |\n| Relation to the cliff | against a translation budget of ~180–300ms, the 166ms US East round trip is 55–92% of the budget | a remote server can't reach the cliff even with generation at 0ms |\n| Measured lever | moving from the US to the Tokyo area cut RTT by ~160ms\n|\nbigger than any model optimization |\n\n*Originally published at LYR Performance Note #008 — part of the series “Infrastructure — deadlines for speed, and floors you cannot move”. The full set is at lyr.jp/en/research.*", "url": "https://wpnews.pro/news/geography-is-the-latency-floor-you-can-t-move", "canonical_source": "https://dev.to/lyr/geography-is-the-latency-floor-you-cant-move-301", "published_at": "2026-08-03 14:05:07+00:00", "updated_at": "2026-08-03 14:14:12.006495+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-products", "developer-tools"], "entities": ["AWS", "lyr.jp", "Tokyo", "Cape Town"], "alternates": {"html": "https://wpnews.pro/news/geography-is-the-latency-floor-you-can-t-move", "markdown": "https://wpnews.pro/news/geography-is-the-latency-floor-you-can-t-move.md", "text": "https://wpnews.pro/news/geography-is-the-latency-floor-you-can-t-move.txt", "jsonld": "https://wpnews.pro/news/geography-is-the-latency-floor-you-can-t-move.jsonld"}}