{"slug": "zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice", "title": "Zero-Copy Go: Why io.Copy Is Lying to You (sendfile/splice)", "summary": "A developer discovered that an inference gateway's p99 latency spike was caused by unnecessary data copying in a logging middleware wrapper, not the AI model itself. Every response body was being copied from kernel space to userspace and back via io.Copy, turning the gateway into a slow photocopier. The author warns that this hidden overhead costs real throughput and money for high-traffic services like AI inference, file downloads, and video streaming.", "body_md": "Member-only story\n\n# Zero-Copy Go: Why `io.Copy`\n\nIs Lying to You (sendfile/splice)\n\n*Your HTTP middleware is probably copying every byte your AI gateway serves. Here’s the throughput you’re leaving on the table — measured, not guessed.*\n\nI run an inference gateway in front of our model servers. Last month we had a p99 latency spike during a traffic burst — requests that normally returned fast were stalling, and the CPU on the gateway boxes were pegged while the GPUs sat idle.\n\nMy first instinct, like an idiot, was to blame the model. “The 70B is slow under load,” I told my team. We spent two days tuning batch sizes and KV-cache before I finally looked at the gateway itself.\n\nThe model wasn’t the problem. **The bytes were.**\n\nEvery response body — multi-megabyte tensors, streamed tokens, the works — was being copied from kernel space into a userspace buffer and back out again, because of a logging middleware wrapper I’d written three months earlier and never thought about again. The gateway was a very expensive, very slow photocopier.\n\nThis is the `io.Copy`\n\nlie. And if you serve any kind of high-throughput traffic — AI inference, file downloads, video — it's costing you real money.", "url": "https://wpnews.pro/news/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice", "canonical_source": "https://blog.devgenius.io/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice-11748e293462?source=rss----4e2c1156667e---4", "published_at": "2026-07-21 20:51:01+00:00", "updated_at": "2026-07-21 21:27:15.310986+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice", "markdown": "https://wpnews.pro/news/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice.md", "text": "https://wpnews.pro/news/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice.txt", "jsonld": "https://wpnews.pro/news/zero-copy-go-why-io-copy-is-lying-to-you-sendfile-splice.jsonld"}}