{"slug": "tensward-finds-643-mb-copied-off-an-h100-for-each-request", "title": "Tensward finds 643 MB copied off an H100 for each request", "summary": "Tensward, founded by Adi Shik, traced 643 MB of data copied off an NVIDIA H100 per request in a November 15 test to llama.cpp state saving, after running an LLM server with sixteen clients that showed 35% GPU utilization, 58 tokens per second throughput, and nearly two seconds to first token. The finding attributes the overhead to how llama.cpp saves state rather than to raw compute limits.", "body_md": "# Tensward finds 643 MB copied off an H100 for each request\n\n**Tensward, founded by Adi Shik, traced the transfers in one H100 test to llama.cpp state saving.**\n\n        By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)\n        · Published \n\nAdi Shik started Tensward after a client deployment turned into a lesson in inference optimisation. He has described trying to deploy a local model in a client's environment in a previous job, calling inference optimisation \"a huge headache\" and saying the experience gave him the idea for Tensward.\n\nIn Tensward's November 15th post, Shik's team put an LLM server on an NVIDIA H100, pointed sixteen clients at it, and watched 'nvidia-smi' report 35% busy. Throughput was 58 tokens per second, and users waited almost two seconds for a first token.…", "url": "https://wpnews.pro/news/tensward-finds-643-mb-copied-off-an-h100-for-each-request", "canonical_source": "https://runtimewire.com/article/tensward-gpu-census-643-mb-h100", "published_at": "2026-11-15 18:42:39+00:00", "updated_at": "2026-11-15 19:21:36.262837+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools"], "entities": ["Tensward", "Adi Shik", "NVIDIA H100", "llama.cpp", "nvidia-smi"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tensward-finds-643-mb-copied-off-an-h100-for-each-request", "markdown": "https://wpnews.pro/news/tensward-finds-643-mb-copied-off-an-h100-for-each-request.md", "text": "https://wpnews.pro/news/tensward-finds-643-mb-copied-off-an-h100-for-each-request.txt", "jsonld": "https://wpnews.pro/news/tensward-finds-643-mb-copied-off-an-h100-for-each-request.jsonld"}}