Tensward, founded by Adi Shik, traced the transfers in one H100 test to llama.cpp state saving.
By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)
· Published
Adi Shik started Tensward after a client deployment turned into a lesson in inference optimisation. He has described trying to deploy a local model in a client's environment in a previous job, calling inference optimisation "a huge headache" and saying the experience gave him the idea for Tensward.
In Tensward's November 15th post, Shik's team put an LLM server on an NVIDIA H100, pointed sixteen clients at it, and watched 'nvidia-smi' report 35% busy. Throughput was 58 tokens per second, and users waited almost two seconds for a first token.…
source & further reading
runtimewire.com — original article
SGLang puts weight updates and LoRA loading behind a dedicated admin key
LiteLLM fixes ignored web-search cap that turned a limit of one into three searches and a 500
llama.cpp patch speeds up one RTX 5090 matrix multiply 4.5x