# Same A100, similar throughput: would both pass a 1-second first-token target? > Source: > Published: 2026-11-13 09:09:05+00:00 [Exulux](https://discuss.huggingface.co/u/Exulux) 1 We tested Llama 3.1 8B Instruct on two A100 SXM 80 GB rentals: RunPod Secure in Kansas and Massed Compute via Shadeform in Des Moines. Both ran BF16 at concurrency 32, with 2,000 input tokens and 500 requested output tokens. Throughput was about 1,135–1,139 output tokens/s across the six measured runs. First-token p95 was 1,251–1,261 ms on the Kansas allocation and 752–990 ms on the Des Moines allocation. At a 1,000 ms target, that difference matters: one allocation passed all three runs, and the other missed all three. The slowest passing run had only about 10 ms of headroom. These were same-instance measurements, so application and network latency still need to be tested. Each result represents one allocation, with different hosts, regions and drivers. The hourly quotes in the case study are historical. [Full results, method and limitations](https://exuluxcompute.com/compare/runpod-vs-shadeform-a100-llama-3-1-8b/) If you’re sizing a deployment now, what first-token p95 limit and peak concurrency does the GPU need to handle? [DHEIVER](https://discuss.huggingface.co/u/DHEIVER) 2 [@Exulux](https://discuss.huggingface.co/u/exulux) The variance in first-token latency (1.25s vs 0.75s) despite identical throughput points to host-level scheduling contention. During the prefill phase, the system is memory-bound and sensitive to OS scheduling noise or hypervisor throttling. Even with GPU compute saturated, delays in memory allocation or context switching directly impact the time to first token, while the decode phase masks this instability until context fills. The issue lies in how the host manages memory pressure during the prefill spike, not the GPU’s raw throughput.