← All posts · 9 min read · Article
LithosAI has consistently ranked first since the public API launch in September for output speed, latency, and end-to-end response time across all four open models in Artificial Analysis: GLM 5.3 Flash, Kimi K3, DeepSeek V4.1-Flash, and GLM 5.3. That is four models and twelve first-place results across speed, latency, and end-to-end response time.
In real deployments, all three measurements matter. Output speed, in tokens per second, is how quickly a response is generated once it starts. Latency, measured as time to first token (TTFT), is how long the caller waits before the first token arrives. End-to-end response time is the time to a complete answer, including input processing and reasoning. For agents, what counts is the time to complete a task, which is a chain of these responses, and that speed only matters at full model quality.
Our goal is to fit more intelligence per second into the agent loop. That matters for latency-sensitive work that requires repeated reasoning and tool use, including agentic coding, SRE, cybersecurity, personalized recommendation agents, and voice AI.
Output speed, time to first token, and response time #
Artificial Analysis measures three things about every endpoint (see its methodology). In simple terms:
- Output speed (tokens per second). How fast tokens arrive once the model has started generating. Higher is better.
- Time to first token (TTFT). How long you wait after sending a request before the first token arrives. It covers processing the prompt. Lower is better.
- End-to-end response time. How long it takes to receive a complete answer, including any reasoning the model does first. Lower is better.
The three are related:
Two providers with the same output speed can still differ widely in end-to-end response time if one takes longer to start.
Higher inference speed isn’t worth it without full model quality. That’s why we also report endpoint accuracy benchmarks.
LithosAI pushes the Pareto frontier for open weight model inference #
Leaderboards are a useful signal, but each one is a snapshot of a single test setup. What ultimately matters is how an engine performs on real production workloads: the prompts, context lengths, concurrency, and tools your agents actually use. We treat benchmarks as a starting point and measure ourselves by the speed, latency, and quality our customers see in production.
Quality is paramount. An agent that answers faster but less accurately does not finish its work sooner: it fails more often, retries more, and costs more in the end. So the Lithos Engine relies only on systems and ML techniques that leave model quality untouched, delivering the best speed at full model quality. On Artificial Analysis, LithosAI models always match the reference endpoint.
We bring cost down through systems work rather than shortcuts. From the start, we have built the Lithos Engine to reduce per-agent, per-loop latency and cost on standard GPUs. Our infrastructure runs Base, Fast, and Ultra workloads on the same GPU nodes, which keeps GPUs highly utilized and lets us offer distinct speed and price points without dedicating hardware to each.
Full results
| LithosAI results by model | |||||
|---|---|---|---|---|---|
| Model | Size | E2E time | Output speed | Price* | Accuracy† |
|---|---|---|---|---|---|
| [GLM 5.3 Flash](#benchmark-results) | 321B | 3.6 s | 1,543 tok/s | $0.21 | Not reported by AA |
| [Kimi K3](#benchmark-results) | 2.8T | 5.7 s | 499 tok/s | $4.30 | 100% |
| [DeepSeek V4.1-Flash](#benchmark-results) | 552B | 2.2 s | 1,453 tok/s | $0.21 | Not reported by AA |
| [GLM 5.3](#benchmark-results) | 753B | 4.1 s | 717 tok/s | $1.30 | 98% |
* Blended price: USD per million tokens at the benchmark’s blended token mix. See [current pricing](https://www.lithosai.com/pricing) for input, cached-input, and output rates.
† Artificial Analysis Endpoint Accuracy Index, relative to a reference endpoint. “Not reported by AA” means Artificial Analysis does not yet support accuracy evaluation for these models.
First place on Artificial Analysis for speed, latency, and end-to-end response time with full model quality #
GLM 5.3 Flash inference on LithosAI
For long-context agents working with 100K-token prompts, LithosAI generates 1,543 tokens per second and returns a complete answer in 3.6 seconds. Other providers top out around 400 tokens per second and quickly fall under 200. LithosAI is 3.4× faster than the best alternative and more than 7× faster than most providers.
The gap carries through to the complete answer. The best alternative takes 10.8 seconds, three times longer than LithosAI, and most providers take between 14 and 43 seconds.
Kimi K3 inference on LithosAI
LithosAI records 499 tokens per second and an end-to-end response time of 5.7 seconds. The best providers report 384 tokens per second and 8.4 seconds, respectively. This corresponds to approximately 29.9% higher output speed and 32.1% lower response time.
Beyond the runner-up, the gap widens. Every other provider delivers 134 tokens per second or less and takes nearly 20 seconds or more to return a complete answer. LithosAI is more than 3.5× faster than the rest of the field and returns answers in under a third of the time.
LithosAI’s Endpoint Accuracy Index for Kimi K3 is 100% of the reference and is classified as within the reference range. This measures agreement with the reference’s benchmark score; it does not imply that every response is correct.
DeepSeek V4.1-Flash inference on LithosAI
LithosAI records 1,453 tokens per second and an end-to-end response time of 2.2 seconds. Against the best providers’ 1,134 tokens per second and 3.4-second response time, these values represent approximately 28.1% higher output speed and 35.3% lower response time.
Beyond the top two alternatives, most providers deliver around 330 tokens per second or less and take 9 seconds or more. LithosAI is more than 4× faster than most providers and returns answers in about a quarter of the time.
GLM 5.3 inference on LithosAI
LithosAI takes first place for GLM 5.3 on both output speed and end-to-end response time, at 717 tokens per second and 4.1 seconds.
Most providers deliver around 200 tokens per second or less and take more than 10 seconds to return a complete answer. LithosAI is more than 3× faster than most providers and returns answers in less than half the time.
LithosAI’s Endpoint Accuracy Index for GLM 5.3 is 98% of the reference. Artificial Analysis classifies the result as within the reference range, accounting for the uncertainty shown in the chart. The score is normalized to the reference endpoint and should not be interpreted as an absolute task-success rate.
Applying the results to your production workload #
The Artificial Analysis results show LithosAI leading across every metric: output speed, latency, and end-to-end response time, at full model quality. The next step is to see what that means for your own agents.
Every model on LithosAI comes in three performance tiers. Base is our lowest price and already fast, Fast is a clear step up for interactive agents, and Ultra is our top speed, up to 1,500 tokens per second per user. Try them on representative end-to-end tasks, measure task completion rate, latency, and total token cost under the load you expect, and choose the tier that fits your goals. Compare tiers and pricing.
And because ultra-fast inference needs ultra-fast sandboxes, LithosBox gives your agents environments that keep pace with the model.