cd/entity/OffloadingConnector· home› entities› OffloadingConnector
grep -l @offloadingconnector /news/*.json | wc -l → 1

OffloadingConnector

mentions 1 type Organization feed RSS

// recent coverage 1 mentions

00:00
2026-09-15
doug.sh
ai-infrastructure

Keeping vLLM's Prefix Cache Warm Between Agent Turns

Tuning vLLM 0.28.0's prefix cache raised the share of prompt tokens served from cache from 55% to 95% on a local Qwen3.8-27B coding agent, cutting average time to first token from 26-28 seconds to 7.3…

// co-occurs with top 7 entities
vLLM 0.28.0 1 Qwen3.8-27B 1 oh-my-pi 1 Bifrost 1 RTX 3090 1 W4A16 AutoRound 1 Claude Code 1
// topics top 5 topics
ai infrastructure 1 large language models 1 ai agents 1 mlops 1 ai tools 1