EdgeSync-LLM

mentions 2 type Organization feed RSS

// recent coverage 2 mentions

21:17

2026-07-11

dev.to

large-language-models

I Got 9.9 Lower TTFT on a Real Android Phone by Reusing llama.cpp KV State

A developer built EdgeSync-LLM, a mechanism that reuses KV cache state from llama.cpp to avoid recomputing shared prefixes during local LLM inference. On a real Android ARM64 phone, the approach achie…

14:10

2026-06-30

github.com

large-language-models

EdgeSync-LLM – KV cache fragment engine for on-device LLM inference (Go/Android)

EdgeSync-LLM, a new KV cache fragment engine for on-device LLM inference, stores and retrieves transformer KV tensors via HNSW approximate nearest-neighbor search, enabling exact hits at ~8ms TTFT and…

// co-occurs with top 8 entities

llama.cpp 2 Android 2 MLC-LLM 1 ONNX Runtime 1 ARM 1 HNSW 1 MiniLM-L6-v2 1 Qwen2.5-0.5B-Instruct 1

// topics top 5 topics

large language models 2 ai infrastructure 2 artificial intelligence 1 ai tools 1 developer tools 1