Proxima

Proxima, an out-of-tree vLLM plugin implementing STAR-KV low-rank KV cache compression, serves 4x more concurrent requests at 8192 context and boots at 16384 context where plain vLLM refuses to start,…

Alibaba's Tongyi Lab released Zvec 0.5.0, an open-source in-process vector database under Apache 2.0, designed to bring SQLite-like simplicity to edge AI and RAG applications. The update introduces Di…