Developer Ruvian Fabrin has built Cerne Code, a free, open-source desktop coding agent that runs 100% locally, compatible with Ollama, llama.cpp, LM Studio, and any OpenAI-compatible API. The agent ca…
Turbovec, a Rust vector index with Python bindings built on Google Research's TurboQuant algorithm, fits a 10 million document corpus into 4 GB of RAM versus 31 GB as float32 and searches faster than …
Qdrant, an AI vector search database supplier, has applied Google Research's TurboQuant compression technique to embeddings, claiming it achieves competitive recall at 8x compression and beats binary …
ForgeVector offers a practical approach to efficient similarity search by building a vector database on TurboQuant's quantized indices, enabling fast approximate nearest neighbor searches with signifi…
A new benchmark evaluates KV-cache optimization techniques—quantization, pruning, and merging—for long-context LLM serving, finding that compression ratio alone poorly predicts end-to-end performance.…
A developer released TurboQuant, a pip-installable adapter for mlx-lm on Apple Silicon that uses a randomized Hadamard transform for data-oblivious, calibration-free quantization of weights and KV cac…
A 27B-parameter Qwen3.6-Coder model running on a consumer AMD Strix Halo mini-PC successfully fixed a bug in the LLMKube Kubernetes operator by correcting a hardcoded timeout that caused request failu…
Huawei released KVarN, a native KV-cache quantization back end for vLLM that delivers up to 5x more cache capacity and 1.3x the throughput of FP16 while maintaining FP16-level accuracy. The calibratio…
Amazon Web Services announced a new method combining Amazon FSx for Lustre with NVIDIA GPUDirect Storage (GDS) to reduce large language model cold-start loading times from minutes to seconds on AWS GP…
Researchers have developed Speculative KV coding, a lossless compression method that reduces the size of large language model KV caches by up to approximately 4x using a smaller, faster predictor mode…