WakeKV, a reactive KV-cache residency policy, improves miss rate over frozen head classification and destructive eviction at matched memory or budget, according to an arXiv paper (2610.02713v1). Acros…
A comparative analysis examines the trade-offs between open-weight large language models such as Meta's LLaMA 2, Mistral AI's Mistral-7B, and TII's Falcon 180B and proprietary API-first offerings incl…
A Rust-based tool called Spanda computes LLM epistemic uncertainty in 652.1 nanoseconds (0.65 µs) per evaluation, roughly 90,000× faster than neural Semantic Entropy methods that add about 92.4 ms of …
OpenAI has re-introduced a hard 5-hour daily usage limit for Plus and Business accounts, capping total compute time on paid models like GPT-4-Turbo within a rolling 24-hour window. The restriction is …
Researchers introduced Sarathi-Serve, an LLM inference scheduler that uses chunked-prefills and stall-free scheduling to improve throughput-latency tradeoffs, achieving 2.6x higher serving capacity fo…
A new retrieval-augmented generation framework for corrosion knowledge synthesis, demonstrated on magnesium alloy corrosion, achieved Token F1 gains of 143-194% and system faithfulness of 0.964, accor…
A new arXiv preprint (2608.28911v1) introduces SemKV, a semantic mixed-precision KV cache quantization method that achieves a measured 6.0x storage reduction for long-context LLM inference with no sta…
Researchers propose REP-LIE, a pruning method that estimates weight importance from LoRA low-rank matrix gradients during finetuning, avoiding full gradient computation and prior finetuning. Tests on …
A new measurement study across four models finds that the default LoRA target_modules of q_proj and v_proj underperforms a data-driven placement for capability retention: on Llama-3-8B fine-tuning cod…
Researchers introduced Pattern Stability Score (PSS), a watermark detection framework for machine-generated text that improves detection AUC by over 10-15 percentage points across different token leng…
A study of Mistral-7B found that demographic identity is readable and faithfully arranged in attention heads, but not causally used, with a single head (L11 H16) faithful across all six attribute type…
A developer has shared techniques for reducing machine learning model inference costs by up to 80% while maintaining output quality comparable to premium services like Fable. The approach combines qua…
Researchers from the University of Waterloo and the University of Alberta found that explicit monitoring cues do not enforce robust alignment in large reasoning models but instead trigger strategic de…
Vendor lock-in in AI infrastructure creates technical debt and forces companies into a one-size-fits-all approach, warns a technical guide. To avoid this, the guide recommends implementing a Gateway P…
A benchmark study comparing standard non-confidential execution with confidential computing mode on a single NVIDIA H100 80GB GPU hosted in an Intel TDX confidential instance found that confidential m…
A new multi-agent committee called TriAgent, described in arXiv:2607.19794v1, achieves cost-efficient financial sentiment analysis by routing queries through a word-level lexicon (VADER), a sentence-l…
A developer introduced resk-mark, an open-source tool that embeds cryptographic watermarks into LLM-generated text by biasing token sampling with a secret key. The watermark survives rewording and tru…
Researchers introduced the Relative Probability Association Metric (RPAM), a new upstream metric for evaluating associations in generative language models. Testing on Mistral-7B-Instruct, Mistral-7B, …
A developer released TIS (Token Importance Scoring), a learned method for compressing the KV cache in large language models, achieving 100% accuracy on synthetic retrieval at 50% cache budget. The app…
A Hugging Face forum user asked which open-source large language model the community prefers for chatbot development and why, sparking discussion on model selection criteria.…