A developer's on-hardware test shows that prefetching mixture-of-experts (MoE) expert weights from disk ahead of matmuls can improve token generation speed on a severely resource-constrained machine, …
LanceDB's guide to data loading for AI/ML explains the process of moving data into algorithms, focusing on PyTorch and LanceDB, and covers the I/O and CPU stages, noting that I/O is rarely the bottlen…
Constraining the output space instead of parsing generated text can make small language models (SLMs) faster and more reliable for narrow automation tasks, according to a technical article benchmarkin…
Independent researcher Rodrigo Campos is seeking an arXiv endorser for cs.CL to submit a paper titled 'Self-State Does Not Travel: Measuring and Building Temporal Transport of Uncertainty in a Small T…
An engineer investigating speculative decoding speedup found that the algorithm's performance ceiling is determined by acceptance rate and cost ratio, not just hardware overhead. Testing on Apple Sili…
A technical guide demonstrates the limitations of base language models like GPT-2 for domain-specific tasks and introduces fine-tuning as a solution, showing that GPT-2 produces inconsistent outputs w…
A developer built EdgeSync-LLM, a mechanism that reuses KV cache state from llama.cpp to avoid recomputing shared prefixes during local LLM inference. On a real Android ARM64 phone, the approach achie…
A new physics-informed AI pipeline uses supervised LoRA fine-tuning on a small language model to reliably output structured engineering data, which a deterministic physics checker then validates. The …
An independent researcher analyzed hidden-state dynamics across seven open-weight LLMs and found recurring functional patterns encoded in representation geometry rather than individual neurons, with f…
A developer benchmarked speculative decoding using Qwen2.5-0.5B-Instruct as the draft model and Qwen2.5-1.5B-Instruct as the target model on a CPU. Across code, JSON, and story generation tasks, specu…
A developer wrote a fused decode-attention kernel that ran 2.2× faster than the baseline in microbenchmarks, but when integrated into a HuggingFace `generate` call for an RL training loop, the decode …