Llama.cpp's default layer-split mode gives a dual-GPU setup the combined VRAM of both cards but runs single-stream generation at roughly the speed of one card, while tensor parallel can nearly double …
Researchers introduced Private Best-of-N (PrivBoN) and Private Inference-Time Pessimism (PrivITP), two differentially private inference-time alignment methods that add calibrated noise to reward score…
A developer testing Qwen3.5 4B via Ollama found that enabling chain-of-thought can return an empty response while the thinking field holds content, and that self-critique can degrade accuracy, citing …
Post-training quantization (PTQ) of large language models fails under extreme low-bit constraints because dynamic activation outliers scale beyond 1⁰⁵, causing representation collapse, according to a …
A new wave of AI hardware and software co-design is replacing heavy floating-point matrix multiplications with addition-only, ternary weight sets {-1, 0, 1}, enabling up to a 32-fold reduction in weig…
Huang et al. published a paper on arXiv 2025 introducing Self Forcing, a method that bridges the train-test gap in autoregressive video diffusion by rolling out the model on its own outputs during tra…