A developer built a 56-task coding benchmark with hidden tests and ran 16 model configurations on the same hardware, finding that a 7.5B-parameter model (gemma-4-e4b) scored 42/56, beating a 24B model…
A fundamental flaw in how large language models identify instructions makes them impossible to fully secure against attacks, researchers argue in a paper presented at the International Conference on M…
A first-principles analysis of LoRA fine-tuning memory usage reveals that 87.3% of VRAM scaling with sequence length for Llama 3.1 8B is consumed by the cross-entropy loss head tensor, not the model, …
Open-weights models are dramatically cheaper per token than closed models, with the five cheapest models on the Artificial Analysis pricing index all being open-weights and the five most expensive all…
Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that large language models suffer from 'role confusion,' mistaking the style of text for its actual content, leading to successful …