I recently lucked into a Samsung Galaxy Book 5 360 Pro(talk about a mouthful) with an Intel 258v 32GB model. I wanted to see how capable it was running local AI models and I’ve been kinda disappointed. Is there something I’m missing?
Ubuntu Server 26.04 - Kernel 7.2
OpenVINO - So incredibly immature, I can run Qwen 3.6 35B A3B if I pull the latest nightlies and build everything myself but it just sucks in comparison to llama.cpp performance wise.
llama.cpp - I tried Sycl and Vulkan, Vulkan was the clear winner with double the tokens per second output.
I thought this generation was supposed to be more capable of running local models? Obviously there’s limitations with RAM amounts but parsing an A3B model and only getting roughly 30 t/s decode and 200 t/s prefill(honestly this is the more painful part) really sucks.