What's your setup for running models locally?
A Hugging Face forum discussion on local model inference drew advice that a 7B or 8B model at 4-bit quantization runs comfortably on 16GB of unified memory on a MacBook Pro, while anything larger starts swapping. The res…