I have little idea what I’m doing, but I’ve been tinkering for the past couple of months and scratching the surface with some LLMs. My linux box is running a Blackwell GPU with 96GB of VRAM and I’ve got 125GB of CPU RAM. Figured I’d wince and pick it up because the trend of us commoners not being allowed to purchase things seemed not great, and so far it looks like I was right although I’m still paying off that credit card from back when it was “affordable-ish.”
I’m just running Ollama models and mostly through OpenWebUI although I think I will be transitioning away from OpenWebUI (installed it without docker and it always needs to phone home over the Internet before it will boot up the server interface). Anyway, for coding my favorite model has been gpt-oss:120b. It does a great job when I post multiple file code into it with hundreds of lines. And it puts it out quicker than I can read it. Sometimes it will get stuck and I’m not sure how much of that is from a long chat and deterioration, but most of the time it figures out issues. Recently it did have an issue with a PHP reading a datetime from a JSON file and the PHP would apply some time zone to it for some reason, and it couldn’t figure out how to fix that even though it was far less lines than I usually pump into it. So I put it to Qwen3.8:27b and it fixed it no problem. But normally it’s my go to, but I’m going to do some more work with Qwen3.8 and also the Qwen3.5:122b and they may take the top spot.
For vision stuff I have my IP cameras just using YOLO for object detection, but during sleep hours and at certain locations on the property I will have it then send the image to Llama4:108b to get more info before I have the script take action (like alert us and yell at bears or moose with a networked loud speaker, or for funsies see if a person is holding an invisible bat before it plays the “What I Believe” rant from Bull Durham). It works pretty well, although when I ask it ten questions in a single prompt and ask it to respond with A, B, C etc for processing whether to play “YMCA” or “Gangham Style” or the Bull Durham rant based on what a person is doing, it doesn’t do as well. But I’m just dipping my toe into the vision stuff. I think I’m about to try Anything LLM and start doing some RAG stuff next.