How can I learn to have my own Local LLM agent?

A Level1Techs forum user with an AMD Ryzen 7 7800X3D, Radeon RX 9070 XT and 32GB of DDR5 asked the community how to start running local LLM agents on existing hardware, citing offline operation, personalization and task automation as goals. Respondents recommended LM Studio with Vulkan or ROCm models for the AMD-based systems, and suggested Ollama with a CLI tool such as opencode, while advising the user to pick a small recurring task and build a narrow harness around a smaller model.

Hi Everyone First I would like to say thank you to everyone here. There are some really incredible people and projects being done here and i feel like a small fish in an enormous ocean. You guys are great keep up the good work and community. For the last 3 years I have heard about LLM’s and how they can do all these amazing things. While i see it with Claude and other LLM’s in the main stream i look up video after video about Local LLM’s and how they are: 1: offline 2: personalized 3: Can write code and help with personal projects 4: Are able to run tasks for you on your own hardware 5: create your own home server/ media server This all sounds awesome and something i want to learn about and take advantage of, that said the price of new hardware is INSANE and i have a lot of older PC hardware laying around my house that i would like to utilize. The List of PC parts available to me: My Main system CachyOS x86 64 AMD Ryzen 7 7800X3D 16 @ 5.05 GHz AMD Radeon RX 9070 XT 32 GB I believe this is DDR5 its a Gskill Flare x5 My Second pc AMD Ryzen 5 8400F Gigabyte B650 Motherboard Intel Arc B570 Sparkle Guardian OC Edition 16GB DDR5 6000MHz RAM 1TB NVMe SSD Phanteks X2S Black Case 650W 80+ Power Supply Parts i have laying around I would appreciate any links to learning materials, videos, guides, or just your own 2 cents on what I’m trying to do. Thank you for the help and community. TLDR: I want to learn how to create local LLM models with the hardware available to me, but am unsure where to start or how to find the information that i can readily use and put in practice. Would appreciate any information or practices that have helped you learn. 2 Likes Easiest way is to setup LM-Studio on the machine and use either vulkan or ROCM models since you’ve got AMD based hardware. Don’t know how well Arc handles it but should also work with vulkan based models. 1 Like agis https://forum.level1techs.com/u/agis 3 You’re off to a good start with your hardware. As @Neon0Blue https://forum.level1techs.com/u/neon0blue said, LMStudio will have you up and playing fast .. Here’s some youtubers to help you on your way .. For starters, there’s this bloke called Wendell. He seems nice and There’s a lot of step by step guides here in the forums too. .. and then .. This guy is pretty new, obviously a good teacher, straight forward step by step instructions to get you on your way. He’s more NVidia focused, but very modest hardware and very practical. And then .. Chuck .. He’s .. unique .. you’ll see. He’s a bunch of videos to get you going fast. Lastly .. .. a lot about a lot, a solid reference. That’ll keep you busy for a bit .. 1 Like You’re great thanks for all the HW. 1 Like agis https://forum.level1techs.com/u/agis 5 No problem. Oh, I should warn you .. be careful with the Wendell videos. Persistent viewing can lead to optimism 1 Like It’s so refreshing and a pleasant way of learning. 1 Like s0me1 https://forum.level1techs.com/u/s0me1 7 I would suggest finding something small you think a llm would help with something that runs nightly or weekly automaticly Work with AI and build a local KB dash board where you document projects ect. My local model does automatic code reviews also Downloads all my saved Instagram reels then transcribes them then posts them to my kb.. you can stretch a smaller model fairly far for specific task by building really small harnesses. To recap find a problem and play with it.. them smash it..lol you can start off with ollama it is easy and i recomend using a cli tool like opencode.. it is worth the learning curve