Hi Everyone
First I would like to say thank you to everyone here. There are some really incredible people and projects being done here and i feel like a small fish in an enormous ocean. You guys are great keep up the good work and community.
For the last 3 years I have heard about LLM’s and how they can do all these amazing things. While i see it with Claude and other LLM’s in the main stream i look up video after video about Local LLM’s and how they are: 1: offline
2: personalized
3: Can write code and help with personal projects
4: Are able to run tasks for you on your own hardware
5: create your own home server/ media server
This all sounds awesome and something i want to learn about and take advantage of, that said the price of new hardware is INSANE and i have a lot of older PC hardware laying around my house that i would like to utilize.
The List of PC parts available to me:
My Main system
CachyOS x86_64
AMD Ryzen 7 7800X3D (16) @ 5.05 GHz AMD Radeon RX 9070 XT
32 GB (I believe this is DDR5 its a Gskill Flare x5)
My Second pc
AMD Ryzen 5 8400F
Gigabyte B650 Motherboard
Intel Arc B570 Sparkle Guardian OC Edition
16GB DDR5 6000MHz RAM
1TB NVMe SSD
Phanteks X2S Black Case
650W 80+ Power Supply
Parts i have laying around
#
#
I would appreciate any links to learning materials, videos, guides, or just your own 2 cents on what I’m trying to do. Thank you for the help and community.
TLDR: I want to learn how to create local LLM models with the hardware available to me, but am unsure where to start or how to find the information that i can readily use and put in practice. Would appreciate any information or practices that have helped you learn.
2 Likes
Easiest way is to setup LM-Studio on the machine and use either vulkan or ROCM models since you’ve got AMD based hardware. Don’t know how well Arc handles it but should also work with vulkan based models.
1 Like
agis 3
You’re off to a good start with your hardware. As @Neon0Blue said, LMStudio will have you up and playing fast ..
Here’s some youtubers to help you on your way ..
For starters, there’s this bloke called Wendell. He seems nice and
There’s a lot of step by step guides here in the forums too.
.. and then ..
This guy is pretty new, obviously a good teacher, straight forward step by step instructions to get you on your way. He’s more NVidia focused, but very modest hardware and very practical.
And then .. Chuck ..
He’s .. unique .. you’ll see. He’s a bunch of videos to get you going fast.
Lastly ..
.. a lot about a lot, a solid reference.
That’ll keep you busy for a bit ..
1 Like
You’re great thanks for all the HW.
1 Like
agis 5
No problem.
Oh, I should warn you .. be careful with the Wendell videos.
Persistent viewing can lead to optimism
1 Like
It’s so refreshing and a pleasant way of learning.
1 Like
s0me1 7
I would suggest finding something small you think a llm would help with(something that runs nightly or weekly automaticly)
Work with AI and build a local KB dash board where you document projects ect.
My local model does automatic code reviews also Downloads all my saved Instagram reels then transcribes them then posts them to my kb.. you can stretch a smaller model fairly far for specific task by building really small harnesses.
To recap find a problem and play with it.. them smash it..lol
you can start off with ollama it is easy and i recomend using a cli tool like opencode.. it is worth the learning curve