# Going squirrel on this new.....ish hobby

> Source: <https://forum.level1techs.com/t/going-squirrel-on-this-new-ish-hobby/260063#post_7>
> Published: 2026-11-16 12:10:43+00:00

AI Disclosure:I will be using AI to search for ideas and suggestions on ways to fix something I break. I unfortunately have only so much time and see it as a tool like you would a calculator.

Where do I “project start”…… I use to be into this stuff 10-15 years ago, but family, work, and volunteer FF took over. Mental health tanked a couple years ago, finally getting out of my funk and not just watching TV to pass the time.

Goal/project/pipe dream

1. Use proxmox and setup a secure host with backups to NAS.
2. Use Ubuntu server  and setup a llama.cpp server with Hermes agent for a 100% local ai digital assistant. Multiple models for different purposes.
3. Setup homarr on my NAS and make a nice dashboard.
4. Program an esp32 to display proxmox system info.
5. I don’t know…… first Devember……

 
4 Likes

 
Clarification - Ubuntu will be a VM in proxmox.

 

 
Welcome back. Since Ubuntu is going to be a VM anyway, figure out early how the NAS storage gets into it, because sharing a drive across VMs and LXCs in Proxmox took special config for me where plain Ubuntu just mounts it. That’s a big part of why I ended up on plain Ubuntu Server with SSH for spinning things up and tearing them down, and I’d only reach for Proxmox again for a bigger deployment. If you stay with Proxmox, get the backups to the NAS working and restore one VM before you start on llama.cpp, so a broken experiment is cheap to undo.

 
1 Like

 
My thoughts are I want the Ai system not to be able to access personal files unless I move them myself. I just want to backup the system for when I break something. I feel it’s easier to restore a backup then rebuild something……

I guess I am gun shy one losing files…… I know I am also bad for no off-site backup.

 
1 Like

 
OK, I got off work early and wanted to show what I am working with. I heard a janitor likes pictures……

### 

Core5 ultra 225f

32 GB RAM  ddr5

16GB NVIDIA GeForce RTX 4060Ti

Asus B860m max gaming ax motherboard

1tb nvme

512mb nvme

Modded Rosewill RSV-R4100U server chassis ([https://makerworld.com/en/models/1262238-rosewill-rsv-r4100u-replacement-front-panel?from=search#profileId-1287123](https://makerworld.com/en/models/1262238-rosewill-rsv-r4100u-replacement-front-panel?from=search#profileId-1287123))

I bought a pre built PC from Costco because I could not beat the price on clearance. It had a intel B570 that i swapped for the 4070. Took everything out of the case it came in and put it in the modified case.

### 

4x WD RED 6TB - in two RAID1 for storage and backup (1-Data and pictures 2-ISO’s and Proxmox backups)

2 x Patriot P300 256gb - in RAID1 for VM’s and docker

16GB RAM

Rack mounted ([https://makerworld.com/en/models/2346242-ugreen-dpx4800-plus-nas-19-inches-rack-mount?from=search#profileId-2565315](https://makerworld.com/en/models/2346242-ugreen-dpx4800-plus-nas-19-inches-rack-mount?from=search#profileId-2565315))

N97 mini PC - WANker

Using as a device that is exposed to the World via Cloudflare, hosing a personal website with wordpress.

And a CyberPower CP2000PFCRM2U PFC Sinewave UPS in a Startech rack.

I hope to have the Ai PC up and running with Hermes by then end of the weeked. If you have any questions or comments fell free.

 

 
**Nov 8th 2026 update** - I have the Ai PC up and running. Slowly getting more comfortable in Proxmox. Used a lot of help from google to get the GPU to work. Had to play around with settings to get the Qwen 3.8 to run with a large context for hermes. I am going to change the models around as I am not happy with DeepHermes, MedGemma, or Gemma4. This is what I have currently in my **model.ini** file.

*[*]
n-gpu-layers = 99
batch-size = 256
ubatch-size = 256
flash-attn = on
cache-type-k = q4_0
cache-type-v = q4_0
jinja = on*

*[qwen27b]
model = /home/XXXXX/llama.cpp/models/Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf
n-gpu-layers = 99
ctx-size = 131072
batch-size = 128
ubatch-size = 128
cache-type-k = q4_0
cache-type-v = q4_0
flash-attn = on*

*[deephermes8b]
model = /home/XXXXX/llama.cpp/models/DeepHermes-3-Llama-3-8B-q4.gguf
n-gpu-layers = 99
ctx-size = 262144*

*[medgemma4b]
model = /home/XXXXX/llama.cpp/models/medgemma-1.5-4b-it-UD-Q8_K_XL.gguf
n-gpu-layers = 99
ctx-size = 262144*

*[gemma4]
model = /home/XXXXX/llama.cpp/models/gemma-4-12b-it-Q4_0.gguf
n-gpu-layers = 99
ctx-size = 262144*

And this is my **llama-server.service**

*[Unit]
Description=Llama.cpp Server Instance
After=network.target local-fs.target nvidia-persistenced.service multi-user.target
Wants=nvidia-persistenced.service*

*[Service]
Type=simple
User=XXXXX
Group=XXXXX
WorkingDirectory=/home/XXXXX/llama.cpp
ExecStartPre=/bin/sh -c ‘nvidia-smi >/dev/null 2>&1 || exit 1’
ExecStart=/home/XXXXX/llama.cpp/build/bin/llama-server
–models-preset /home/XXXXX/llama.cpp/models/models.ini
–models-max 1
–port 8080
Restart=always
RestartSec=15*

*[Install]
WantedBy=multi-user.target*

Hermes is also operational with telegram and web search integration. I plan on researching skills this week at work to try that next weekend, and try a couple different models.

 
1 Like
