Dual R9700 llama.cpp Rocm tensor-split, working P2P on Proxmox, Qwen3.8-27B

A Proxmox VE 9 host with two Radeon AI PRO R9700 32 GB GPUs achieved working peer-to-peer (P2P) transfers at ~13 GiB/s and ran Qwen3.8 27B UD-Q8_K_XL at ~31 t/s (non-MTP) and ~49-50 t/s (MTP n=3) using ROCm and tensor split 1:1, after setting 'cpu: host,guest-phys-bits=44' in the Proxmox config to fix PCIe atomic ops. The user reports that enabling mmproj caused random repeating '//////' outputs across multiple quants and builds, and disabling mmproj stopped the issue.

A little preamble, I have two R9700’s on an AM4 board with a 3900X both of which are hung off the CPU now at x8x8 Gen 4. For layer split it is not as important to be CPU direct but for Tensor parallelism as many people on here have stated its pretty important. The thing I have been struggling with is getting this all to behave inside a VM on Proxmox. below is the some technical details behind it. Now I will admit I am not an expert in this but I am sharing my experience getting Qwen3.8 27B Unsloth’s UD-Q8 K XL quant to run without going insane with CPU usage and limiting host memory footprint. The resulting setup. Host: Proxmox VE 9 Guest: Ubuntu 26.04 GPUs: 2x Radeon AI PRO R9700 32 GB Model: Qwen3.8 27B UD-Q8 K XL GGUF Backend: ROCm Split: tensor 1:1 P2P copy: ~13 GiB/s each direction Non-MTP: ~31 t/s single stream MTP n=3: ~49-50 t/s single stream P3: ~19-25 t/s/request with 3 simultaneous streams The problems I needed to solve as I found them. amdgpu … PCIE atomic ops is not supported cpu: host,guest-phys-bits=44 in the proxmox config which puts the BAR low enough that P2P started working. cat /sys/module/amdgpu/parameters/pcie p2p returned Y. PP512: 871.65 t/s TG128: 31.42 t/s PP512: 934.58 t/s TG128: 28.58 t/s Some other things that became rather troublesome along the way. Enabling mmproj in any one of Qwen 35B or 27B 3.6 or 3.8 models seems to give you a random chance of the repeating ////////’s of annoyance as I like to call them. This is something I experienced across atleast 6 different quants of different models from different people including the originals and atleast another 6 different builds of llama.cpp from lemonade, unsloth, custom, release builds, vulkan and rocm backends. The only thing that seemed to stop it so far fingers crossed is disabling mmproj. I am not an expert, there is likely some things in here wrong or incomplete and I am willing to tinker some more and learn. I don’t do this stuff for work, I am purely doing it for the fun. I’ve done a lot more reading here than posting and let me tell you this place has been an invaluable resource.