AM4 motherboard with quad R9700's behind plx88096 ROMBAR Woes

A user on an AM4 X570-D4U-2L2T/BCM motherboard resolved missing PCIe BAR space that blocked four R9700 AI GPUs behind a PLX88096 switch by disabling Legacy CSM support in the BIOS, which also restored the onboard 10GbE controller. The fix left peer-to-peer (P2P) disabled in AMD-SMI topology inside the VM passed through all four devices, with the access table showing only self-links ENABLED between 0000:01:00.0 through 0000:04:00.0. The user had applied kernel arguments including pci=realloc=on, amd_iommu=on, iommu=pt and pcie_aspm=off before finding the CSM setting was the cause.

So i recently got my PLX88096 chip in the mail, and have used it to install 4 R9700 AI’s onto the system, however it seems my poor prosumer grade AM4 X570-D4U-2L2T/BCM motherboard has had enough of my shenanigans and just doesnt have the BAR space to deal with it My onboard 10Gbe also stopped working, but i suspect this is just a side effect of the lack of BAR space I’ve added the Dmesg logs and my kernel opts Please obi-wan janitobi’s, you guys are my only hope: Kernel Args: GRUB CMDLINE LINUX DEFAULT=“quiet amd iommu=on iommu=pt pcie aspm=off amdgpu.ras enable=0 pci=realloc=on amdgpu.runpm=0 amdgpu.gpu recovery=1 pcie acs override=downstream,multifunction, default hugepagesz=1G hugepagesz=1G hugepages=108 transparent hugepage=always” DMESG: See attachment dmesg.txt https://forum.level1techs.com/uploads/short-url/cuEyL12crKwewszQIO6ydsM2IAr.txt 46.6 KB For your viewing pleasure or distaste, depending on tolerance : It WORKS , it was Legacy CSM support being enabled, i have a new problem now though, even though everything is now properly showing inside the VM P2P is still disabled in AMD-SMI topology: ACCESS TABLE: 0000:01:00.0 0000:02:00.0 0000:03:00.0 0000:04:00.0 0000:01:00.0 ENABLED DISABLED DISABLED DISABLED 0000:02:00.0 DISABLED ENABLED DISABLED DISABLED 0000:03:00.0 DISABLED DISABLED ENABLED DISABLED 0000:04:00.0 DISABLED DISABLED DISABLED ENABLED WEIGHT TABLE: this is inside the VM that’s being passed through all these devices