Does anyone have experience working with ROCm and CUDA on the same machine?

A user with an AMD RX 7900 XT is considering adding lower-end Nvidia RTX GPUs to run multiple local AI models on different cards, but is concerned about potential issues with running ROCm and CUDA side-by-side. The user asks whether there are material limitations beyond setup inconvenience, planning to put a mid-tier model on the 7900 and lower-tier models on the RTXs.

I have a RX 7900 XT in my machine and might be able to get one or two lower end RTXs. I have been experimenting with running models locally and thought it would be interesting to try and load multiple models on different gpus and route between them. However, here and there I read that heterogeneous hardware can be a real headache eg, using ROCm and CUDA side-by-side and that it should be avoided. I don’t actually have a link to where I read this, it has been just in passing and that was the impression I got. I get having all the various tools installed and needing to set things up to work with both are extra steps, but is there a material limitation or disadvantage to doing this? Something beyond annoyance or inconvenience? I’m interested this for the sake of knowing, but for added context my current plan is to put a mid-tier model on my 7900 and then lower tier models on the RTXs.