I am currently in a similar position deciding on what GPUs I choose for the future.
I made a spreadsheet to compare all the options. Prices are from Ebay in Euro and may change daily. But they give a ballpark at least.
Of course, I didn’t include the A100 yet. But regarding performance indicator of Techpowerup it would be relative 78, well below the reference RTX3090 with 100 and the RTX4500 with 137. On the other side internal memory bandwidth is nearly twice as high, which is good for inference. So I would assume that inference performance is similar. But the A100 would certainly win regarding training.
So there may be additional factors:
For me there are fewer variables. I have a EPYC system with 7 slots and need no extra switch. As I am on a lower budget I target 64 GB memory and will probly choose 4x RTX5080 as best compromise. Cost per GB and per performance is lower than for 2x RTX4500. And 4x RTX5080 has about twice the performance than 2x RTX4500. I would buy the 4500 only in the single-slot version. YMWV. Currently I run Qwen 27B Q6 with tensor split on a RTX4080 and an old RTX6000 Quadro with 30 - 50 t/s. I hope to double this at least. The speed should be above an RTX5090 or even a RTX6000 Blackwell, but for much less money.