Earlier quoted context omitted.
> I'd much prefer paying 3x cost for 3x VRAM Why not just buy 3 card then? These cards doesn't require active cooling anyways and you can just fit 3 in decent sized case. You will get 3x VRAM speed and 3x compute. And if your usecase is llm inference, it will be a lot faster than 1x card with 3x VRAM.
because then instead of RAM bandwidth now you're dealing with PCIe BW which is way less.
Currently running them different VMs to be able to make full use of them, used to have them running in different docker containers however OOM Exceptions would frequently bring down the whole server, which running in VMs helped resolve.