25L Portable NV-linked Dual 3090 LLM Rig

Post by mertleee » Tue, Sep 23, 2025, 3:59 PM UTC

dotnet00 wrote: post #73

Gracana wrote: post #68

Earlier quoted context omitted.

$2.5k is about $1k more than you'd spend on a pair of 3090s, and people I know who've bought blower 4090s say they sound like hair driers.

Blowers are loud, but they're easier to pack together, particularly given how most motherboards don't seem to space their two slots sufficiently to accomodate the massive coolers on recent GPUs.

Post by magicalhippo » Tue, Sep 23, 2025, 4:21 PM UTC

writebetterc wrote: post #98

someperson wrote: post #94

Earlier quoted context omitted.

What an indictment on NVidia market segmentation that there's an industry doing aftermarket VRAM upgrades on gaming cards due their intentionally hobbled VRAM. I wish AMD and Intel Arc would step up their game.

Intel Arc Pro B60 will come in a 48GB dual-GPU model. So yeah, hardware is gonna be there, and the 24GB model will be $599 from Sparkle. I assume 48GB will be cheaper than a hacked RTX 4090. Look at this: https://www.maxsun.com/products/intel-arc-pro-b60-dual-48g-t... https://www.sparkle.com.tw/files/20250618145718157.pdf

Post by naabb » Tue, Sep 23, 2025, 4:24 PM UTC

NicoJuicy wrote: post #53

lifeinthevoid wrote: post #42

I built a similar system, meanwhile I've sold one of the RTX 3090's. Local inference is fun and feels liberating, but it's also slow, and once I was used to the immense power of the giant hosted models, the fun quickly disappeared. I've kept a single GPU to still be able to play a bit with light local models, but not anymore for serious use.

If you have a 24 gb 3090. Try out qwen:30b-a3b-instruct-2507-q4_K_M ( ollama ) It's pretty good.

Post by BizarroLand » Tue, Sep 23, 2025, 5:05 PM UTC

juliangoldsmith wrote: post #67

glax wrote: post #52

Earlier quoted context omitted.

Sorry for going off topic. But your insight will be helpful on my build I'm thinking about a low budget system, which will be using 1.X99 D8 MAX LGA2011-3 Motherboard - It has 4 pcie 3.0 x16 slots, dual cpu socket. They are priced around $260 with both the cpu 2. 4X AMD MI50 32G cards - They are old now, but they have 32 gigs of vram and also can be sources at $110 each The whole setup would not cost more than $1000,…

I'd use caution with the Mi50s. I bought a 16GB one on eBay a while back and it's been completely unusable. It seems to be a Radeon VII on an Mi50 board, which should technically work. It immediately hangs the first time an OpenCL kernel is run, and doesn't come back up until I reboot. It's possible my issues are due to Mesa or driver config, but I'd strongly recommend buying one to test before going all in. There ar…

Post by tryauuum » Tue, Sep 23, 2025, 5:26 PM UTC

tacomagick wrote: post #78

jychang wrote: post #48

Earlier quoted context omitted.

Yeah, this page seems to be not great for beginners and also useless for people with experience. A 2x 3090 build is okay for inference, but even with nvlink you're a bit handicapped for training. You're much better off with getting a 4090 48GB from China for $2.5k and just using that. Example: https://www.alibaba.com/trade/search?keywords=4090+48gb&pric... Also, this phrasing is concerning: > WARNING - these componen…

Simply replacing the 3090's with 4090's would provide a major performance uplift assuming your model fits. (I have rented both 3090 and 4090 systems online for research, this comment is based on my personal experience, it is well worth the price increase and the hourly rate for the inference speed you get)