Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

25L Portable NV-linked Dual 3090 LLM Rig

reddit.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#61

Post by fkyoureadthedoc »

Tepix wrote: post #29

OK, here's my quick critique of the article (having built a similar AM4-based system in 2023 for 2300€): 1) [I thought] The page is blocking cut & paste. Super annoying! 2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs. 3) The shopping link for the mainboard l…

> The page is blocking cut & paste. Super annoying!

I've been running Don't F* With Paste* for years for this

https://chromewebstore.google.com/detail/dont-f-with-paste/n...

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#62

Post by segmondy »

ismailmaj wrote: post #56
vladgur wrote: post #14

Earlier quoted context omitted.

I am exploring options just for fun. a used 3090 is around $900 on ebay. a used rtx 6000 ADA is around $5k 4 3090s are slower at inference and worse at training than 1 rtx 6000. 4x3090 would consume 1400W at load. Rtx 6000 would consume 300W at load. If you god forbid live in California and your power averages 45 cents per kwh, 4x3090 would be $1500+ more per year to operate than a single RTX 6000[0] [0] Back of the…

To make matters worse, the RTX3090 was released during the crypto craze and so a decent amount of the second hand market could contain overused GPUs that won’t last long, even if 3xxx to 4xxx performance difference is not that high, I would avoid the 3xxx series totally for resell value.

I have rig of 7 3090s that I bought from crypto bros, they are lasting quite alright and have been chugging along fine for the last 2 years. GPUs are electronic devices not mechanical devices, they rarely blow up.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#63

Post by segmondy »

vladgur wrote: post #14
tensorlibb wrote: post #2

I'm a huge fan of OpenRouter and their interface for solid LLM's but I recently jumped into fine tuning / modifying my own vision models for FPV drone detection (just for fun) and my daily workstation and it's 2080 just wasn't good enough. Even in 2025 it's cool how solid a setup dual 3090's still are. nvlink is an absolute must but it's incredibly powerful. I'm able to run the latest Mistral thinking models and rela…

I am exploring options just for fun. a used 3090 is around $900 on ebay. a used rtx 6000 ADA is around $5k 4 3090s are slower at inference and worse at training than 1 rtx 6000. 4x3090 would consume 1400W at load. Rtx 6000 would consume 300W at load. If you god forbid live in California and your power averages 45 cents per kwh, 4x3090 would be $1500+ more per year to operate than a single RTX 6000[0] [0] Back of the…

... and this is why napkin calculation is terrible. Even running a GPU at load doesn't mean you are going to use the full wattage. 4 3090 running inference on large model barely uses 350watts combined.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#64

Post by elsombrero »

imiric wrote: post #59
elsombrero wrote: post #55

Earlier quoted context omitted.

On my 2x 3090s I am running glm4.5 air q1 and it runs at ~300pp and 20/30 tk/s works pretty well with roo code on vscode, rarely misses tool calls and produces decent quality code. I also tried to use it with claude code with claude code router and it's pretty fast. Roo code uses bigger contexts, so it's quite slower than claude code in general, but I like the workflow better. this is my snippet for llama-swap ``` mo…

Thanks, but I find it hard to believe that a Q1 model would produce decent results. I see that the Q2 version is around 42GB, which might be doable on 2x 3090s, even if some of it spills over to CPU/RAM. Have you tried Q2?

well, I tried it and it works for me. llm output is hard to properly evaluate without actually using it.

I read a lot of good comments on r/localllama, with most people suggesting qwen3 coder 30ba3b, but I never got it to work as well as GLM 4.5 air Q1.

As for using Q2, it will fit in vram, but with very small context or spill over to RAM, but with quite an impact on speed depending on your setup. I have slow ddr4 ram and going for Q1 has been a good compromise for me, but YMMV.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#65

Post by jonbiggums22 »

segmondy wrote: post #62
ismailmaj wrote: post #56

Earlier quoted context omitted.

To make matters worse, the RTX3090 was released during the crypto craze and so a decent amount of the second hand market could contain overused GPUs that won’t last long, even if 3xxx to 4xxx performance difference is not that high, I would avoid the 3xxx series totally for resell value.

I have rig of 7 3090s that I bought from crypto bros, they are lasting quite alright and have been chugging along fine for the last 2 years. GPUs are electronic devices not mechanical devices, they rarely blow up.

I've noticed on ebay there are a lot of 3090s for sale that seem to have rusted or corroded heatsinks. I actually can't recall seeing this with used GPUs before but maybe I just haven't paying attention. Does this have to do with running them flat out in a basement or something?

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#67

Post by juliangoldsmith »

glax wrote: post #52
Tepix wrote: post #29

OK, here's my quick critique of the article (having built a similar AM4-based system in 2023 for 2300€): 1) [I thought] The page is blocking cut & paste. Super annoying! 2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs. 3) The shopping link for the mainboard l…

Sorry for going off topic. But your insight will be helpful on my build I'm thinking about a low budget system, which will be using 1.X99 D8 MAX LGA2011-3 Motherboard - It has 4 pcie 3.0 x16 slots, dual cpu socket. They are priced around $260 with both the cpu 2. 4X AMD MI50 32G cards - They are old now, but they have 32 gigs of vram and also can be sources at $110 each The whole setup would not cost more than $1000,…

I'd use caution with the Mi50s. I bought a 16GB one on eBay a while back and it's been completely unusable.

It seems to be a Radeon VII on an Mi50 board, which should technically work. It immediately hangs the first time an OpenCL kernel is run, and doesn't come back up until I reboot. It's possible my issues are due to Mesa or driver config, but I'd strongly recommend buying one to test before going all in.

There are a lot of cheap SXM2 V100s and adapter boards out now, which should perform very well. The adapters unfortunately weren't available when I bought my hardware, or I would have scooped up several.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#68

Post by Gracana »

jychang wrote: post #48
Tepix wrote: post #29

OK, here's my quick critique of the article (having built a similar AM4-based system in 2023 for 2300€): 1) [I thought] The page is blocking cut & paste. Super annoying! 2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs. 3) The shopping link for the mainboard l…

Yeah, this page seems to be not great for beginners and also useless for people with experience. A 2x 3090 build is okay for inference, but even with nvlink you're a bit handicapped for training. You're much better off with getting a 4090 48GB from China for $2.5k and just using that. Example: https://www.alibaba.com/trade/search?keywords=4090+48gb&pric... Also, this phrasing is concerning: > WARNING - these componen…

$2.5k is about $1k more than you'd spend on a pair of 3090s, and people I know who've bought blower 4090s say they sound like hair driers.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#69

Post by hengheng »

Tepix wrote: post #29

OK, here's my quick critique of the article (having built a similar AM4-based system in 2023 for 2300€): 1) [I thought] The page is blocking cut & paste. Super annoying! 2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs. 3) The shopping link for the mainboard l…

I've also learned the hard way to Google "AM4 main board tier list" before buying.

Some boards can run a 5950X in name only, while others can comfortably run it close to double its spec power all day. VRMs are a real differentiator for this tier of hardware.

(If anyone can comment on the airflow required for 400-500W Epyc CPUs with the tiny VRM heatsinks that Supermicro uses, I'm all ears.)

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#70

Post by prmoustache »

Tepix wrote: post #46
prmoustache wrote: post #44

Earlier quoted context omitted.

I have js enabled and I can copy text on this page.

In general I can too, but try copying items from the "key specifications". Or perhaps I just had the impression because you can't mark text because I can't tell which text is marked and which isn't when marking text under "Key Specifications". Mea culpa.

yeah the selection is dark grey over black so it is not super visible but you can copy text.
Post reply on HN