Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

25L Portable NV-linked Dual 3090 LLM Rig

reddit.com

Post reply on HN
Print view View original

81–90 of 126 posts

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#81

Post by loudmax »

There was an interesting post to r/LocalLLaMA yesterday from someone running inference mostly on CPU: https://carteakey.dev/optimizing%20gpt-oss-120b-local%20infe...

One of the observations is how much difference memory speed and bandwidth makes, even for CPU inference. Obviously a CPU isn't going to match a GPU for inference speed, but it's an affordable way to run much larger models than you can fit in 24GB or even 48GB of VRAM. If you do run inference on a CPU, you might benefit from some of the same memory optimizations made by gamers: favoring low-latency overclocked RAM.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#82

Post by dwood_dev »

jonbiggums22 wrote: post #65
segmondy wrote: post #62

Earlier quoted context omitted.

I have rig of 7 3090s that I bought from crypto bros, they are lasting quite alright and have been chugging along fine for the last 2 years. GPUs are electronic devices not mechanical devices, they rarely blow up.

I've noticed on ebay there are a lot of 3090s for sale that seem to have rusted or corroded heatsinks. I actually can't recall seeing this with used GPUs before but maybe I just haven't paying attention. Does this have to do with running them flat out in a basement or something?

Run near a saltwater source without AC and that will happen.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#83

Post by noboostforyou »

nxobject wrote: post #71
jychang wrote: post #48

Earlier quoted context omitted.

Yeah, this page seems to be not great for beginners and also useless for people with experience. A 2x 3090 build is okay for inference, but even with nvlink you're a bit handicapped for training. You're much better off with getting a 4090 48GB from China for $2.5k and just using that. Example: https://www.alibaba.com/trade/search?keywords=4090+48gb&pric... Also, this phrasing is concerning: > WARNING - these componen…

Are the Alibaba 4090s modded to reach 48GB VRAM? (I ask only to figure how why they're that cheap...)

Yes, they are modded by replacing the individual VRAM modules. https://www.tomshardware.com/pc-components/gpus/usd142-upgra...

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#84

Post by vidarh »

jgalt212 wrote: post #72
fkyoureadthedoc wrote: post #61

Earlier quoted context omitted.

> The page is blocking cut & paste. Super annoying! I've been running Don't F* With Paste* for years for this https://chromewebstore.google.com/detail/dont-f-with-paste/n...

Interesting. I guess our content-based marketing pages need to move to canvas-based rendering. That's probably bum too. Straight to serving up jpgs.

> Straight to serving up jpgs.

Back in my Amiga-days we had PowerSnap[1] which did the bargain basement version of OCR: Check the font settings of the window you wanted to cut and paste from, and try to match the font to the bitmap, to let you copy and paste from apps that didn't support it, or from UI element you normally couldn't.

These days, just throwing the image at an AI model would be far more resilient...

I think we've gotten to the point where it would be hard to compose an image that humans can read but an AI model can't, and easy to compose an image an AI can read but humans can't, so I suspect the only option for your marketing department will be to try to prompt inject the AI into buying your product.

(Oh, look, I have written nearly this same comment once before, 11 years ago, on HN[2] - I was wrong about how it worked, and Orgre was right, and my follow up reply appears to be closer to what it actually does)

[1] https://aminet.net/package/util/cdity/PowerSnap22a

[2] https://news.ycombinator.com/item?id=7631161

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#85

Post by jacquesm »

ericdotlee wrote: post #79
jacquesm wrote: post #12

Earlier quoted context omitted.

To be fair though, the 4090 and 5090 are much easier capable of saturating PCI express than the 3090 is, even at 4 lanes per card the 3090 rarely manages to saturate the links, it still handsomely pays off to split down to 4 lanes and add more cards. I used: https://c-payne.com/ Very high quality and manageable prices.

I've purchase 16 of these - cpayne is great! Hope he finds a US distributor to help with tariffs a bit!

What blew me away is the quality and price point of what obviously can't be a very high volume product. This guy makes amazing stuff.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#86

Post by OakNinja »

glax wrote: post #52
Tepix wrote: post #29

OK, here's my quick critique of the article (having built a similar AM4-based system in 2023 for 2300€): 1) [I thought] The page is blocking cut & paste. Super annoying! 2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs. 3) The shopping link for the mainboard l…

Sorry for going off topic. But your insight will be helpful on my build I'm thinking about a low budget system, which will be using 1.X99 D8 MAX LGA2011-3 Motherboard - It has 4 pcie 3.0 x16 slots, dual cpu socket. They are priced around $260 with both the cpu 2. 4X AMD MI50 32G cards - They are old now, but they have 32 gigs of vram and also can be sources at $110 each The whole setup would not cost more than $1000,…

Better to buy one used 3090 than those old cards. Everything is not vram. Or, you can do nothing without vram but you can’t do anything with just vram.

To use the second pair of pcie slots, you _must_ have two cpus installed. Just saying in case someone finds a board with just one cpu socket populated.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#87

Post by akulbe »

segmondy wrote: post #62
ismailmaj wrote: post #56

Earlier quoted context omitted.

To make matters worse, the RTX3090 was released during the crypto craze and so a decent amount of the second hand market could contain overused GPUs that won’t last long, even if 3xxx to 4xxx performance difference is not that high, I would avoid the 3xxx series totally for resell value.

I have rig of 7 3090s that I bought from crypto bros, they are lasting quite alright and have been chugging along fine for the last 2 years. GPUs are electronic devices not mechanical devices, they rarely blow up.

How do you have a rig that fits that many cards?? those things take 3 slots apiece.

Pictures, or it never happened! :D

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#88

Post by ProllyInfamous »

robotswantdata wrote: post #36
suladead wrote: post #17

I built pretty much this exact rig myself, but now it's gathering dust, any other uses for this rather than localLLMS

Heating

I use an older machine/GPU for wintertime heating, mining Monero (xmrig).

Should one get lucky and guess the next valid block, that pays the entire month's electricity — since an electric space heater would already be consuming the exact same amount of kWH as this GPU, there is no "negative cost" to operate.

This machine/GPU used to be my main workhorse, and still has ollama3.2 available — but even with HBM, 8GB of VRAM isn't really relevant in LLM-land.

Re: 25L Portable NV-linked Dual 3090 LLM Rig

#89

Post by lifeinthevoid »

XCSme wrote: post #37
tensorlibb wrote: post #2

I'm a huge fan of OpenRouter and their interface for solid LLM's but I recently jumped into fine tuning / modifying my own vision models for FPV drone detection (just for fun) and my daily workstation and it's 2080 just wasn't good enough. Even in 2025 it's cool how solid a setup dual 3090's still are. nvlink is an absolute must but it's incredibly powerful. I'm able to run the latest Mistral thinking models and rela…

I bought a 2nd 3090 2 years ago for like 800eur, still a good price even today I think. It's in my main workstation, and my idea was to always have Ollama running locally. The problem is that once I have a (large-ish) model running, all my VRAM is almost full and GPU struggles to do things like playing back a YouTube video. Lately I haven't used local AI much, also because I stopped using any coding AIs (as they wast…

I run my desktop environment on the iGPU and the AI stuff on the dGPUs.
Post reply on HN