Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

theverge.com

Post reply on HN
Print view View original

301–310 of 776 posts

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#301

Post by SideQuark »

ActionHank wrote: post #191
blixt wrote: post #156

Pretty interesting watching their tech explainers on YouTube about the changes in their AI solutions. Apparently they switched from CNNs to transformers for upscaling (with ray tracing support) if I understood correctly though for frame generation makes even more sense to me. 32 GB VRAM on the highest end GPU seems almost small after running LLMs with 128 GB RAM on the M3 Max, but the speed will most likely more than…

They are intentionally keeping the VRAM small on these cards to force people to buy their larger, more expensive offerings.

So you're saying more VRAM costs more money? What a novel idea!

Conversely, this means you can pay less if you need less.

Seems like a win all around.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#302

Post by benob »

holoduke wrote: post #296
jbarrow wrote: post #288

The increasing TDP trend is going crazy for the top-tier consumer cards: 3090 - 350W 3090 Ti - 450W 4090 - 450W 5090 - 575W 3x3090 (1050W) is less than 2x5090 (1150W), plus you get 72GB of VRAM instead of 64GB, if you can find a motherboard that supports 3 massive cards or good enough risers (apparently near impossible?).

Can you actually use multiple videocards easily with existing AI model tools?

Yes, multi-GPU on the same machine is pretty straightforward. For example ollama uses all GPUs out of the box. If you are into training, the huggingface ecosystem supports it and you can always go the manual route to put tensors on their own GPUs with toolkits like pytorch.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#303

Post by buildbot »

christkv wrote: post #153

575W TDP for the 5090. A buddy has 3x 4090 in a machine with a 32 core AMD cpu must be putting out close to 2000W of heat at peak if he switched to 5090. Uff

I have a very similar setup, 3x4090s. Depending on the model I’m training, the GPUs use anywhere from 100-400 watts, but don’t get much slower when power limited to say, 250w. So they could power limit the 5090s if they want and get pretty decent performance most likely.

The cat loves laying/basking on it when it’s putting out 1400w in 400w mode though, so I leave it turned up most of the time! (200w for the cpu)

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#304

Post by vonneumannstan »

blixt wrote: post #156

Pretty interesting watching their tech explainers on YouTube about the changes in their AI solutions. Apparently they switched from CNNs to transformers for upscaling (with ray tracing support) if I understood correctly though for frame generation makes even more sense to me. 32 GB VRAM on the highest end GPU seems almost small after running LLMs with 128 GB RAM on the M3 Max, but the speed will most likely more than…

If you want to run LLMs buy their H100/GB100/etc grade cards. There should be no expectation that consumer grade gaming cards will be optimal for ML use.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#305

Post by dijit »

Aardwolf wrote: post #273
dijit wrote: post #84

Earlier quoted context omitted.

From all the communication I’ve had with Nvidia, the prevailing sentiment was that the 4090 was an 8K card, that happened to be good for AI due to vram requirements from 8K gaming. However, I’m a AAA gamedev CTO and they might have been telling me what the card means to me .

Why does 8K gaming require more VRAM? I think the textures and geometry would have the same resolution (or is that not the case? but in 4K if you walk closer to the wall you'd want higher texture resolution as well anyway, if the graphics artists have made the assets at that resolution anyway) 8K screen resolution requires 132 megabytes of memory to store the pixels (for 32-bit color), that doesn't explain gigabytes…

You’re only thinking of the final raster framebuffer, there are multiple raster and shader stages. Increasing the native output has an nearly exponential increase in memory requirements.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#306

Post by vonneumannstan »

ActionHank wrote: post #191
blixt wrote: post #156

Pretty interesting watching their tech explainers on YouTube about the changes in their AI solutions. Apparently they switched from CNNs to transformers for upscaling (with ray tracing support) if I understood correctly though for frame generation makes even more sense to me. 32 GB VRAM on the highest end GPU seems almost small after running LLMs with 128 GB RAM on the M3 Max, but the speed will most likely more than…

They are intentionally keeping the VRAM small on these cards to force people to buy their larger, more expensive offerings.

No gamers need such high VRAM, if you're buying Gaming cards for ML work you're doing it wrong.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#307

Post by buildbot »

aurbano wrote: post #216
christkv wrote: post #153

575W TDP for the 5090. A buddy has 3x 4090 in a machine with a 32 core AMD cpu must be putting out close to 2000W of heat at peak if he switched to 5090. Uff

2kW is literally the output of my patio heater haha

They work as effective heaters! I haven’t used my (electric) heat all winter, I just use my training computer’s waste heat instead.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#308

Post by holoduke »

viraj_shah wrote: post #286
Kelteseth wrote: post #268

Earlier quoted context omitted.

Why not go for AMD? I just got a 7900XTX for 850 euros, it runs ollama or comfyUI via WSl2 quite nicely.

Do you have a good resource for learning what kinds of hardware can run what kinds of models locally? Benchmarks, etc? I'm also trying to tie together different hardware specs to model performance, whether that's training or inference. Like how does memory, VRAM, memory bandwidth, GPU cores, etc. all play into this. Know of any good resources? Oddly enough I might be best off asking an LLM.

To prevent custom implementations is recommended to get a Nvidia card. Minimum 3080 to get some results. But if you want video you should go for either 4090 or 5090. ComfUI is a popular interface which you can use for graphical stuff. Images and videos. Local text models I would recommend to use the Misty app. Basically a wrapper and downloader for various models. Tons of youtube videos on how to achieve stuff.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#309

Post by Kelteseth »

viraj_shah wrote: post #286
Kelteseth wrote: post #268

Earlier quoted context omitted.

Why not go for AMD? I just got a 7900XTX for 850 euros, it runs ollama or comfyUI via WSl2 quite nicely.

Do you have a good resource for learning what kinds of hardware can run what kinds of models locally? Benchmarks, etc? I'm also trying to tie together different hardware specs to model performance, whether that's training or inference. Like how does memory, VRAM, memory bandwidth, GPU cores, etc. all play into this. Know of any good resources? Oddly enough I might be best off asking an LLM.

I tested ollama with 7600XT at work and the mentioned 7900XTX. Both run fine with their VRAM limitations. So you can just switch between different quantization of llama 3.1 or the vast amount of different models at https://ollama.com/search

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#310

Post by Macha »

theandrewbailey wrote: post #260
pier25 wrote: post #228

AI is going to push the price closer to $3000. See what happened with crypto a couple of years back.

The ~2017 crypto rush told Nvidia how much people were willing to spend on GPUs, so they priced their next series (RTX 2000) much higher. 2020 came around, wash, rinse, repeat.

Note the 20 series bombed, largely because of the price hikes coupled with meager performance gains, so the initial plan was for the 30 series to be much cheaper. But then the 30 series scalping happened and they got a second go at re-anchoring what people thought of as reasonable GPU prices. Also they have diversified other options if gamers won't pay up, compared to just hoping that GPU-minable coins won over those that needed ASICs and the crypto market stayed hot. I can see nVidia being more willing to hurt their gaming market for AI than they ever were for crypto.

Also also, AMD has pretty much thrown in the towel at competing for high end gaming GPUs already.

Post reply on HN