Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

theverge.com

Post reply on HN
Print view View original

761–770 of 776 posts

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#761

Post by ryao »

Salgat wrote: post #514
numpy-thagoras wrote: post #389

Similar CUDA core counts for most SKUs compared to last gen (except in the 5090 vs. 4090 comparison). Similar clock speeds compared to the 40-series. The 5090 just has way more CUDA cores and uses proportionally more power compared to the 4090, when going by CUDA core comparisons and clock speed alone. All of the "massive gains" were comparing DLSS and other optimization strategies to standard hardware rendering. Som…

The 5090's core increase (30%) is actually underwhelming compared to the 3090->4090 increase (60% more), but the real game changer is the memory improvements, both in size and bandwidth.

They held back. Had they used 32Gbps GDDR7, they would have reached 2.0TB/sec memory bandwidth. 36Gbps GDDR7 would have let them reach 2.25TB/sec. The GB202 also reportedly has significantly more compute cores, TMUs, ROPs, tensor cores and RT cores than the 5090 uses:

https://www.techpowerup.com/gpu-specs/nvidia-gb202.g1072

Maybe there is a RTX 5090 Ti being held in reserve. They could potentially increase the compute on it by 13% and the memory bandwidth on it by 25% versus the 5090.

I wonder if anyone will try to solder 36Gbps GDDR7 chips onto a 5090 and then increase the memory clock manually.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#762

Post by janwas »

ryao wrote: post #753
janwas wrote: post #752

Earlier quoted context omitted.

I agree server CPUs are underprovisioned for memBW. Each core's share is 2-4 GB/s, whereas each could easily drive 10 GB/s (Intel) or 20+ (AMD). I also agree "some" (for example low-arithmetic-intensity) workloads will not benefit from a second L1 read port. But surely there are other workloads, right? If I want to issue one FMA per cycle, streaming from two arrays, doesn't that require maintaining two loads per cycl…

In an ideal situation where your arrays both fit in L1 cache and are in L1 cache, yes. However, in typical real world situations, you will not have them fit in L1 cache and then what will happen after the reads are issued will look like this: * Some time passes * Load 1 finishes * Some time passes * Load 2 finishes * FMA executes As we are doing FMA on arrays, this is presumably part of a tight loop. During the first…

OK, we agree that L1-resident workloads see a benefit. I also agree with your analysis if the loads actually come from memory.

Let's look at a more interesting case. We have a dataset bigger than L3. We touch a small part of it with one kernel. That is now in L1. Next we do a second kernel where each of the loads of this part are L1 hits. With two L1 ports, the latter is now twice as fast.

Even better, we can work on larger parts of the data such that it still fits in L2. Now, we're going to do the above for each L1-sized piece of the L2. Sure, the initial load from L2 isn't happening as fast as 2x64 bytes per cycle. But still, there are many L1 hits and I'm measuring effective FMA throughput that is _50 times_ as high as the memory bandwidth would allow when only streaming from memory. It's simply a matter of arranging for reuse to be possible, which admittedly does not work with single-pass algorithms like a checksum.

Do you find this reasoning convincing?

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#763

Post by [deleted] »

janwas wrote: post #762
ryao wrote: post #753

Earlier quoted context omitted.

In an ideal situation where your arrays both fit in L1 cache and are in L1 cache, yes. However, in typical real world situations, you will not have them fit in L1 cache and then what will happen after the reads are issued will look like this: * Some time passes * Load 1 finishes * Some time passes * Load 2 finishes * FMA executes As we are doing FMA on arrays, this is presumably part of a tight loop. During the first…

OK, we agree that L1-resident workloads see a benefit. I also agree with your analysis if the loads actually come from memory. Let's look at a more interesting case. We have a dataset bigger than L3. We touch a small part of it with one kernel. That is now in L1. Next we do a second kernel where each of the loads of this part are L1 hits. With two L1 ports, the latter is now twice as fast. Even better, we can work on…

[deleted]

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#764

Post by ryao »

janwas wrote: post #762
ryao wrote: post #753

Earlier quoted context omitted.

In an ideal situation where your arrays both fit in L1 cache and are in L1 cache, yes. However, in typical real world situations, you will not have them fit in L1 cache and then what will happen after the reads are issued will look like this: * Some time passes * Load 1 finishes * Some time passes * Load 2 finishes * FMA executes As we are doing FMA on arrays, this is presumably part of a tight loop. During the first…

OK, we agree that L1-resident workloads see a benefit. I also agree with your analysis if the loads actually come from memory. Let's look at a more interesting case. We have a dataset bigger than L3. We touch a small part of it with one kernel. That is now in L1. Next we do a second kernel where each of the loads of this part are L1 hits. With two L1 ports, the latter is now twice as fast. Even better, we can work on…

The purpose of L1 cache is to avoid long round trips to memory. What you describe is L1 cache doing what it is intended to do. Unfortunately, I do not have your code, so it is not clear to me that it benefits from doing 2 AVX-512 loads per cycle.

I am also not sure what CPU this is. On recent AMD processors at the very least, it should be impossible to get FMA throughput that is 50 times higher from L1 cache bandwidth than system memory bandwidth. On the Ryzen 7 9800X3D for example, a single core is limited to about 64GB/sec. 50 times more would be 3.2TB/sec, which is ~5 times faster than possible to load from L1 cache even with 2 AVX-512 loads per cycle.

I wonder if you are describing some sort of GEMM routine, which is a place where 50 times more FMA throughput is possible if you do things in a clever way. GEMM is somewhat weird, since without copying to force things into L1 cache, it does not run at full speed, and memory bandwidth from RAM is always below peak memory bandwidth, even without the memcpy() trick to force things into L1 cache. That excludes the case where you stuff GEMV in GEMM, where it does become memory bandwidth bound.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#765

Post by janwas »

ryao wrote: post #764
janwas wrote: post #762

Earlier quoted context omitted.

OK, we agree that L1-resident workloads see a benefit. I also agree with your analysis if the loads actually come from memory. Let's look at a more interesting case. We have a dataset bigger than L3. We touch a small part of it with one kernel. That is now in L1. Next we do a second kernel where each of the loads of this part are L1 hits. With two L1 ports, the latter is now twice as fast. Even better, we can work on…

The purpose of L1 cache is to avoid long round trips to memory. What you describe is L1 cache doing what it is intended to do. Unfortunately, I do not have your code, so it is not clear to me that it benefits from doing 2 AVX-512 loads per cycle. I am also not sure what CPU this is. On recent AMD processors at the very least, it should be impossible to get FMA throughput that is 50 times higher from L1 cache bandwidt…

The code is unfortunately not (yet) open source. The CPU with 50x is an SKX Gold, and it is similar for Zen4. I compute this ratio as #FMA * 4 / total system memory bandwidth. We are indeed not fully memBW bound :)

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#766

Post by mczarnek »

dagmx wrote: post #48
yakaccount4 wrote: post #36

Earlier quoted context omitted.

3 Generated frames sounds like a lot of lag, probably a sickening amount for many games. The magic of "blackwell flip metering" isn't quite described yet.

It’s 3 extrapolated frames not interpolated. So would be reduced lag at the expense of greater pop-in. There’s also the new reflex 2 which uses reprojection based on mouse motion to generate frames that should also help, but likely has the same drawback.

I'm hearing it's actually interpolated frames, not extrapolated.. I would like better confirmation of that though.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#767

Post by mczarnek »

vel0city wrote: post #184
perching_aix wrote: post #120

Earlier quoted context omitted.

> It’s 3 extrapolated frames not interpolated. Do you have a source for this? Doesn't sound like a very good idea. Nor do I think there's additional latency mind you, but not because it's not interpolation.

Interpolation means you have frame 1 and frame 2, now compute the interstitial steps between these two. Extrapolation means you have frame 1, and sometime in the future you'll get a frame 2. But until then, take the training data and the current frame and "guess" what the next few frames will be. Interpolation requires you to have the final state between the added frames, extrapolation means you don't yet know what t…

What if you used the last two frames instead of just one.. then it's pretty much same thing

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#768

Post by mczarnek »

ryao wrote: post #756
perching_aix wrote: post #572

Earlier quoted context omitted.

This doesn't imply "extrapolation" instead of interpolation.

That does imply extrapolation since interpolation requires 2 frames and they are only using 1.

It does imply extrapolation but I'm hearing doesn't actually provide it and they actually interpolate.

Which fits with them having same latency.. have to wait for that next frame.

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#769

Post by kcb »

ryao wrote: post #757
kcb wrote: post #140

Earlier quoted context omitted.

In all those historical cases the second tier card was a cut down version of the top tier one. Now the 4080 and 5080 are a different chip and there's a gulf of a performance gap between them and the top tier. That's the issue I am highlighting, the 5080 is half a 5090, in the past a 3080 was only 10% off a 3090 performance wise.

It was not actually. The last time this was the case was Maxwell: https://www.techpowerup.com/gpu-specs/nvidia-gm200.g772 Beginning with Pascal, Nvidia’s top GPU was not available in consumer graphics cards: https://www.techpowerup.com/gpu-specs/nvidia-gp100.g792 Turing was a bit weird since instead of having a TU100, they instead had Volta’s GV100: https://www.techpowerup.com/gpu-specs/nvidia-tu102.g813 https://www.…

This is not relevant to what is being discussed. I clearly mean top tier consumer GPU.

3080/3090 - Same die

2080 ti/Titan RTX - Same die

1080 ti/Titan Xp - Same die

980 ti/Titan X - Same die

780/Titan - Same die

670/680 - Same die

570/580 - Same die

470/480 - Same die

Re: Nvidia announces next-gen RTX 5090 and RTX 5080 GPUs

#770

Post by menaerus »

ryao wrote: post #750
menaerus wrote: post #748

Earlier quoted context omitted.

> They do not. The chip has 105MB L3 cache and they tested on 128MB of memory. This exceeds the size of L3 cache and thus, it is not a proper test of L3 cache. First, you claimed that there was no L3 BW test. Now, I am not even sure if you're trolling me or lacking knowledge or what at this point? Please do tell what you consider a "proper test of L3 cache"? And why do you consider their test invalid? I am curious be…

> First, you claimed that there was no L3 BW test. I claimed that they did not provide figures for L3 cache bandwidth. They did not. > Now, I am not even sure if you're trolling me or lacking knowledge or what at this point? You should be grateful that a professional is taking time out of his day to explain things that you do not understand. > Please do tell what you consider a "proper test of L3 cache"? And why do y…

You're going a long way not to address reasonably simple questions I had. You're very combative for no obvious reason - I think I had my arguments laid out in the most objective form I could but unfortunately you seem to be very triggered by those especially by the logical concerns I raised. You are in the wrong here simply because you're assuming that all the experience you have is representative of all other experience people have in this industry. There are much larger challenges than designing a filesystem, you know. No need to be so vain.

Not only you're ending up being very disrespectful but you're also pulling out the appeal to authority argument. Also something Brendan Gregg did on me here at HN meaning that no experience can substitute the amount of ego in some guys.

FWIW you can be let assured that you can't match my experience but that's not the argument I would ever pull off. I like to be proved wrong. This is a way I learn new things. BTW I designed my first CPU 15 years ago but during the career I learned to put my ego aside, discuss objectively, think critically and learn on my own reasoning mistakes from other people. Many of these points you are obviously lacking so this is a waste of time for me - I see no way to drive this discussion further but thanks anyway.

Post reply on HN