Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

nicolasdickenmann.com

Post reply on HN
Print view View original

51–60 of 93 posts

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#51

Post by adrian_b »

jcranmer wrote: post #19
gdiamos wrote: post #8

I'm not sure why the article dismisses cost. Let's say X=10% of the GPU area (~75mm^2) is dedicated to FP32 SIMD units. Assume FP64 units are ~2-4x bigger. That would be 150-300mm^2, a huge amount of area that would increase the price per GPU. You may not agree with these assumptions. Feel free to change them. It is an overhead that is replicated per core. Why would gamers want to pay for any features they don't use?…

> Assume FP64 units are ~2-4x bigger. I'm not a hardware guy, but an explanation I've seen from someone who is says that it's not much extra hardware to add to a 2×f32 FMA unit the capability to do 1×f64. You already have all of the per-bit logic, you mostly just need to add an extra control line to make a few carries propagate. So the size overhead of adding FP64 to the SIMD units is more like 10-50%, not 100-300%.

Most of the logic can be reused, but the FP64 multiplier is up to 4 times larger. Also some shifters are up to 2 times larger (because they need more stages, even if they shift the same number of bits). Small size increases occur in other blocks.

Even so, the multipliers and shifters occupy only a small fraction of the total area, a fraction that is smaller then implied by their number of gates, because they have very regular layouts.

A reduction from the ideal 1:2 FP64/FP32 throughput to 1:4 or in the worst case to 1:8 should be enough to make negligible the additional cost of supporting FP64, while still keeping the throughput of a GPU competitive with a CPU.

The current NVIDIA and AMD GPUs cannot compete in FP64 performance per dollar or per watt with Zen 5 Ryzen 9 CPUs. Only Intel B580 is better in FP64 performance per dollar than any CPU, though its total performance is exceeded by CPUs like 9950X.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#52

Post by nsajko »

adrian_b wrote: post #46

While implementing double-precision by double-single may be a solution in some cases, the article fails to mention the overflow/underflow problem, which is critical in scientific/technical computing (a.k.a. HPC). With the method from the article, the exponent range remains the same as in single precision, instead of being increased to that of double precision. There are a lot of applications for which such an exponen…

Yeah, double-word floating-point loses many of the desirable properties of the usual floating-point.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#53

Post by adrian_b »

wmf wrote: post #10
gdiamos wrote: post #8

I'm not sure why the article dismisses cost. Let's say X=10% of the GPU area (~75mm^2) is dedicated to FP32 SIMD units. Assume FP64 units are ~2-4x bigger. That would be 150-300mm^2, a huge amount of area that would increase the price per GPU. You may not agree with these assumptions. Feel free to change them. It is an overhead that is replicated per core. Why would gamers want to pay for any features they don't use?…

Why would gamers want to pay for any features they don't use? Obviously they don't want to. Now flip it around and ask why HPC people would want to force gamers to pay for something that benefits the HPC people... Suddenly the blog post makes perfect sense.

NVIDIA could make 2 separate products, a GPU for gamers and a FP accelerator for HPC.

Thus everybody would pay for what they want.

The problem is that both NVIDIA and AMD do not want to make, like AMD did until a decade ago and NVIDIA stopped doing a few years earlier, a FP accelerator of reasonable size and which would be sold at a similar profit margin with their consumer GPUs.

Instead of this, they want to sell only very big FP accelerators and at huge profit margins, preferably at 5-digit prices.

This makes impossible for small businesses and individual users to use such FP accelerators.

Those are accessible only for big companies, who can buy them in bulk and negotiate lower prices than the retail prices, and who will also be able to keep them busy for close to 24/7, in order to be able to amortize the excessive profit margins of the "datacenter" GPU vendors.

One decade and a half ago, the market segmentation was not yet excessive, so I was happy to buy "professional" GPUs, with unlocked FP64 throughput, at a price about twice greater in comparison with consumer GPUs.

Nowadays, I can no longer afford such a thing, because the similar GPUs are no longer 2 times more expensive, but 20 to 50 times more expensive.

So during the last 2 decades, first I shifted much of my computations from CPUs to GPUs, but then I had to shift them back to CPUs, because there are no upgrades for my old GPUs, any newer GPU being slower, not faster.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#54

Post by KeplerBoy »

adrian_b wrote: post #48
imtringued wrote: post #30

Earlier quoted context omitted.

Nvidia started their GPGPU adventure by acquiring a physics engine and porting it over to run on their GPUs. Supporting linear algebra operations was pretty much the goal from the start.

They were also full of lies when they have started their GPGPU adventure (like also today). For a few years they have repeated continuously how GPGPU can provide about 100 times more speed than CPUs. This has always been false. GPUs are really much faster, but their performance per watt has oscillated during most of the time around 3 times and sometimes up to 4 times greater in comparison with CPUs. This is impressiv…

GPGPU is doing better than ever.

Sure FP64 is a problem and not always available in the capacity people would like it to be, but there are a lot of things you can do just fine with FP32 and all of that research and engineering absolutely is done on GPU.

The AI-craze also made all of it much more accessible. You don't need advanced C++ knowledge anymore to write and run a CUDA project anymore. You can just take Pytorch, JAX, CuPy or whatnot and accelerate your numpy code by an order of magnitude or two. Basically everyone in STEM is using Python these days and the scientific stack works beautifully with nvidia GPUs. Guess which chip maker will benefit if any of that research turns out to be a breakout success in need of more compute?

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#55

Post by adrian_b »

dahart wrote: post #21
fp64enjoyer wrote: post #12

Earlier quoted context omitted.

This is so interesting, especially given that it is in theory possible to emulate FP64 using FP32 operations. I do think though that Nvidia generally didn't see much need for more FP64 in consumer GPUs since they wrote in the Ampere (RTX3090) white paper: "The small number of FP64 hardware units are included to ensure any programs with FP64 code operate correctly, including FP64 Tensor Core code." I'll try adding an…

> it is in theory possible to emulate FP64 using FP32 operations I’d say it’s better than theory, you can definitely use float2 pairs of fp32 floats to emulate higher precision. Quad precision using too, using float4. Here’s the code: https://andrewthall.com/papers/df64_qf128.pdf Also note it’s easy to emulate fp64 using entirely integer instructions. (As a fun exercise, I attempted both doubles and quads in GLSL: ht…

What is easy to do is to emulate FP128 with FP64 (double-double) or even FP256 with FP64.

The reason is that the exponent range of FP64 is typically sufficient to avoid overflows and underflows in most applications.

On the other hand, the exponent range of FP32 is insufficient for most scientific-technical computing.

For an adequate exponent range, you must use either three FP32 per FP64, or two FP32 and an integer. In this case the emulation becomes significantly slower than the simplistic double-single emulation.

With the simpler double-single emulation, you cannot expect to just plug it in most engineering applications, e.g. SPICE for electronic circuit simulation, and see that the application works. Some applications could be painstakingly modified to work with such an implementation, but that is not normally an option.

So to be interchangeable with the use of standard FP64 you really must also emulate the exponent range, at the price of much slower emulation.

I did this at some point in the past, but today it makes no sense in comparison with the available alternatives.

Today, the best FP64 performance per dollar by far, is achieved with Ryzen 9950X or Ryzen 9900X, in combination with Inter Battlemage B580 GPUs.

When money does not matter, you can use AMD Epyc in combination with AMD "datacenter" GPUs, which would achieve much better performance per watt, but the performance per dollar would be abysmally low.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#56

Post by david-gpu »

adrian_b wrote: post #53
wmf wrote: post #10

Earlier quoted context omitted.

Why would gamers want to pay for any features they don't use? Obviously they don't want to. Now flip it around and ask why HPC people would want to force gamers to pay for something that benefits the HPC people... Suddenly the blog post makes perfect sense.

NVIDIA could make 2 separate products, a GPU for gamers and a FP accelerator for HPC. Thus everybody would pay for what they want. The problem is that both NVIDIA and AMD do not want to make, like AMD did until a decade ago and NVIDIA stopped doing a few years earlier, a FP accelerator of reasonable size and which would be sold at a similar profit margin with their consumer GPUs. Instead of this, they want to sell on…

Throughout this article you have been voicing a desire for affordable and high-througput fp64 processors, blaming vendors for not building the product you desire at a price you are willing to pay.

We hear you: your needs are not being met. Your use case is not profitable enough to justify paying the sky-high prices they now demand. In particular, because you don't need to run the workload 24/7.

What alternatives have you looked into? For example, Blackwell nodes are available from the likes of AWS.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#57

Post by [deleted] »

zokier wrote: post #45
wtallis wrote: post #27

Earlier quoted context omitted.

Your framing is what's backwards. NVIDIA artificially nerfed FP64 for a long time before they started making multiple specialized variants of their architectures. It's not a conspiracy theory; it's historical fact that they shipped the same die with drastically different levels of FP64 capability. In a very real way, consumers were paying for transistors they couldn't use, subsidizing the pro parts.

> subsidizing the pro parts. You got this wrong way around. It's the high margin (pro) products subsidizing low margin (consumer) products.

[deleted]

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#58

Post by brookst »

wtallis wrote: post #2

It's amazing to step back and look at how much of NVIDIA's success has come from unforeseen directions. For their original purpose of making graphics chips, the consumer vs pro divide was all about CAD support and optional OpenGL features that games didn't use. Programmable shaders were added for the sake of graphics rendering needs, but ended up spawning the whole GPGPU concept, which NVIDIA reacted to very well wit…

Maybe some luck. But there’s also a principle that if you optimize the hell out of something and follow customer demand, there’s money to be made.

Nvidia did a great job of avoiding the “oh we’re not in that market” trap that sunk Intel (phones, GPUs, efficient CPUs). Where Intel was too big and profitable to cultivate adjacent markets, Nvidia did everything they could to serve them and increase demand.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#59

Post by brookst »

fooblaster wrote: post #24
gdiamos wrote: post #9

Earlier quoted context omitted.

Most people don't appreciate how many dead end applications NVIDIA explored before finding deep learning. It took a very long time, and it wasn't luck.

It was definitely luck, greg. And Nvidia didn't invent deep learning, deep learning found nvidias investment in CUDA.

So it could just as easily have been Intel or AMD, despite them not having CUDA or any interest in that market? Pure luck that the one large company that invested to support a market reaped most of the benefits?
Post reply on HN