Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

nicolasdickenmann.com

Post reply on HN
Print view View original

61–70 of 93 posts

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#61

Post by soulofmischief »

BoredPositron wrote: post #44
justincormack wrote: post #34

Earlier quoted context omitted.

My understanding is this was not enforceable in Europe, and maybe elsewhere

Code 43 was a worldwide thing before we found a workaround.

Yep, I do GPU passthrough to virtual machines because I would not let Windows touch my bare metal. You have to patch your ROM headers and hide the fact that you're in a VM from the OS.

So even as an end-user, a single person, I cannot naturally use my card how I please without significant technical investment. Imagine buying a $1000 piece of equipment and then being told what you can and can't do with it.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#62

Post by dgacmu »

arcanus wrote: post #43
numbers_guy wrote: post #38

A question that has been bugging me for a while is what will NVIDIA do with its HPC business? By HPC I mean clusters intended for non-AI related workloads. Are they going to cater to them separetely, or are they going to tell them to just emulate FP64?

Hopper had 60 TF FP64, Blackwell has 45 TF, and Rubin has 33 TF. It is pretty clear that Nvidia is sunsetting FP64 support, and they are selling a story that no serious computational scientist I know believes, namely that you can use low precision operations to emulate higher precision. See for example, https://www.theregister.com/2026/01/18/nvidia_fp64_emulation... It seems the emulation approach is slower, has more…

This is kind of amazing - I still have a bunch of Titan V's (2017-2018) that do 7 TF FP64. 8 years old and managing 1/4 of what Rubin does, and the numbers are probably closer if you divide by the power draw.

(Needless to say, the FP32 / int8 / etc. numbers are rather different.)

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#63

Post by dahart »

adrian_b wrote: post #55
dahart wrote: post #21

Earlier quoted context omitted.

> it is in theory possible to emulate FP64 using FP32 operations I’d say it’s better than theory, you can definitely use float2 pairs of fp32 floats to emulate higher precision. Quad precision using too, using float4. Here’s the code: https://andrewthall.com/papers/df64_qf128.pdf Also note it’s easy to emulate fp64 using entirely integer instructions. (As a fun exercise, I attempted both doubles and quads in GLSL: ht…

What is easy to do is to emulate FP128 with FP64 (double-double) or even FP256 with FP64. The reason is that the exponent range of FP64 is typically sufficient to avoid overflows and underflows in most applications. On the other hand, the exponent range of FP32 is insufficient for most scientific-technical computing. For an adequate exponent range, you must use either three FP32 per FP64, or two FP32 and an integer.…

Oh yes I forgot to mention it, you’re absolutely right, Thall’s method for df64 and qf128 gives you double/quad precision mantissa with single-precision exponent ranges, and the paper is clear about that.

FWIW, my own example (emulating doubles/quads with ints) gives the full exponent range with no wasted bits since I’m just emulating IEEE format directly.

Of course there are also bignum libraries that can do arbitrary precision. I guess one of the things I meant to convey but didn’t say directly is that using double precision isn’t export controlled, as one might interpret the top of thi thread, but a certain level of fp64 performance might be.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#64

Post by dahart »

wtallis wrote: post #27
botusaurus wrote: post #25

this article is so dumb. NVIDIA delivered what the market wanted - gamers dont need FP64, they dont waste silicon on it. now enterprise doesnt want FP64 anymore and they are reducing silicon for it too weird way to frame delivering exactly what the consumer wants as a big market segmentation fuck the user conspiracy

Your framing is what's backwards. NVIDIA artificially nerfed FP64 for a long time before they started making multiple specialized variants of their architectures. It's not a conspiracy theory; it's historical fact that they shipped the same die with drastically different levels of FP64 capability. In a very real way, consumers were paying for transistors they couldn't use, subsidizing the pro parts.

> consumers were paying for transistors they couldn’t use

This is Econ 101 these days. It’s cheaper to design and manufacture 1 product than 2. Many many products have features that are enabled for higher paying customers, from software to kitchen appliances to cars, and much much more.

The combined product design is also subsidizing some of the costs for everyone, so be careful what you wish for. If you could use all the transistors you have, you’d be paying more either way, either because design and production costs go up, or because you’re paying for the higher end model and being the one subsidizing the existence of the high end transistors other people don’t use.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#65

Post by wtallis »

zokier wrote: post #45
wtallis wrote: post #27

Earlier quoted context omitted.

Your framing is what's backwards. NVIDIA artificially nerfed FP64 for a long time before they started making multiple specialized variants of their architectures. It's not a conspiracy theory; it's historical fact that they shipped the same die with drastically different levels of FP64 capability. In a very real way, consumers were paying for transistors they couldn't use, subsidizing the pro parts.

> subsidizing the pro parts. You got this wrong way around. It's the high margin (pro) products subsidizing low margin (consumer) products.

In general, yes, but when consumer parts are spending silicon area on features they can't use, it is happening in the other direction too.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#66

Post by pjc50 »

wtallis wrote: post #2

It's amazing to step back and look at how much of NVIDIA's success has come from unforeseen directions. For their original purpose of making graphics chips, the consumer vs pro divide was all about CAD support and optional OpenGL features that games didn't use. Programmable shaders were added for the sake of graphics rendering needs, but ended up spawning the whole GPGPU concept, which NVIDIA reacted to very well wit…

The counter question is: why have AMD been so bad by comparison?

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#67

Post by [deleted] »

wtallis wrote: post #16
gdiamos wrote: post #8

I'm not sure why the article dismisses cost. Let's say X=10% of the GPU area (~75mm^2) is dedicated to FP32 SIMD units. Assume FP64 units are ~2-4x bigger. That would be 150-300mm^2, a huge amount of area that would increase the price per GPU. You may not agree with these assumptions. Feel free to change them. It is an overhead that is replicated per core. Why would gamers want to pay for any features they don't use?…

> Assume FP64 units are ~2-4x bigger. I'm pretty sure that's not a remotely fair assumption to make. We've seen architectures that can eg. do two FP32 operations or one FP64 operation with the same unit, with relatively low overhead compared to a pure FP32 architecture. That's pretty much how all integer math units work, and it's not hard to pull off for floating point. FP64 units don't have to be—and seldom have bee…

[deleted]

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#68

Post by fp64enjoyer »

adrian_b wrote: post #46

While implementing double-precision by double-single may be a solution in some cases, the article fails to mention the overflow/underflow problem, which is critical in scientific/technical computing (a.k.a. HPC). With the method from the article, the exponent range remains the same as in single precision, instead of being increased to that of double precision. There are a lot of applications for which such an exponen…

Yeah fair enough. The exponent of an FP32 has only 8 bits instead of 11 bits. I'll make an edit to make this explicit.

It's also fairly interesting how Nvidia handles this for the Ozaki scheme: https://docs.nvidia.com/cuda/cublas/#floating-point-emulatio.... They generally need to align all numbers in a matrix row to the maximum exponent (of a number in the row) but depending on scale difference of two numbers this might not be feasible without extending the number of mantissa bits significantly. So they dynamically (Dynamic Mantissa Control) decide if they use Ozaki's scheme or execute on native FP64 hardware. Or they let the user decide on the number of mantissa bits (Fixed Mantissa Control) which is faster but has no longer the guarantees for FP64 precision.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#69

Post by dboreham »

gbin wrote: post #35
cherryteastain wrote: post #33

Earlier quoted context omitted.

Nobody cared about deep learning back in 2007, when CUDA released. It wasn't until the 2012 AlexNet milestone that deep neural nets start to become en vogue again.

I clearly remember Cuda being made for HPC and scientific applications. They added actual operations for neural nets years after it was already a boom. Both instances were reactions, people already used graphics shaders for scientific purposes and cuda for neural nets, in both cases Nvidia was like oh cool money to be made.

Parallel computing goes back to the 1960s (at least). I've been involved in it since the 1980s. Generally you don't create an architecture and associated tooling for some specific application. The people creating the architecture only have a sketchy understanding of application areas and their needs. What you do is have a bright idea/pet peeve. Then you get someone to fund building that thing you imagined. Then marketing people scratch their heads as to who they might sell it to. It's at that point you observed "this thing was made for HPC, etc" because the marketing folks put out stories and material that said so. But really it wasn't. And as you note, it wasn't made for ML or AI either. That said in the 1980s we had "neural networks" as a potential target market for parallel processing chips so it's aways there as a possibility.

Re: 15 years of FP64 segmentation, and why the Blackwell Ultra breaks the pattern

#70

Post by adrian_b »

david-gpu wrote: post #56
adrian_b wrote: post #53

Earlier quoted context omitted.

NVIDIA could make 2 separate products, a GPU for gamers and a FP accelerator for HPC. Thus everybody would pay for what they want. The problem is that both NVIDIA and AMD do not want to make, like AMD did until a decade ago and NVIDIA stopped doing a few years earlier, a FP accelerator of reasonable size and which would be sold at a similar profit margin with their consumer GPUs. Instead of this, they want to sell on…

Throughout this article you have been voicing a desire for affordable and high-througput fp64 processors, blaming vendors for not building the product you desire at a price you are willing to pay. We hear you: your needs are not being met. Your use case is not profitable enough to justify paying the sky-high prices they now demand. In particular, because you don't need to run the workload 24/7. What alternatives have…

I think that you might have confused me with the author of the article.

American companies have a pronounced preference for business-to-business products, where they can sell large quantities in bulk and at very large profit margins that would not be accepted by small businesses or individual users, who spend their own money, instead of spending the money of an anonymous employer.

If that is the only way for them to be profitable, good for them. However such policies do not deserve respect. They demonstrate the inefficiencies in the management of these companies, which prevent them from competing efficiently in markets for low-margin commodity products.

From my experience, I am pretty certain that a smaller die version of the AMD "datacenter" GPUs could be made and it could be profitable, like such GPUs were a decade ago, when AMD was still making them. However today they no longer have any incentive to do such things, as they are content with selling a smaller number of units, but with much higher margins, and they do not feel any pressure to tighten their costs.

Fortunately at least in CPUs there has been a steady progress and AMD Zen 5 has been a great leap in floating-point throughput, exceeding the performance of older GPUs.

I am not blaming vendors for not building the product that I desire, but I am disappointed that years ago they have fooled me to waste time in porting applications to their products, which I bought instead of spending money for something else, but then they have discontinued such products, with no upgrade path.

Because I am old enough to remember what happened 15 to 20 years ago, I am annoyed about the hypocrisy of some discourses of the NVIDIA CEO, which have been repeated for several years after introducing CUDA, which were more or less equivalent with promises that the goal of NVIDIA is to put a "supercomputer" on the desk of everyone, only for him to pivot completely from these claims and remove FP64 from "consumer" GPUs, in order to be able to sell "enterprise" GPUs at inflated prices. Then soon this prompted AMD to imitate the same strategy.

Post reply on HN