Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

blogs.nvidia.com

Post reply on HN
Print view View original

251–260 of 347 posts

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#251

Post by fluffything »

godelski wrote: post #210
blopeur wrote: post #4

Earlier quoted context omitted.

I am suspecting that this is specifically targeted at the HPC market as the lack of FP64 has always been a hindrance to HPC deployment. You have to remember that the HPC market is $35B today. HPE makes $3B a year alone from that, maybe more with Cray acquisition. So it's no surprise that NVIDIA wants to position themselves on that market. Plus, you have too look at the long game with MLX acquisition ( heavy player in…

Remember too that AMD and Intel won the contracts for Aurora, Frontier, and El Cap (the three exascale machines for the DOE). I imagine MLX is a big part of getting the next contracts as well as seeing that a lot of these projects are IO bound, not compute. If you can bring supercomputing like abilities to datacenters or AI labs, that'd be a huge advantage. If you could easily split a huge model across 64 GPUs and tr…

> Remember too that AMD and Intel won the contracts for Aurora, Frontier, and El Cap (the three exascale machines for the DOE).

This isn't very surprising I think. AMD cards often have higher FLOPs than NVIDIA's, and I can imagine that they run HPL really well.

I can't wait to try these systems. I want to see what the OpenMP performance there looks like for normal applications.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#252

Post by csdreamer7 »

Polylactic_acid wrote: post #236
adev_ wrote: post #121

Earlier quoted context omitted.

> AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because what good is that if you can't program it?. Correction: Nobody will be able to use the AMD hardware (outside of computer graphics) because everybody has been locked-in with CUDA on Nvidia. They can not even change even if they want to: it is pure madness to reprogram an entire GPGPU software stack e…

Amd doesn't even have software for GPGPU on some of their cards. I have an rx5700xt and I cant use it for anything but gaming because ROCm doesn't support navi cards, a whole year after its release.

As a 5700 owner, I agree.

It gets even worse. There was recently a regression in the 5.4, 5.5, and 5.6 kernels that hit me hard for a week or so on Manjaro last month. System just decided to lock up or restart. Thought the graphics card had died when it happened once on Windows. Working fine now-these drivers have been out for 10 months now.

Even worse, AMD has locked down the releases of some of their 'GPUOpen' software.

https://www.phoronix.com/scan.php?page=news_item&px=Radeon-R...

https://www.phoronix.com/scan.php?page=news_item&px=GPUOpen-...

I did not expect the second one to be open source; just not on their GPUOpen website.

I did expect the first one to 'stay' open source. Not to be made proprietary on their 'GPUOpen' website.

I am definitely keeping an eye on Intel graphics now.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#253

Post by fluffything »

__alexs wrote: post #80
chupasaurus wrote: post #75

Earlier quoted context omitted.

Epic demo was running on PS5 so no NVIDIA involvement there.

Oh yes, I forgot they've entirely lost this console generation.

And the entire HPC exa-flop super computers as well.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#254

Post by m463 »

slaymaker1907 wrote: post #116
ethbro wrote: post #47

Earlier quoted context omitted.

Hypothetically, from an ISA perspective, why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions? (as in, way more fully than MMX / SSE / AVX) Naive question, because I literally don't know the link between CPU instruction stream and GPGPU instruction stream. But it seems like there would be an opportunity to seize the higher (open) ground at the ISA level, and then force Nvidia to implement…

GPUs have gone far beyond just SIMD these days. To effectively program a GPU, you need to program it like a GPU, not a CPU. In particular, while most people are aware that GPUs don't like branching at a high level, branching can actually be fine as long as each block (small group of processors in the GPU) take the same branch. Block 1 taking the branch while block 2 not taking the branch will have little impact on pe…

learning about SIMT helped my understanding of the differences.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#255

Post by GauntletWizard »

m463 wrote: post #248
chrischen wrote: post #193

Earlier quoted context omitted.

And yet we grant patents so liberally, giving a windfall to the first person who files.

Remember the long view - patents cause people to hurry to publish and share their ideas publically. Why shouldn't they be granted liberally. In a few years, the temporary monopoly falls away and the benefit passes to everyone. I think they should work to make them even cheaper and easier to file.

20 years is a long time. For some fields, it is perfectly reasonable, but 20 year patents on many recent CS inventions would have significantly bottlenecked development of the industry - Look at how much mess was created by the JPEG patents, for example, and similar problems have existed for every other not-explicitly-libre A/V codec.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#256

Post by sl1ck731 »

m463 wrote: post #248
chrischen wrote: post #193

Earlier quoted context omitted.

And yet we grant patents so liberally, giving a windfall to the first person who files.

Remember the long view - patents cause people to hurry to publish and share their ideas publically. Why shouldn't they be granted liberally. In a few years, the temporary monopoly falls away and the benefit passes to everyone. I think they should work to make them even cheaper and easier to file.

I believe a few years is 20 years though. I haven't thought of patents from this perspective but 20 years is still a long time (and large chunk of your working years) to benefit from something.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#258

Post by Jhsto »

fluffything wrote: post #226
Reelin wrote: post #209

Earlier quoted context omitted.

> any bystander You aren't seriously implying than any bystander is capable of extending LLVM to map CUDA to SPIR-V? What percentage of present day gainfully employed software engineers do you suppose even has the background knowledge? How many hours do you suppose the work would require?

If LLVM has a SPIRV backend, probably very little. For a proof of concept, a bachelor CS thesis would probably do. Clang already has a CUDA parser, and all the code to lower CUDA specific constructs to LLVM-IR, some of which are specific for the PTX backend. If you try to compile CUDA code for a different target, like SPIRV, you'll probably get some errors saying that some of the LLVM-IR instructions generated by cla…

FWIW, there already exists LLVM to SPIR-V compiler: https://github.com/KhronosGroup/SPIRV-LLVM-Translator

Alas, this supports SPIR-V to 1.1.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#259

Post by XCSme »

tbenst wrote: post #157

Important to remember the half-precision tensorcore misrepresentations where the 8x improvement over fp32 claimed on Imagenet with tensorcores (V100) was actually only 1.2-2x [1,2]. Furthermore, there are major precision issues with network architectures like variational autoencoders and many others. We use V100s for Richardson-Lucy like deconvolutions for example, where we have near-exact photon counts up to 10,000…

I am not that much into ML, just fiddled with it a bit, is tf32=fp16?

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#260

Post by lumost »

diffrinse wrote: post #240
lumost wrote: post #231

Earlier quoted context omitted.

Part of Nvidia's advantage comes from building the hardware and software side by side. No one was seriously tackling GPGPU until Nvidia created Cuda, and if you look at the rest of the graphics stack Nvidia is the one driving the big innovations. GPUs are sufficiently specialized in both interface and problem domain that GPU enhanced software is unlikely to appear without a large vendor driving development, and it wo…

Would you agree that that your observations somewhat imply that a competitive free market is not a fit for all governable domains (and don't mistake governable for government there, we're still talking about shepherding of innovation)?

I think you would need to take a fine tooth comb to the definitions here. I could see a few different options emerge for non-Nvidia software including

- Cloud providers wishing to provide lower CapEx solutions in exchange for increased OpeX and margin. - Large Nvidia customers forming a foundation to shepherd Open implementations of common technology components

From a free market perspective both forms of transaction would be viable and incentivized, but neither option necessarily leads to an open implementation.

Post reply on HN