Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

blogs.nvidia.com

Post reply on HN
Print view View original

171–180 of 347 posts

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#171

Post by monocasa »

adev_ wrote: post #154
adev_ wrote: post #152

Earlier quoted context omitted.

> Do you work for AMD I do not. And I use NVidia hardware regularly for GPGPU. But I hate fanboyism. > NVIDA open-source their CUDA implementation to the LLVM project 5 years ago Correction: Google developped an internal CUDA implementation for their own need based on LLVM that Nvidia barely supported it for their own need afterwards. Nothing is "stable" nor "branded" in this work.... Consequently, 99% of public Open…

> That you can't compile CUDA to AMD GPUs isn't NVIDIA's fault, it's AMD, for deciding to pursue OpenCL first, then HSA, and now HIP. Using a branded & under patent concurrent proprietary technology and copying its API for your own implementation is Maddness that will lead you for sure in front of a court. It seems that even Google understood that the hard way ( https://en.wikipedia.org/wiki/Google_v._Oracle_America…

Google v Oracle is still unsettled.

Most other legal precedent was that it was fine to clone an API.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#172

Post by pjmlp »

adev_ wrote: post #134
pjmlp wrote: post #93

Earlier quoted context omitted.

NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent, while hip only works on Linux. As for Python, guess who is on the forefront of GPU Programming with Python, https://www.nvi…

> NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. I don't bite this argument. Nvidia made close to no effort to support OpenCL and promoted their own technology CUDA. Even in 2020, OpenCL support for Nvidia hardware is close to nonexistent. When the main actor of the market d…

OpenCL isn't also supported on Android, where Google pushes Renderscript instead, their own C99 dialect, yet I don't see any uprising against Google.

If the 139 member companies (taking NVidia out) listed here aren't able to provide the same quality in hardware, programming language and eco-system improvements against those from NVidia, and vote for a NVidia employee as chairman, then they deserve what they get.

https://www.khronos.org/members/list

It is so easy to find a villain instead of acknowledging failure.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#173

Post by DonHopkins »

bredren wrote: post #86
epmaybe wrote: post #65

Earlier quoted context omitted.

Did you see his video before the keynote? I had a good chuckle. https://www.youtube.com/watch?v=So7TNRhIYJ8

That’s a fancy kitchen.

And he sure has a LOT of really nice spatulas!

His Spatula City Frequent Buyer Card must have a lot of stamps on it.

https://www.youtube.com/watch?v=2XbCWmY0eqY

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#174

Post by enos_feedler »

ethbro wrote: post #69
bitL wrote: post #60

Earlier quoted context omitted.

> why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions I think there is that latency vs bandwidth trade-off where CPU likes lower latency and GPU higher bandwidth, but you can't achieve the same with a single chip.

I guess this is fundamentally a homogeneous vs heterogeneous ISA question. I.e. is your ISA intended to operate one chip, or multiple cooperative chips / complexes?

Does it make sense for a hardware ISA to express cooperation between chips? I would think HW ISA is meant to control it's local microarchitecture. I could see a virutal ISA or compiler IR built with a multi chip view.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#175

Post by pjmlp »

not2b wrote: post #169
fluffything wrote: post #94

Earlier quoted context omitted.

> They make good hardware, but there's a lot of lock-in, and not a lot of transparency. This sounds like you'd like NVIDIA to open-source all their software. I see this type of request a lot, but I don't see it happening. NVIDIA's main competitive advantage over AMD and Intel is its software stack. AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because wh…

It would suffice for NVIDIA to open-source enough specifications and perhaps some subset of core software to enable others to build high quality open source (or even proprietary) software that targets NVIDIA's architecture. They can't hire every programmer in the world; if other programmers can build high-performance software that takes advantage of their platform, that increases the value of their hardware. Your com…

Anyone is free to target PTX and do their own compiler on top.

In fact, given that it is there since version 3, there are compilers available for almost all major programing languages, including managed ones.

While OpenCL is a C world, and almost no one cares about the C++ extensions and even less vendors care about SPIR-V.

Also the community doesn't seem to be bothered that for a long time, the only SYCL implementation was a commercial one from CodePlay, trying to extend their compilers outside the console market.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#176

Post by pjmlp »

taurath wrote: post #148
fluffything wrote: post #94

Earlier quoted context omitted.

> They make good hardware, but there's a lot of lock-in, and not a lot of transparency. This sounds like you'd like NVIDIA to open-source all their software. I see this type of request a lot, but I don't see it happening. NVIDIA's main competitive advantage over AMD and Intel is its software stack. AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because wh…

This has literally been a back and forth argument since a 100 point post on slashdot was a groundbreaking event. I don't see it changing any time soon - honestly if anything on tech forums this argument frequently overshadows just how well NVIDIA is doing.

It is just like game forums as well.

The culture here and on those forums couldn't be further apart.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#177

Post by adev_ »

monocasa wrote: post #171
adev_ wrote: post #154

Earlier quoted context omitted.

> That you can't compile CUDA to AMD GPUs isn't NVIDIA's fault, it's AMD, for deciding to pursue OpenCL first, then HSA, and now HIP. Using a branded & under patent concurrent proprietary technology and copying its API for your own implementation is Maddness that will lead you for sure in front of a court. It seems that even Google understood that the hard way ( https://en.wikipedia.org/wiki/Google_v._Oracle_America…

Google v Oracle is still unsettled. Most other legal precedent was that it was fine to clone an API.

> Most other legal precedent was that it was fine to clone an API.

CUDA is more than an API. It is a technology under copyright and very likely patented too. Even the API itself contains multiple reference to "CUDA" in function calls and variable name.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#178

Post by adev_ »

fluffything wrote: post #165
adev_ wrote: post #152

Earlier quoted context omitted.

> Do you work for AMD I do not. And I use NVidia hardware regularly for GPGPU. But I hate fanboyism. > NVIDA open-source their CUDA implementation to the LLVM project 5 years ago Correction: Google developped an internal CUDA implementation for their own need based on LLVM that Nvidia barely supported it for their own need afterwards. Nothing is "stable" nor "branded" in this work.... Consequently, 99% of public Open…

> Correction: Google developped an internal CUDA implementation for their own need based on LLVM that Nvidia barely supported it for their own need afterwards. This is widely inaccurate. While Google did developed a PTX backend for LLVM, the student that worked on that as part of a GSOC got later hired by NVIDIA, and ended up contributing the current NVPTX backend that clang uses today. The PTX backend that Google co…

> While Google did developed a PTX backend for LLVM, the student that worked on that as part of a GSOC got later hired by NVIDIA, and ended up contributing the current NVPTX backend that clang uses today.

You more or less reformalized what I said. It might become used one day behind a proprietary blob, rebranded blob of NVidia, but fact is that today, close to nobody use it for production in the wild and it is not even supported officially.

> This is false. The NV part of the backend name (NVPTX) literally brands this backend as NVIDIAs PTX backend.

It does not mean it's stable or used. I do not now a single major GPGPU software in existence that ever used it in an official distribution. Like I said.

> CUDA Fortran just fine

CUDA fortran, yes you said it, CUDA fortran. The rest is OpenACC.

> The Parallel STL work actually originated with the GCC parallel STL, the Intel TBB, and NVIDIA Thrust libraries

My apologies for that. I was ignoring this precedent work.

> AMD is nowhere to be found in this type of work.

I do not think I ever said anything about AMD.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#179

Post by boulos »

QuixoticQuibit wrote: post #97
rrss wrote: post #87

Earlier quoted context omitted.

'brain floating point' is also bad, but no one cares because it's just bfloat16. If this becomes popular, it will just be tf32 or tfloat32 or something.

I never knew what the “b” in “bfloat” was in all these new DL chips… until today. Man that’s bad.

Disclosure: I work on Google Cloud.

I wouldn't worry about it. Looks like the Anandtech article [1] doesn't either :)

> bfloat16, a format popularized by Intel

[1] https://www.anandtech.com/show/15801/nvidia-announces-ampere...

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#180

Post by jdsully »

andromeduck wrote: post #166
xnyan wrote: post #51

Earlier quoted context omitted.

>That's why Intel realized that fab technology was the true differentiator. But now the situation is completely reversed. Intel has faced all kinds of problems, costs, and delays due ultimately to the fact that they made a bad choice on their chip architecture but were forced to make it work because they invested so much in the fab. What TSMC is fabbing for nvidia is working out really well, and if it was not nvidia…

I think it's the other way around. The architecture was being limited by their fabs ability to yeild large chips and in the absence of any CPU perf pressure from AMD the natural push would lean more towards increacong graphics performance in order to push more pixels. As in I think Intel probably had the same yeild issues as everyone else ~10-32 nm but only Intel had the high margins small chip volume to make it prof…

The architecture is definitely far ahead of anyone else. When you look at Intel chips still being competitive despite manufacturing being a generation behind and with 1/6th the cache per core.

I'm an AMD shareholder and my biggest fear is Intel figuring out their manufacturing.

Post reply on HN