Live data from Hacker News

Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

blogs.nvidia.com

271–280 of 347 posts

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#271
post #18

Earlier quoted context omitted.

I don't like that people downvoted you for asking a question. If someone thinks the question is stupid or not doesn't mean that a downvote is warranted. (nor an upvote, answer the question and move on.) To answer though; it's just a coincidence, as you might already know Nvidia uses famous scientists (especially in the field of electricity) as the names of their microarchitectures. * Volta (Alessandro Volta, inventor…

And even before that there was: * GeForce (for Andrea Geforce, the first to use the color electric green) * Riva (for Jose Riva, the discoverer that you can use TNT to generate electricity.)

Who is Andrea Geforce? I can't find any information about him/her.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#272
post #18

I'm confused. Is there any relationship between the recent Ampere Arm64 servers ( https://news.ycombinator.com/item?id=22475036 ) and Nvidia's "Ampere Architecture", or is it just a case of them using the same name?

I don't like that people downvoted you for asking a question. If someone thinks the question is stupid or not doesn't mean that a downvote is warranted. (nor an upvote, answer the question and move on.) To answer though; it's just a coincidence, as you might already know Nvidia uses famous scientists (especially in the field of electricity) as the names of their microarchitectures. * Volta (Alessandro Volta, inventor…

Tesla was also the name of a Czechoslovak elektronics company, known among other thing for their electron microscopes. Even though this Tesla is long gone, it has a lasting legacy here in Brno due to many electron microscope manufacturers (Delong, FEI, Thermo Fisher, etc.) being present and often libking their origin or many employees to the old Tesla company.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#273
post #207
post #175

Earlier quoted context omitted.

Anyone is free to target PTX and do their own compiler on top. In fact, given that it is there since version 3, there are compilers available for almost all major programing languages, including managed ones. While OpenCL is a C world, and almost no one cares about the C++ extensions and even less vendors care about SPIR-V. Also the community doesn't seem to be bothered that for a long time, the only SYCL implementat…

> the community doesn't seem to be bothered that for a long time, the only SYCL implementation was a commercial one Bothered has nothing to do with it. Implementing low level toolchains generally seems to require both a gargantuan effort and an incredible depth of knowledge. If it didn't, I think tooling and languages in general would be significantly better across the board. What am I supposed to do, implement a SYC…

Which is why CUDA won, most researchers can't be bothered to keep writing C based shaders with printf debugging.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#274

5 petaflops in DGX? That alone would put one of those babies into TOP500 top 50, and a superPOD would make no. 1, no? Well, if it could do that performance on Linpack/Rmax.

No, the A100 has a 19.5 TFLOP theoretical peak for SGEMM[1], real world benchmarks will likely achieve 93% of that, and so the DGX A100 will be 145 TFlops of FP32 SGEMM performance or 0.145 FP32 PFLOPS. Maybe in 72 FP64 TFLOPS. FP64 is what the TOP500 benchmaks count.[2]

The 5 "petaflops" number is a creatively constructed marketing number based on FP16 TensorCore "flops", sparse matrix calculations, and then multiplying by 8x for some reason. They basically take the 19.5 FP32 TFLOPS number and multiply it by 32x to get to the claimed 624 "TFLOPS" for a single A100. 8 * 624 = 5 "petaflops". I see they get 2x by actually using FP16 instead of FP32, 2x from counting sparse matrix ops as dense ops, and 8x from somewhere else that I have no idea.

[1] https://devblogs.nvidia.com/nvidia-ampere-architecture-in-de...

[2] https://www.top500.org/resources/frequently-asked-questions/

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#275
post #23

Earlier quoted context omitted.

Yeah. That's been my general problem with adopting NVidia for anything. They make good hardware, but there's a lot of lock-in, and not a lot of transparency. That introduces business risk. I'm not in a position where I need GPGPU, but if there wasn't that risk, and generally there were mature, open standards, I'd definitely use it. The major breakpoint would be when libraries like Numpy do it natively, and better yet…

> They make good hardware, but there's a lot of lock-in, and not a lot of transparency. This sounds like you'd like NVIDIA to open-source all their software. I see this type of request a lot, but I don't see it happening. NVIDIA's main competitive advantage over AMD and Intel is its software stack. AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because wh…

To be fair, Nvidia has open sourced some key libraries lately. See cutlass and cufftdx.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#276
post #172
post #134

Earlier quoted context omitted.

> NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. I don't bite this argument. Nvidia made close to no effort to support OpenCL and promoted their own technology CUDA. Even in 2020, OpenCL support for Nvidia hardware is close to nonexistent. When the main actor of the market d…

OpenCL isn't also supported on Android, where Google pushes Renderscript instead, their own C99 dialect, yet I don't see any uprising against Google. If the 139 member companies (taking NVidia out) listed here aren't able to provide the same quality in hardware, programming language and eco-system improvements against those from NVidia, and vote for a NVidia employee as chairman, then they deserve what they get. http…

OpenCL used to be supported on Android (But not required on by Google). Currently, Vulkan _is_ required by Google[1] and is probably the future path to GPU compute.

[1] https://www.androidpolice.com/2019/05/07/vulkan-1-1-will-be-...

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#277
post #172

Earlier quoted context omitted.

OpenCL isn't also supported on Android, where Google pushes Renderscript instead, their own C99 dialect, yet I don't see any uprising against Google. If the 139 member companies (taking NVidia out) listed here aren't able to provide the same quality in hardware, programming language and eco-system improvements against those from NVidia, and vote for a NVidia employee as chairman, then they deserve what they get. http…

OpenCL used to be supported on Android (But not required on by Google). Currently, Vulkan _is_ required by Google[1] and is probably the future path to GPU compute. [1] https://www.androidpolice.com/2019/05/07/vulkan-1-1-will-be-...

OpenCL is only supported via hacks to install shared libraries into one own's device.

Vulkan even if optional until Android 10, it is supported by the SDK since version 7, which is something that OpenCL never had.

No serious Android developer would make their life even harder than it already is with the official APIs, by making use of an API that requires device owners to manually install libraries via ADB.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#278
post #216
post #93

Earlier quoted context omitted.

NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent, while hip only works on Linux. As for Python, guess who is on the forefront of GPU Programming with Python, https://www.nvi…

It's not about blame. It's about getting to an ecosystem where GPGPU is used for things beyond deep learning, bitcoin mining, video encoding, and similar niche applications to one where I can fluidly use MIMD to speed up my JavaScript and Python with first-class language constructs to support that. If that happens: 1) We'll get back on some kind of curve where computer performance starts increasing again. 2) The GPU…

It is easy to talk about the proprietary practices done by NVidia, yet none of the other GPGU device makers that are on Khronos weren't able to offer a better experience.

So Khronos has 140 members, about 10 of them producing hardware, and they can't provide a proper developer experience, with software that looks like EE toolchains of the 90's.

The market has already exploded, and CUDA has won.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#279

Earlier quoted context omitted.

And even before that there was: * GeForce (for Andrea Geforce, the first to use the color electric green) * Riva (for Jose Riva, the discoverer that you can use TNT to generate electricity.)

Wow Riva, that brand name brings back memories. Almost as magical as 3dfx and Voodoo. (which NVIDIA ultimately acquired). Truly game changing hardware. Thanks Scott, Gary and Ross (founders of 3Dfx) you cards gave me and my friends thousands of hours of enjoyment.

I bought a Voodoo as my first 3D card and was so disappointed that it did not support my motherboard (PCI 2.1, I had PCI 2.0), so I had to return to the shop and exchanged it for a TNT. :(

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#280
post #18

Earlier quoted context omitted.

I don't like that people downvoted you for asking a question. If someone thinks the question is stupid or not doesn't mean that a downvote is warranted. (nor an upvote, answer the question and move on.) To answer though; it's just a coincidence, as you might already know Nvidia uses famous scientists (especially in the field of electricity) as the names of their microarchitectures. * Volta (Alessandro Volta, inventor…

Tesla was also the name of a Czechoslovak elektronics company, known among other thing for their electron microscopes. Even though this Tesla is long gone, it has a lasting legacy here in Brno due to many electron microscope manufacturers (Delong, FEI, Thermo Fisher, etc.) being present and often libking their origin or many employees to the old Tesla company.

And a museum in Belgrad.
Post reply on HN