Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

blogs.nvidia.com

Post reply on HN
Print view View original

341–347 of 347 posts

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#341

Post by pjmlp »

wegs wrote: post #315
pjmlp wrote: post #278

Earlier quoted context omitted.

It is easy to talk about the proprietary practices done by NVidia, yet none of the other GPGU device makers that are on Khronos weren't able to offer a better experience. So Khronos has 140 members, about 10 of them producing hardware, and they can't provide a proper developer experience, with software that looks like EE toolchains of the 90's. The market has already exploded, and CUDA has won.

You're missing the point, and taking everything as an attack on NVidia. It's not an attack on NVidia, or have anything to do with Nvidia versus Khronos. It's clear you've got enough baggage there that I won't go there (not like I was trying to go there in the first place). But the market hasn't exploded. Most computers have built-in Intel graphics, and most apps can't make use of GPU. NVidia won the battle with AMD,…

Not at all, my complain is the poor service that Khronos keeps doing pushing their half backed supported APIs, expecting that OEMs pick up the tooling part.

What ends up happening is that OEMs, coming traditionally from the EE and embedded mindset almost never provide any tooling worthwhile using.

This is why platform APIs always end up winning the hearts of most developers that aren't into FOSS mindset.

Interesting that you mention Intel, they would rather have you using Open API or ispc, instead of pure Khronos APIs.

And Intel keeps failing at their GPU story anyway.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#342

Post by catblast »

morgancmartin wrote: post #310
catblast wrote: post #264

Earlier quoted context omitted.

The thing that's a little different about Tesla in the US at least, is he is so incredibly fetishized by eg high-profile idiots: https://theoatmeal.com/comics/tesla , or conspiracy nutjobs like the International Tesla Institute ( http://teslatech.info/ttevents/prgframe.htm http://tesla.org/tesla_fair_abq.htm ) publishing and promoting Tesla related conspiracies and hawking investments in snake-oil technology like Ran…

I've always enjoyed the oatmeal's comics. What is your reason for referring to him as an idiot?

Actually at first I thought idiot was a bit harsh.. but then I reread what I had forgotten about that post - I think if he's juvenile to promote the vandalism of wikipedia over a completely fantastical, out of context reinterpretation of history to label Thomas Edison a "douchebag", well then it is fair to label him an idiot on a limited forum.

Also, there's no accounting for taste but I find even his non-serious comics puerile and not terribly funny - pretty much one step above Taboola chum, or Jim Davis for millenials. Most of the "humor" and overplayed hook is simply describing everyday things with odd adjectives, ie hair cave=vagina, saliva=evil mouth juice, wow. There's probably a term for this trope.. Anyway, it gets clicks on Facebook.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#343

Post by fluffything »

wegs wrote: post #329
fluffything wrote: post #323

Earlier quoted context omitted.

Please, show us how to train Alexa or BERT on a 486. That'll definitely win you the Turing and Gordon Bell prices, and probably the Peace Nobel price for all those power savings!

Please show me a business (aside from Amazon, obviously) who needs Alexa. Most businesses need a word processor, a spreadsheet, and some kind of database for managing employees and inventory. A 486 does that just fine. Most businesses derive additional value from having more, but that's always an ROI calculation. ROI has two pieces: return, and investment. Basic business analytics (regressions, hard-coded rules, and…

> Please show me a business (aside from Amazon, obviously) who needs Alexa.

I'll bite.

Have you ever been getting a haircut, and the hair dresser had to stop to pick up the phone to make an appointment?

Have you ever go to actually pick up a pizza at a small pizzeria and noticed that from 4 employees, 3 are making pizzas, and one is 99% of their time on the phone?

Every single business that you've ever used in your life would be better off with an Alexa that can handle the 99% most common user interactions.

In fact, even small pizzerias and hair salons nowadays are using third-party online booking systems with chat bots. Larger companies are able to turn a 200 people call center into a 20 man operation by just using an Alexa to at least identify customers and resolve the most common questions.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#344

Post by fluffything »

lumost wrote: post #313
fluffything wrote: post #290

Earlier quoted context omitted.

Early tech investments are risky, but if your competition has tech 10 years more advanced than yours, there is probably no amount of money that would allow you to catch up, surpass, and make enough profits to recover the investment, mainly because you can't buy time, and your competitor won't stop to innovate, they are making a profit and you aren't, etc. So to me the main realization here is that in tech, if one com…

This is a good callout, unlike manufacturing the supply chain is almost universally vertically integrated for large software projects. While it's possible to make a kit car that at least some people would buy, most of the big tech companies have reached the point of requiring hundreds of engineers for years to compete. Caveat that time has shown that the monopolies tend to decay over time for various reasons, the tec…

The problem with vertically integrated technology is that if a huge advancement appears at the lowest level of the stack that would require a whole re-implementation of the whole stack, a new startup building things from scratch can overthrown a large competitor that would need to "throw" their stack away, or evolve it without breaking backward compatibility, etc.

Once you have put a lot of money into a product, it is very hard to start a new one from scratch and let the old one die.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#345

Post by sabalaba »

rrss wrote: post #322
sabalaba wrote: post #274

Earlier quoted context omitted.

No, the A100 has a 19.5 TFLOP theoretical peak for SGEMM[1], real world benchmarks will likely achieve 93% of that, and so the DGX A100 will be 145 TFlops of FP32 SGEMM performance or 0.145 FP32 PFLOPS. Maybe in 72 FP64 TFLOPS. FP64 is what the TOP500 benchmaks count.[2] The 5 "petaflops" number is a creatively constructed marketing number based on FP16 TensorCore "flops", sparse matrix calculations, and then multipl…

> 8x from somewhere else that I have no idea. 8 GPUs in the box.

No, I already multiplied 624 TFLOPS / GPU * 8 GPU = 4992 TFLOPS (the 5 petaflops number).

I'm saying that you are still missing another 8x on the way from 19.5 TFLOPS / GPU to 624 TFLOPS / GPU. 19.5 (base FP32 theoretical peak performance) * 2 (FP16 instead of FP32) * 2 (counting sparse matrix ops as dense ops) * 8 (unknown) = 624 TFLOPS.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#346

Post by xigency »

Reelin wrote: post #209
xigency wrote: post #155

Earlier quoted context omitted.

It sounds like people want nvidia to write drivers for AMD. This criticism makes even less sense when any bystander could implement CUDA suppport on AMD by connecting open source software.

> any bystander You aren't seriously implying than any bystander is capable of extending LLVM to map CUDA to SPIR-V? What percentage of present day gainfully employed software engineers do you suppose even has the background knowledge? How many hours do you suppose the work would require?

Late response I know, but I would say anyone who needs that feature could learn to do it, at least if they are on Hacker News. Maybe bystander isn't the most accurate term, but certainly anyone with criticism could take the gauntlet.

LLVM is very well documented and so are these standards. The open source community is also huge and full of talented contributors and more are always welcome to join. I think there's a reason why Linux and GitHub exist.

So in short, if it's a question of motivation and it's something you need, then become motivated to make it happen. That's more likely to happen then convincing a company to invest in supporting a competitor.

Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

#347

Post by rrss »

sabalaba wrote: post #345
rrss wrote: post #322

Earlier quoted context omitted.

> 8x from somewhere else that I have no idea. 8 GPUs in the box.

No, I already multiplied 624 TFLOPS / GPU * 8 GPU = 4992 TFLOPS (the 5 petaflops number). I'm saying that you are still missing another 8x on the way from 19.5 TFLOPS / GPU to 624 TFLOPS / GPU. 19.5 (base FP32 theoretical peak performance) * 2 (FP16 instead of FP32) * 2 (counting sparse matrix ops as dense ops) * 8 (unknown) = 624 TFLOPS.

FP16 tensorcore = 312 tflops

x 2 (counting sparse as dense) = 624 tflops

x 8 GPUs = 5 "pflops"

The missing 8x you are looking for is just because tensorcore math is much faster than their normal fma path.

Post reply on HN