Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

tomshardware.com

Post reply on HN
Print view View original

121–130 of 182 posts

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#121

Post by noogle »

agnosticmantis wrote: post #116
noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

A significant part of the 2021 peak may be explained by the crypto craze from which Nvidia benefited greatly and which has almost completely vanished since. Thinking about it, it’s hard to believe how fast the hype cycle moved on from crypto. Only 1-2 years ago every media person, influencer, YouTuber, tweeter etc. were talking about/selling/shilling some kind of crypto, and now all of it seems to have moved on to AG…

Cryptocurrencies still had high barriers for entry for the public at large - not really a means of payment, and high risk as an investment.

Generative AI is used by millions, has very low barrier for entry (it's even free!) and most importantly does not require a network effect so can be valuable immediately.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#122

Post by noogle »

gitfan86 wrote: post #82
noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

It is unknown how much pricing power NVDA has. Can they 3x the price of everything And still sell out?

Why not? They seem to be a lot of leeway before any specific company will find it cheaper to design their own chips, or even to move to AMD (ROCm is not as well supported).

Perhaps someone like OpenAI has both the expertise and incentive to do so, but not many others.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#123

Post by ChuckNorris89 »

touisteur wrote: post #107
ChuckNorris89 wrote: post #8

So this is why Nvidia isn't lowering the price on the GPUs despite them sitting on the shelves and not selling. They make enough money from customers in the data center and supercomputer businesses that gaming is just a small market.

H100s are not sitting on shelves, even at the 35kUSD price sticker. Consumer GPUs, probably yes. Even for datacenter compute workloads that would not go for H100, the L40 is supposedly 3xA40 in FP32 FLOPs but still on the same memory bandwidth, so who knows what kind of performance you'll get whenever you can get your OEM to build you one......

>H100s are not sitting on shelves, even at the 35kUSD price sticker.

How could they be sitting on shelves, as they're never put on shelves to begin with, since they're never sold to consumers?

Obviously I was talking about consumer GPUs.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#124

Post by noogle »

smoldesu wrote: post #83
noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

It doesn't necessarily depend on Nvidia hardware. Nothing stops you from training an AI on an adequately advanced ASIC or FPGA, in theory. Nvidia does accelerate it though, and they're also offering unparalleled performance-per-dollar to the audience that's in the market. In a way, it feels like Nvidia is embarrassingly aware of this. They were the reluctant shovel salesman during the cryptocurrency gold rush, and th…

It's the ecosystem - everyone else is using CUDA, so you need a very good incentive to stray away from that ecosystem. a x2-3 cost of hardware won't justify such move.

The cryptomarket was less favorable to Nvidia because it harmed the loyal customers (gamers, AI) for a temporary market (crypto) that indeed largely declined.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#125

Post by ipsum2 »

binkHN wrote: post #25

Technical detail: > Each A3 supercomputer is packed with 4th generation Intel Xeon Scalable processors backed by 2TB of DDR5-4800 memory. But the real "brains" of the operation come from the eight Nvidia H100 "Hopper" GPUs, which have access to 3.6 TBps of bisectional bandwidth by leveraging NVLink 4.0 and NVSwitch.

So it's not a super-computer, its a single server with 8 GPUs. Hilarious branding.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#126

Post by faeriechangling »

noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

Because making decisions on account of an asset's price being higher 2 years ago is just falling victim to price anchoring? Would Nvidia not be worth buying in 2020 because its price was much lower in 2018 and thus must be overvalued in 2020?

Investments should be based on the actual value of the company relative to its price, as well as relative to other investment oppertunities. Trying to making a profit by trading based on historical stock prices will get you whipped by quants who are already doing a much better job of that sort of thing than you could ever hope to do.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#127

Post by vasco »

cloudking wrote: post #42
IanCal wrote: post #9

Kinda feels like the main thing google launches is waiting lists.

Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.

And once you finish opening up the service kill it very quickly because it doesn't make as much money as search and start working on the next thing.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#128

Post by espadrine »

IanCal wrote: post #65
cloudking wrote: post #42

Earlier quoted context omitted.

Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.

I get it, but the launches are always about what you can now do then slowly followed with "some partners can register interest". For example for palm/bard this was my experience: "Hey we have this amazing LLM!" "Great, given you are a company can I pay you money above your costs for this service?" "No but you can register for updates about when the wait-list will open " They announced cool features for Google docs as…

Looking back at the promises made at I/O 2022, most of the products were released timely (for instance, Docs auto-summary, an AI feature, came out in March for Workspace), although some could be in a better spot:

- Immersive mode in Maps (also AI, using NeRF) has only recently added just 5 cities,

- The screenshot-then-Multisearch Near Me is technically shipped, but it seems super-rough; I screenshot my keyboard and it suggested a specific brand of pasta across nearby supermarkets,

- I am still waitlisted for access to LaMDA through the AI test kitchen (and given this year’s I/O, things seem to take a different direction).

There is no question that ChatGPT’s release in particular went by a more successful playbook comparatively.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#129

Post by m00dy »

plumeria wrote: post #117
MuffinFlavored wrote: post #94

Earlier quoted context omitted.

they have to pay for the power and the staff to set them up/manage it

I wish that humanity gets to harvest static energy one day, so that everyone is able to run the experiments required in large-scale deep learning research, not only a handful of deep pocketed organizations.

don't worry, smartest people I know are working on this problem.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#130

Post by danielmarkbruce »

ipsum2 wrote: post #125
binkHN wrote: post #25

Technical detail: > Each A3 supercomputer is packed with 4th generation Intel Xeon Scalable processors backed by 2TB of DDR5-4800 memory. But the real "brains" of the operation come from the eight Nvidia H100 "Hopper" GPUs, which have access to 3.6 TBps of bisectional bandwidth by leveraging NVLink 4.0 and NVSwitch.

So it's not a super-computer, its a single server with 8 GPUs. Hilarious branding.

Given "supercomputer" isn't an agreed upon term, and this single server is significantly higher performance than anything most people get to use, the claim isn't that bad.
Post reply on HN