Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

Post reply on HN
Print view View original

291–300 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#291

Post by zer0c00ler »

throwaway2037 wrote: post #268
bboygravity wrote: post #247

Earlier quoted context omitted.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

I like this question. You raise a very good point. Occam's Razor tells me the simplest explanation is "core competency". Running AI-SaaS is just a very different business from creating GPUs (including the required software ecosystem). As a counterpoint: Look at Microsoft. Traditionally, they have been pretty good at writing (and making money from) enterprise software, but not very good at hardware. (XBox is the one B…

Microsoft's mice and keyboards also were exceptionally good

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#293

Post by Difwif »

bayindirh wrote: post #263
gammalost wrote: post #261

I wonder when we as an industry will start to address the scaling issues in LLMs.It is obviously in Nvidias interest to keep pushing out bigger and better GPUs, but what is the collective interest? It is already proven that good language models are possible given enough resources. The challenge now is to put these models in a solution which do not require unfathomable amounts of resources for the average use cases.

Wasteful software development is easy and keeps momentum for development. As long as growth is king, quick and dirty will always beat well optimized and smaller systems. This is not a problem with AI only, but with every software we use. Only two groups try to optimize things and try to fit into smaller systems. Passionate programmers and people who is paid to do this (e.g.: phone manufacturers' software teams, etc.)…

Not sure it's fair to characterize modern LLMs as 'wasteful software development" or unoptimized. The implementations do quite an impressive level of optimization with what hardware is available. New theoretical methods w.r.t quantization represent most of our software optimization techniques and we're probably hitting the limit of that shortly with ternary or binary gates.

To your point enthusiasts and developers with strong financial motivation have fairly optimized code. AI definitely falls into the later group. These are not like your typical web app :)

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#294

Post by throw0101b »

oblio wrote: post #200
throw0101b wrote: post #136

Earlier quoted context omitted.

> If you think there is a chance that everyone gets bored of AI and moves on to some other fad that is not in Nvidia’s wheelhouse, then it’s probably down from here. You may wish to look at history to see how things can work out: Cisco had a P/E ratio of 148 in 1999: * https://www.dividendgrowthinvestor.com/2022/09/cisco-systems... The share price tanked, but that does not mean that people got bored of the Internet a…

Regarding bubbles: https://en.m.wikipedia.org/wiki/South_Sea_Company https://en.m.wikipedia.org/wiki/Tulip_mania

Neither of these were technologically based speculations.

> https://en.wikipedia.org/wiki/Tulip_mania

While widely used as an example, most of the well-known stories about this were actually made up, and it wasn't as bad as it is often made out to be.

Quinn and Turner, when they wrote about bubbles:

* https://www.goodreads.com/book/show/48989633-boom-and-bust

* https://old.reddit.com/r/AskHistorians/comments/i2wfsm/i_am_...

purposefully excluded it because their research found it wasn't actually a thing. (Though for the general public it can be an illustrative parable.)

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#295

Post by bayindirh »

Difwif wrote: post #293
bayindirh wrote: post #263

Earlier quoted context omitted.

Wasteful software development is easy and keeps momentum for development. As long as growth is king, quick and dirty will always beat well optimized and smaller systems. This is not a problem with AI only, but with every software we use. Only two groups try to optimize things and try to fit into smaller systems. Passionate programmers and people who is paid to do this (e.g.: phone manufacturers' software teams, etc.)…

Not sure it's fair to characterize modern LLMs as 'wasteful software development" or unoptimized. The implementations do quite an impressive level of optimization with what hardware is available. New theoretical methods w.r.t quantization represent most of our software optimization techniques and we're probably hitting the limit of that shortly with ternary or binary gates. To your point enthusiasts and developers wi…

Currently AI folks create bigger models and feed larger training data sets, because we still don't know the efficiency limits. IOW, currently we can't accelerate the learning beyond a certain point with less training data. This is esp. true on LLM/GenAI space.

On the other areas where NNs are used, what I can see is training models with less data is not only plausible, but very possible, esp. in image processing, probably an image carries more information than a single sentence.

I think AI falls on the spectrum with a slight bias to the latter group. Because if you can shrink a model 10% and lose a month, you'd rather have a 10% bigger model now, and reap the money^H^H^H^H^H fame.

I don't know anything about a typical web app, because I'm not a typical developer developing web apps. :)

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#297

Post by paulmd »

sgift wrote: post #269
coryrc wrote: post #258

Earlier quoted context omitted.

I'm not sure your implication. My understanding of the project is AMD didn't want to invest in it anymore.

IMHO there's reason to believe that was what was discussed here plays a role in that decision: https://news.ycombinator.com/item?id=39592689 - namely NVidia trying to forbid such APIs.

that’s literally old news, it’s from ‘20 or ‘21 and just got noticed iirc

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#298

Post by _factor »

jazzyjackson wrote: post #232
_factor wrote: post #162

Earlier quoted context omitted.

Better question: why does a simple search for “What color is a labrador retriever” require any compute time when the answer can be cached? This is a simple example, but 90% of my searches don’t require an llm to process a simple question.

One time I came across a git repo that let me download a gigabyte of prime numbers and I thought to myself, is that more or less efficient than me running a program locally to generate a gigabyte of prime numbers? The compute for a direct answer like that is fractions of a penny, it might be better to create answers on the fly than store an index of every question anyone has asked (well, that's essentially what the w…

It’s in interesting question. I assume they’re using accelerators, and the alternative is a disk or memory hit. It still seems expensive to me.

https://www.linkedin.com/pulse/rising-cost-llm-based-search-...

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#299

Post by lewhoo »

nojvek wrote: post #296

The issue with bigger LLM or even MMMs is that the bigger they are the more they are cramming and regurgitating the training data, and that opens up to lawsuits. Making NNs generalize the way humans do it is still a hard problem.

Is this indeed established ? Could you provide a link or three ?

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#300

Post by blagie »

fragmede wrote: post #286
blagie wrote: post #278

Earlier quoted context omitted.

Genuine answer: Setting up Facebook WOULD be extremely challenging for a company like SpaceX. There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software. To give an example in a closer domain: Look at how long Google lost money on cloud services through 2022 (over $15B in loses),…

Eh I mean Google moved GCP's revenue around because Microsoft was doing that to make Azure look bigger than GCP. If you can't beat em, join em. Google's got long term contracts with a lot of companies and the government, so GCP isn't going to shut down anytime soon. Their consumer products division has problems with product longevy, but we're not paying them corporation level money or signing serious contracts when b…

Having been down this road a few times, this:

> What I've heard is Azure is a pain in the ass, and things take three times as long to set up there, for some reason.

Doesn't matter. The cost here is a rounding error. What does matter is something like this:

https://developer.chrome.com/docs/extensions/develop/migrate...

https://workspaceupdates.googleblog.com/2021/05/Google-Docs-...

https://workspace.google.com/blog/product-announcements/elev...

https://killedbygoogle.com/

https://www.tomsguide.com/news/g-suite-free-shutdown

Etc.

These sorts of behaviors take out whole swaths of businesses wholesale. It's random, and you never know when it will happen to you.

It's the difference between managing a classroom with:

- an annoying kid throwing spitballs every day (Azure)

- the quiet kid who, one day, brings an assault rifle, a few extra mags, and starts spraying bullets into the cafeteria (Google).

Yes, one is a constant source of annoyance, but really, it's very manageable when you consider the alternative.

(Oracle, in the school analogy, is the mean kid who spreads false rumors about you. As far as I can tell, there is never a sound, long-term business reason to pick Oracle. Most of the reason Oracle is chosen is they're very good at setting up offerings which align to misaligned incentives; they're very often the right choice for maximizing some quarterly or annual objective so someone gets their bonus. In return, the firm is usually completely milked by Oracle a few years down the line. By that point, the decision maker has typically collected their bonus, moved on, and it's no longer their problem.)

Post reply on HN