Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

11–20 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#11
post #7

I haven't listened to Jensen speak before, but am I the only one who thought the presentation wasn't very polished? Not a knock on anything he has accomplished, just an observation that sorta surprised me

When he had that slide up with generating everything I was kind of expecting that he'd say this whole keynote is generated including him. That'd have been crazy.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#12
post #7

I haven't listened to Jensen speak before, but am I the only one who thought the presentation wasn't very polished? Not a knock on anything he has accomplished, just an observation that sorta surprised me

He said he didn't rehearse well. I think it makes him come across very genuinely, not some dumb hyperpolished corporate blabla

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#13

Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.

guy is messing it up bigtime and in real-time as well. sheesh. none of his jokes are landing. “we had 2 customers. we have more now”. long pause. screen behind him covered with logos of all his customers. pause. pause. finally applause. ok on to the next tidbit.

whole conference has been proceeding like this now. look if you invite cramer and the wall street crowd, you should throw in some dollar figures. like - who is paying for all this. how much. and why. talk is entirely about token generation bandwidth, exaflops and petaflops, data parallel vs tensor parallel vs pipeline parallel - do you honestly think cramer knows the difference between an ml pipeline and an oil pipeline ?

i am watching this conf with my kid - proper GenZ member - who got up after 5 mins and said man who is this comedian, his jokes are so bad, and left :(

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#14
post #9
post #6

FP8 being 2.5x Hopper is kind of disappointing after such a long time. Since its 2 fused chips, that means it’s 25% effective delta. though it seems most of the progress has been on memory throughput and power use which is still very impressive. I wonder how this will trickle down to the consumer segment.

Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.

30x is the type of number that when you see it in a generational improvement, you should ignore it as marketing fluff.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#16

Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.

Well, stock price is not a good short term indicator about Nvidia developments, nor any company for that matter. Nvidia is doing a very good job.

That being said, their stock is absolutely and hilariously overvalued.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#17

Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.

> A lot of people were hoping for a big pop on some big development

They are waiting for earnings projections for such a pop since right now it is extremely overbought and struggling to move past >$1,000 per share.

For now, Microsoft and OpenAI will use these chips, but in the long term they are just looking at this and plotting to build their own chips and reducing their dependence on Nvidia and will be ready to switch once their contracts have run out.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#19
post #7

I haven't listened to Jensen speak before, but am I the only one who thought the presentation wasn't very polished? Not a knock on anything he has accomplished, just an observation that sorta surprised me

Yeah he said he didn't rehearse and it really shows.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#20
post #9
post #6

FP8 being 2.5x Hopper is kind of disappointing after such a long time. Since its 2 fused chips, that means it’s 25% effective delta. though it seems most of the progress has been on memory throughput and power use which is still very impressive. I wonder how this will trickle down to the consumer segment.

Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.

The 30x number is for a really narrow scenario tbh. Running a GPT 1.8T parameters (w/ MOE) on one GB200
Post reply on HN