[flagged]
[deleted]
Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
21–30 of 340 posts
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#22I haven't listened to Jensen speak before, but am I the only one who thought the presentation wasn't very polished? Not a knock on anything he has accomplished, just an observation that sorta surprised me
The products, animations and slides are doing some heavy lifting. Most jokes don't land and his presentation is somewhat confusing at times (e.g. star trek intro token count)
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#23Earlier quoted context omitted.
Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.
30x is the type of number that when you see it in a generational improvement, you should ignore it as marketing fluff.
From how I understood it, it means they optimised the entire stack from CUDA to the networking interconnects specifically for data centers, meaning you get 30x more inference per dollar for a datacenter. This is probably not fluff, but it's only relevant for a very very specific use-case, ie enterprises with the money to buy a stack to serve thousands of users with LLMs.
It doesn't matter for anyone who's not microsoft, aws or openai or similar.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#24FP8 being 2.5x Hopper is kind of disappointing after such a long time. Since its 2 fused chips, that means it’s 25% effective delta. though it seems most of the progress has been on memory throughput and power use which is still very impressive. I wonder how this will trickle down to the consumer segment.
Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.
Yeah and the 30x is largely due to the increase in factors like packaging and throughput. It's not indicative of general purpose performance which is what I was talking about.
Again, I do think the throughput and energy efficiency gains are impressive, but the raw performance gain is lower than I'd have expected for the massive leap in node size etc
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#25Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.
Well, stock price is not a good short term indicator about Nvidia developments, nor any company for that matter. Nvidia is doing a very good job. That being said, their stock is absolutely and hilariously overvalued.
Tell me more about why you believe their stock is hilariously overvalued.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#26Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.
guy is messing it up bigtime and in real-time as well. sheesh. none of his jokes are landing. “we had 2 customers. we have more now”. long pause. screen behind him covered with logos of all his customers. pause. pause. finally applause. ok on to the next tidbit. whole conference has been proceeding like this now. look if you invite cramer and the wall street crowd, you should throw in some dollar figures. like - who…
Nah, wallstreet doesn’t understand what it’s looking at.
That’s fine, it’s a developer conference for a founder lead company that hasn’t reached the “stock price is the product” state. He’s not trying to optimize the next 5 days of stock.
There’s a full ecosystem grab with Nim there, a new GPU that forces every major datacenter to adopt (or their competitors will massively increase their compute density)
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#27Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.
guy is messing it up bigtime and in real-time as well. sheesh. none of his jokes are landing. “we had 2 customers. we have more now”. long pause. screen behind him covered with logos of all his customers. pause. pause. finally applause. ok on to the next tidbit. whole conference has been proceeding like this now. look if you invite cramer and the wall street crowd, you should throw in some dollar figures. like - who…
This is a developer's conference, not a financial one.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#28Earlier quoted context omitted.
Well, stock price is not a good short term indicator about Nvidia developments, nor any company for that matter. Nvidia is doing a very good job. That being said, their stock is absolutely and hilariously overvalued.
Tell me more about why you believe their stock is hilariously overvalued.
Because he missed the train. My guess.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#29Earlier quoted context omitted.
Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.
The 30x number is for a really narrow scenario tbh. Running a GPT 1.8T parameters (w/ MOE) on one GB200
[deleted]
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#30Earlier quoted context omitted.
Jensen revealed later that the LLM inference is 30x due to architectural improvements, it's massive. I don't know if it's latency or just 2-3x performance boost with 30x more customers served in the same chip. Either way, 30x is massive.
The 30x number is for a really narrow scenario tbh. Running a GPT 1.8T parameters (w/ MOE) on one GB200
'narrow scenario,' perhaps, but one that also happens to closely match rumors for GPT4's size