Live data from Hacker News

Nvidia announces financial results for second quarter fiscal 2024

nvidianews.nvidia.com

221–230 of 444 posts

Re: Nvidia announces financial results for second quarter fiscal 2024

#221
post #206

Earlier quoted context omitted.

I remember meeting with them in the mid aughts when they were first talking to HPC folks about using their cards for science. I'll never forget what the chief scientist from nVidia said. "What is the color of a NaN? That is, when you render a texture with a nan value, what does it look like? I'll tell: it's nvidia green."

It's a quality meme but I'm having trouble figuring out the settings that make it work. It looks like RGBA8 would be blue: >>> struct.pack('f',math.nan) b'\x00\x00\xc0\x7f' maybe that becomes green if you composite over white or something? Or maybe there is a common type of NaN that fills some of the unspecified bits? ("Just use the particular NaN that makes it green" is cheating unless you have an excuse)

It was an arbitrary decision by the engineers who made the early GPUs, they just mapped NaN to an RGB

It was a nice way to debug tensors: render them to the screen, the green sticks out.

Re: Nvidia announces financial results for second quarter fiscal 2024

#222

Earlier quoted context omitted.

For much of tech hardware world, declining costs and increasing performance have been the general trends for as long as most of us have been alive.

Off the top of my head the only thing I can think of that did that was TVs.

Personal computers have plummeted in price over the decades

Re: Nvidia announces financial results for second quarter fiscal 2024

#223
post #206

Earlier quoted context omitted.

I remember meeting with them in the mid aughts when they were first talking to HPC folks about using their cards for science. I'll never forget what the chief scientist from nVidia said. "What is the color of a NaN? That is, when you render a texture with a nan value, what does it look like? I'll tell: it's nvidia green."

That is a funny way to signal their commitment to HPC! But compared to other tooling (non GPU) CUDA is still really clunky. Way ahead of everything else in the GPGPU space but still surprisingly clunky. Also I don't get what they are fearing with all their "Account required for download" (e.g. for CuDNN) what are they fearing? And is it really worth the trade-off for the pain it causes for dev environments and CI pip…

No, you're not missing anything, NVIDIA's software is super clunky by the standards of most of the software world. However, for the last decade, the competition has been much worse: OpenCL development on AMD would be riddled with VRAM leaks, hard lockups, invisible limits on things like function length and registers that would cause the hard lockups when you tripped over them without any indication as to what you did wrong or how to fix it, that sort of thing. Cryptic error messages would lead to threads scattered around the internet, years old, with pleas for help and no happy endings.

The thing that caused me to ragequit the AMD ecosystem was when I took an OpenCL program I had been fighting for two days straight and ran it on my buddy's Nvidia system in hopes of getting an error message that might point me in the right direction. Instead, the program just ran, and it ran much faster, even though the nvidia card was theoretically slower.

In terms of quality, I expect the competition to catch up in a generation or two, but then there is still the decade+ of legacy code to consider. Hopefully with how fast AI/ML churns that isn't actually an insurmountable obstacle.

Re: Nvidia announces financial results for second quarter fiscal 2024

#224

Earlier quoted context omitted.

> Large enterprises are already putting them into production. I have direct experience with it. For what function have you experienced LLMs being used at significant scale in production right now? It seems unlikely most enterprises have built up sufficient technical know-how to run these workloads in-house already. > They already face serious competitors in google and aws with TPU and inferentia TPUv4s aren't widely…

Customer support. I've never seen so many people get up to speed so fast on something. It's unlike anything ever. Yep. I believe so. I don't believe Pat. With the relevant audience, Nvidia is the brand which is the strongest. Google could just drop support for TPU, AMD isn't viewed as currently credible for people doing the work, nor Intel. You may be right, only time will tell. I've been involved with GPU use for ge…

> Customer support. I've never seen so many people get up to speed so fast on something. It's unlike anything ever.

Are you saying that you've seen enterprises developing, training and running their own in-house LLMs from scratch directly on large (i.e. 100s to 1000s) GPU clusters, whether on-prem or cloud, for this to be relevant to Nvidia?

Pardon my skepticism but it seems odd that a generic Fortune 500 co has the in-house talent and will-power to manage large distributed training runs when much easier and cheaper alternatives like OpenAI/open-source models or one of the Google/MS/AWS MLaaS options are available.

> With the relevant audience, Nvidia is the brand which is the strongest. Google could just drop support for TPU, AMD isn't viewed as currently credible for people doing the work, nor Intel.

I think we're confusing some things here. Right now, there is no good alternative for anyone requiring H100s for loyalty to even matter, this could very easily change with the next generation of accelerator chips.

Intel had the strongest CPU "brand" for a while and enterprises/datacenters readily switched to AMD when it became the better option.

> You may be right, only time will tell. I've been involved with GPU use for general purpose workloads since Cuda was launched. At every step of the way there was apparently credible competition at different layers of the stack just around the corner. That's over 15 years. OpenCL, ASICs, FPGA, Intel this that and the other, AMD this that and the other. TPU. Others I've forgotten.

The TAM, and profit margin, for enterprise-grade GPUs (or accelerators) is several orders of magnitude larger than it has ever been including the crypto craze.

Re: Nvidia announces financial results for second quarter fiscal 2024

#225

Incredible company. It’s absolutely insane how far ahead they are with the investments they made over a decade ago. So nice to see a “hard” engineering (from silicon to software) SV-founded company getting all this recognition. Especially after what has felt like a decade of SV hype software companies dominating the mainstream financial markets pre-pandemic with a spate of overpriced IPOs or large ad-revenue generati…

Yeah. NVidia was a docile looking company and in 2012, they were merely a gaming oriented hardware shop.

These companies exist today. Which small or ignored companies do you think have a bright future?

Re: Nvidia announces financial results for second quarter fiscal 2024

#226
post #178

Earlier quoted context omitted.

That theory is still valid, the issue is that the competition can't or won't even try to make a better product or a cheaper one. The rule only applies if there exist competing products in the first place. If there was any, the prices would go down as we have seen a billion times.

So, why’s that not happening, and what’s it imply for the rest of the theoretical framework?

It’s not happening because 10,000 people who have more intimate knowledge of the business than you or I ever will have made decisions to best suit their current conditions. This isn’t an exception to the rule, you’re just looking at a small timeframe and a remarkably performant company. Why is it so bad for a company to be successful when they have provided so much back to society in the form of R&D? Besides, if I’m doing ML my boss has paid for the card anyway so the price doesn’t concern me.

Re: Nvidia announces financial results for second quarter fiscal 2024

#227
post #178

Earlier quoted context omitted.

That theory is still valid, the issue is that the competition can't or won't even try to make a better product or a cheaper one. The rule only applies if there exist competing products in the first place. If there was any, the prices would go down as we have seen a billion times.

This is the point. This is a textbook situation that would be perfect for a competitor to come in and undercut. However not only is that not happening, nobody is even trying. Making the “theory” pretty worthless if it’s not even applicable in cases that would naturally produce this market entrant. The reality is that private equity does not actually want to compete with large global brands.

> This is a textbook situation that would be perfect for a competitor to come in and undercut.

Is it? A competitor can enter the market and undercut by producing a cheaper and otherwise undifferentiated commodity-type product. Nvidia's focus is adding moats that prevent competing on pure specs such as CUDA, design, and so on.

Re: Nvidia announces financial results for second quarter fiscal 2024

#228
post #208
post #142

Earlier quoted context omitted.

100 gig, that's considered cute nowadays. https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances... 3.2 terabits.

I don't think that machine has a single nic with that bandwidth- I'd guess it's 8 400Gbps cards or something similar.

Correct, it's 8x 400G cards, one per GPU.

Re: Nvidia announces financial results for second quarter fiscal 2024

#229
post #206

Earlier quoted context omitted.

I remember meeting with them in the mid aughts when they were first talking to HPC folks about using their cards for science. I'll never forget what the chief scientist from nVidia said. "What is the color of a NaN? That is, when you render a texture with a nan value, what does it look like? I'll tell: it's nvidia green."

It's a quality meme but I'm having trouble figuring out the settings that make it work. It looks like RGBA8 would be blue: >>> struct.pack('f',math.nan) b'\x00\x00\xc0\x7f' maybe that becomes green if you composite over white or something? Or maybe there is a common type of NaN that fills some of the unspecified bits? ("Just use the particular NaN that makes it green" is cheating unless you have an excuse)

They mean big-endian NaN, taking only the first 3 bytes. No alpha channel.

https://encycolorpedia.com/76b900 says Nvidia green #76b900.

Re: Nvidia announces financial results for second quarter fiscal 2024

#230

Earlier quoted context omitted.

Should have sold a call credit spread instead! I'll get right on that...after I go look up what that means. :-) I'm but a simple options trader who sells calls to unload stock I didn't want anymore anyway, and the premium is the icing on that cake. Left some money on the table this time, but I otherwise would have just sold the shares outright, and I did make some bank regardless. Gonna be missing that sweet, sweet $…

A call credit spread simply means buying an even more out-of-the-money call along with the one you sold. It would have reduced the premium collected, but the long call would appreciate on sudden moves like today's.

Hmm…that actually sounds like a nice hedge. I’ll keep that in mind next time a similar situation comes up. Thanks.
Post reply on HN