Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

blog.tensorwave.com

Post reply on HN
Print view View original

81–90 of 273 posts

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#81

Post by dukeyukey »

jononor wrote: post #48
gizmo wrote: post #39

Earlier quoted context omitted.

> The DAX is only 40 companies, most of which make real products rather than advertising mechanisms This, as the kids say, is just cope. American big tech makes real products. Google is not just ads. Apple is not. Amazon is not. Tesla is not. NVidia is not. Netflix is not. NVidia might be overvalued because of the current AI hype but that does not diminish their real accomplishments! Europe has almost no real tech co…

Why is producing big companies a goal? High standards of living for all seems to a much better goal. And that can be done with small or big companies - so long as economic production is high enough and distributed well enough.

Big companies means efficiencies of scale. Small companies that succeed and grow inevitably become big companies. If they don't, it means the qualities that make them effective don't scale to the rest of the economy.

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#82

Post by Refusing23 »

huntertwo wrote: post #77

AMD has better seemingly better hardware - but not the production capacity to compete with Nvidia yet. Will be interesting to see margins compress when real competition catches up. Everybody thinks it’s CUDA that makes Nvidia the dominant player. It’s not - almost 40% of their revenue this year comes from mega corporations that use their own custom stack to interact with GPUs. It’s only a matter of time before compet…

> but not the production capacity to compete with Nvidia yet.

thats just a question of negotiating with tsmc or their few competitors

(also didn't tsmc start production of some factories in the US and/or EU?)

I mean, nvidia use tsmc, so does amd.

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#83

Post by raverbashing »

earthnail wrote: post #43
raverbashing wrote: post #19

Earlier quoted context omitted.

Yes But there's a long list of German companies not on the DAX (though Germany DAX really deserves to be worth less than NVidia)

The DAX is made of the 40 most valuable German companies. That’s how it is defined. So the companies not in it, again by definition, matter less.

> DAX is made of the 40 most valuable *listed* German companies

https://www.famcap.com/top-500-german-family-businesses-the-...

Not all of those are listed, or listed in Frankfurt

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#84

Post by Refusing23 »

Rinzler89 wrote: post #16
sva_ wrote: post #13

I try to be optimistic about this. Competition is absolutely needed in this space - $NVDA market cap is insane right now, about $0.6 trillion more than the entire Frankfurt Stock Exchange.

It's more how little the Frankfurt stock Exchange is worth. And European devs keep wondering why our wages are lower than in the US for the same work. That's why.

stock value doesnt reflect a company's income or ability to pay their workers

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#85

Post by Rinzler89 »

lycopodiopsida wrote: post #73
Rinzler89 wrote: post #52

Earlier quoted context omitted.

Because that famous EU welfare is funded via taxes. Having well performing companies funds your welfare system. Currently EU welfare systems are under massive strain and huge waiting lists due to ageing population and economy that hasn't kept up to fund it. There's no free lunch here. You need big companies with scale that pay huge wages as those mean a lot more tax revenue. Saying no to that kind money out of some m…

[flagged]

[flagged]

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#86

Post by jononor »

dukeyukey wrote: post #81
jononor wrote: post #48

Earlier quoted context omitted.

Why is producing big companies a goal? High standards of living for all seems to a much better goal. And that can be done with small or big companies - so long as economic production is high enough and distributed well enough.

Big companies means efficiencies of scale. Small companies that succeed and grow inevitably become big companies. If they don't, it means the qualities that make them effective don't scale to the rest of the economy.

They become big but not necessarily huge - like the 10 biggest tech companies that people here put as the benchmark. For that one needs organizations that also continuously increases their scope - going into new markets, consolidating exiting markets, buying up existing players. And if they are to continue being "European" then they must resist being bought up by the huge US or global tech companies. The latter is a big problem at least here in Norway. We have some companies that grow quite big and successful - but it is generally just a question of time before they get swallowed by a huge corp, often with US headquarters. Example: Atmel, now Microchip

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#87

Post by leonardovida »

gizmo wrote: post #39
CapeTheory wrote: post #22

Earlier quoted context omitted.

The DAX is only 40 companies, most of which make real products rather than advertising mechanisms. Making real physical things just doesn't scale, and never will. While I would enjoy a US tech salary, I'm not sure we want a world where all manufacturing is set aside to focus on the attention economy. Nvidia value deserves to be much higher than any company on the DAX (maybe all of them together, as it currently is) -…

> The DAX is only 40 companies, most of which make real products rather than advertising mechanisms This, as the kids say, is just cope. American big tech makes real products. Google is not just ads. Apple is not. Amazon is not. Tesla is not. NVidia is not. Netflix is not. NVidia might be overvalued because of the current AI hype but that does not diminish their real accomplishments! Europe has almost no real tech co…

> There is one exception, founded in 1984 Europe clearly has many problems stimulating investments and creating a competitive environment for startups and tech companies. But you can't say ASML is the only one "real" tech company. What about Adyen, Spotify, Klarna, N26, Revolut, etc?

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#88

Post by huntertwo »

Refusing23 wrote: post #82
huntertwo wrote: post #77

AMD has better seemingly better hardware - but not the production capacity to compete with Nvidia yet. Will be interesting to see margins compress when real competition catches up. Everybody thinks it’s CUDA that makes Nvidia the dominant player. It’s not - almost 40% of their revenue this year comes from mega corporations that use their own custom stack to interact with GPUs. It’s only a matter of time before compet…

> but not the production capacity to compete with Nvidia yet. thats just a question of negotiating with tsmc or their few competitors (also didn't tsmc start production of some factories in the US and/or EU?) I mean, nvidia use tsmc, so does amd.

Yes it is - but Nvidia has larger contracts _right now_. Nvidia has been investing more money in producing more GPUs for longer, so it’s only natural that they have an advantage now.

But now that there’s a larger incentive to produce GPUs, their moat will eventually fall.

TSMC runs at 100% capacity for top tier processes - their bottleneck is more foundries. These take time to build. So the question becomes - how long can Nvidia remain dominant? It could be quarters or it could be years before any real competitor convinces large customers to switch over.

Microsoft and Google are producing their own AI hardware too - nobody wants to depend solely on Nvidia, but they’re currently forced to if they want to keep up.

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#89

Post by mistermann »

Rinzler89 wrote: post #37
threeseed wrote: post #29

Earlier quoted context omitted.

We are in the middle of an LLM bubble. Nvidia problem will sort itself out naturally in the coming months/years.

Same thing was said about Nvidia's crypto bubbles, and then look what happened. Jensen isn't stupid. He's making accelerators for anything so that they'll be ready to catch the next bubble that depends on crazy compute power that can't be done efficiently on CPUs. They're so far the only semi company beating Moore's law by a large margin due to their clever scaling tech while everyone else is like "hey look our new p…

I think there's also a very high prospect of virtual worlds with virtual people (SFW or otherwise) becoming popular, rendered with Apple/META goggles...that could require insane amounts of compute. And this is just one possibility. Relatively cheap multimodal smart glasses you wear when out and around that offload compute to the cloud are another.

Nvidia could just as easily triple in short order as get cut in half from here imho.

Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference

#90

Post by drawback »

ptero wrote: post #55
dailykoder wrote: post #31

Earlier quoted context omitted.

Okay, so you are saying I should move to america, where apparently a lot of people struggle hard to even get a job? Nah, then ill get my very good wagie pennies here and have plenty jobs available, plus good health insurrance and whatnot.

That is a perfectly reasonable approach. On the other side, working in the tech on the US salary, if not blown on luxurious lifestyle, gives one a good chance of quickly reaching financial semi-independence (say, 3 years of full living expenses in the bank), which can be a powerful feature. My 2c.

Or do both, work for a US company remotely. Most large ones even have an entity in the larger EU countries that can employ you with all the benefits of being an EU employee. They might not pay the exact same as in the US but still considerably above market rate.
Post reply on HN