The comparison is between setups with different amounts of GPU RAM and there's no quantification of final performance/price.
AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
51–60 of 273 posts
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#52Earlier quoted context omitted.
> The DAX is only 40 companies, most of which make real products rather than advertising mechanisms This, as the kids say, is just cope. American big tech makes real products. Google is not just ads. Apple is not. Amazon is not. Tesla is not. NVidia is not. Netflix is not. NVidia might be overvalued because of the current AI hype but that does not diminish their real accomplishments! Europe has almost no real tech co…
Why is producing big companies a goal? High standards of living for all seems to a much better goal. And that can be done with small or big companies - so long as economic production is high enough and distributed well enough.
Because that famous EU welfare is funded via taxes. Having well performing companies funds your welfare system.
Currently EU welfare systems are under massive strain and huge waiting lists due to ageing population and economy that hasn't kept up to fund it.
There's no free lunch here. You need big companies with scale that pay huge wages as those mean a lot more tax revenue. Saying no to that kind money out of some made up idealism is just silly copium.
The EU income taxes paid by a single FANG salary employee would be the equivalent of the taxes paid by ~10 average workers. Pretty sure Germany and every other EU country would like to have such taxpayers contributing into the welfare system and not say no to it.
Europe's share of global GDP gas been on a constant decline at the expense of US and Chinese growth. Yeah it's nice to have a better welfare system than China or the US, but how will you fund it in the future if you keep having less money? Political idealism doesn't pay your food and rent.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#53Earlier quoted context omitted.
Okay, so you are saying I should move to america, where apparently a lot of people struggle hard to even get a job? Nah, then ill get my very good wagie pennies here and have plenty jobs available, plus good health insurrance and whatnot.
German unemployment is 3.2%. US unemployment is 4.0%. Neither of these are at all high by historical standards. https://www.bls.gov/news.release/empsit.nr0.htm https://www.destatis.de/EN/Press/2024/06/PE24_217_132.html
Sure, and I can only judge from what I'm hearing, but the internet does make it sounds like it's very hard to find programming jobs as a graduate or junior. I don't know what the truth is. Maybe it's just a minority crying out loud because they don't get hired at FAANG.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#54Earlier quoted context omitted.
Yes But there's a long list of German companies not on the DAX (though Germany DAX really deserves to be worth less than NVidia)
The DAX is made of the 40 most valuable German companies. That’s how it is defined. So the companies not in it, again by definition, matter less.
The CDAX index has about 360 companies and appears to have a market cap of around 2 trillion EUR vs 1.7 trillion for the DAX 40.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#55Earlier quoted context omitted.
It's more how little the Frankfurt stock Exchange is worth. And European devs keep wondering why our wages are lower than in the US for the same work. That's why.
Okay, so you are saying I should move to america, where apparently a lot of people struggle hard to even get a job? Nah, then ill get my very good wagie pennies here and have plenty jobs available, plus good health insurrance and whatnot.
That is a perfectly reasonable approach. On the other side, working in the tech on the US salary, if not blown on luxurious lifestyle, gives one a good chance of quickly reaching financial semi-independence (say, 3 years of full living expenses in the bank), which can be a powerful feature. My 2c.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#56Earlier quoted context omitted.
Same thing was said about Nvidia's crypto bubbles, and then look what happened. Jensen isn't stupid. He's making accelerators for anything so that they'll be ready to catch the next bubble that depends on crazy compute power that can't be done efficiently on CPUs. They're so far the only semi company beating Moore's law by a large margin due to their clever scaling tech while everyone else is like "hey look our new p…
They got extremely lucky with AI following crypto. The timing was close to perfect. I'm not sure there will be another wave like that at all for a long while.
There could be other "AI" waves after LLM. And we have still not hit the self-diving car wave, that could happen the next 20 years (or 40). And neither general-purpose robots, that couuuuld also happen. Personalized medicine has also not happened yet. Nor virtual reality, which might take off one day (or not). There are still many industries that could go big in terms of computational demands.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#57Earlier quoted context omitted.
They got extremely lucky with AI following crypto. The timing was close to perfect. I'm not sure there will be another wave like that at all for a long while.
Maybe but it's not like all those AI compute units or whatever Nvidia called them will be thrown in the dumpster after the AI bubble pops. There's a lot of problems the can be solved on them and researcher are always looking for new problems to solve as compute becomes accesibile. I'm tired of hearing about Nvidia's "luck". There was no luck involved. Nvidia shiped Cuda on consumer GPUs since 2006. That's almost 20 y…
There is no recurring revenue on them though. So NVIDIA needs to continue selling obscene amount of new chips.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#58Why the hell are we doing 128 input token benchmarks in 2024. This is not representative of most workloads, and prefill perf is incredibly important.
For understanding: What would be a suitable input length in your oppinion? And why isnt this a good one: Are real-life queries shorter? Or longer? If i count one word as a token, then in my case most of the queries are less than 128 words.
Including the initialization prompt and your history if you have one?
I use ChatGPT for a very simple task, to map chat messages to one of 5 supported function calls, and the function definitions alone already take up 200 tokens I think
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#59Earlier quoted context omitted.
> The DAX is only 40 companies, most of which make real products rather than advertising mechanisms This, as the kids say, is just cope. American big tech makes real products. Google is not just ads. Apple is not. Amazon is not. Tesla is not. NVidia is not. Netflix is not. NVidia might be overvalued because of the current AI hype but that does not diminish their real accomplishments! Europe has almost no real tech co…
Why is producing big companies a goal? High standards of living for all seems to a much better goal. And that can be done with small or big companies - so long as economic production is high enough and distributed well enough.
Because tech innovation requires tons of R&D and you can't afford to do that otherwise. Europeans use American laptops running an American operating system to watch American movies in an American browser.
European economic production is nowhere near high enough and now Europe is struggling to provide for its aging population and doesn't have enough good jobs for younger people. I support redistribution generally, but the wealth has to be created first or there won't be anything to redistribute.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#60Earlier quoted context omitted.
It's more how little the Frankfurt stock Exchange is worth. And European devs keep wondering why our wages are lower than in the US for the same work. That's why.
Okay, so you are saying I should move to america, where apparently a lot of people struggle hard to even get a job? Nah, then ill get my very good wagie pennies here and have plenty jobs available, plus good health insurrance and whatnot.
>Okay, so you are saying I should move to america
Please stop breaking HN rules. I never said that. HN rules state you need to reply to the strongest interpretation of someone's argument, not the weakest that's easiest to criticize.
I just pointed out once country's economics performance for comparison, if you're want to extrapolate from that that you should move there, that's your issues to deal with, but not my argument.