Without proper statistical metrics (why use average when 95% percentile is widely used?) and performance/watt this is a useless comparison.
And performance/price -> that's the bottom line.
AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
111–120 of 273 posts
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#112Earlier quoted context omitted.
Language aside, the entire point of the EU is the single market so you don’t have 26 different rule sets. (There are some exceptions such as health care but that is no different in the US.)
Except you do have 26 different rules. It's a single market on paper as the eu only mandates a small subset of common rules and regulations such as removing tarrifs or freedom of movement, but have you ever tried in practice to launch your company from Belgium to France or from Netherlands to Belgium or from Austria to Germany, or from Romania to Italy? It's much more difficult when the rubber hits the road as every…
Europe is a single market with regards to imports and exports, and that's what matters most for a business. Low wages more than compensate for regulatory annoyances. No shortage of subsidies either.
California has more burdensome regulations and higher taxes than other states and yet it's home to silicon valley.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#113Earlier quoted context omitted.
>How can a wealthy continent with 750 million people produce no big tech companies? It's a big problem. Much more difficult to scale a product across 26 different countries and nearly as many languages and regulatory jurisdictions. US is one country, not a collection of countries fighting each other, meaning your product is instantly available to 300M people speaking the same language under (nearly) the same regulati…
>US is one country, not a collection of countries fighting each other The US is a republic of 50 states. Each state has a huge amount of sovereignty and autonomy. There are 50 state-level regulatory jurisdictions. Not to mention the local-level of government. But in spite of this, the US does not over-regulate. This is the big difference to Europe (I say this as an American expat living in Europe).
Have you tried launching a product in your new EU country and then taking abroad to another EU country? You'll find out it's not exactly like doing the same thing between US states. And I'm not even talking about the language barrier.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#114AMD has better seemingly better hardware - but not the production capacity to compete with Nvidia yet. Will be interesting to see margins compress when real competition catches up. Everybody thinks it’s CUDA that makes Nvidia the dominant player. It’s not - almost 40% of their revenue this year comes from mega corporations that use their own custom stack to interact with GPUs. It’s only a matter of time before compet…
> their own custom stack to interact with GPUs
lol completely made up.
are you conflating CUDA the platform with the C/C++ like language that people write into files that end with .cu? because while some people are indeed not writing .cu files, absolutely no one is skipping the rest of the "stack" (nvcc/ptx/sass/runtime/driver/etc).
source: i work at one of these "mega corps". hell if you don't believe me go look at how many CUDA kernels pytorch has https://github.com/pytorch/pytorch/tree/main/aten/src/ATen/n....
> Everybody thinks it’s CUDA that makes Nvidia the dominant player.
it 100% does
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#115Earlier quoted context omitted.
Language aside, the entire point of the EU is the single market so you don’t have 26 different rule sets. (There are some exceptions such as health care but that is no different in the US.)
Yes, and it's doing a mediocre-at-best job of it. You can remove tariffs and harmonise regulations, but the EU is trying to unite countries with cultural borders older than Christianity.
I would argue that the EU is quite a successful organization given the task of setting up general market rules for 26 countries.
Obviously there is a lot to criticize about the EU and I can offer you a gigantic list there too. However, I do not see any clear failure of the EU’s approach as a single market so far. Additionally part of the philosophy was establishing peace in a region that was torn up by wars for a lot longer than Christianity exists. I would argue the EU was quite successful there too.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#116Earlier quoted context omitted.
The DAX is only 40 companies, most of which make real products rather than advertising mechanisms. Making real physical things just doesn't scale, and never will. While I would enjoy a US tech salary, I'm not sure we want a world where all manufacturing is set aside to focus on the attention economy. Nvidia value deserves to be much higher than any company on the DAX (maybe all of them together, as it currently is) -…
> but how much of that current value is real rather than an AI speculation bubble? I mean, by definition given that it trades freely their market cap is real. Your market cap today is what the market thinks your future cash flows are worth. The bubble and the bubble popping should in theory both be priced into Nvidia's market cap. What isnt' is events the market doesn't anticipate, AMD coming out with a current gener…
> can do inference as well as the H100 is something the market hasn't priced in.
I think probability of that would still be priced in. Not sure what the exact probability is, though.
But if say it was clear that AMD can come up with a competitive option, then NVDA stock would drop. But if it was clear the other way that AMD can't do it, NVDA price would increase.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#117Earlier quoted context omitted.
Large corporate customers like Microsoft and Meta do not use CUDA. They all use custom software. AMD doesn’t have enough GPUs to sell them yet, that’s the real bottleneck.
That's a pretty big claim, that Microsoft and Meta have their own proprietary cuda-replacement stack. Do you have any evidence for that claim?
I'm guessing what they meant is that they use toolchains that are retargetable to other GPUs (and typically compile down to PTX (nVidia assembly language) on nVidia GPUs rather than go through CUDA source -- GCC and clang can both target PTX). For example XLA and most SYSCL toolchains support much more than nVidia.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#118Earlier quoted context omitted.
Yes, and it's doing a mediocre-at-best job of it. You can remove tariffs and harmonise regulations, but the EU is trying to unite countries with cultural borders older than Christianity.
I would argue that the EU is quite a successful organization given the task of setting up general market rules for 26 countries. Obviously there is a lot to criticize about the EU and I can offer you a gigantic list there too. However, I do not see any clear failure of the EU’s approach as a single market so far. Additionally part of the philosophy was establishing peace in a region that was torn up by wars for a lot…
It's definitely successful, and I was probably too harsh there. But I genuinely think the barriers that are left are damn near insurmountable. An awful lot has to change before a Greek tech workers can move to Sweden as easily as a Virginian can move to California.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#119Earlier quoted context omitted.
Why is producing big companies a goal? High standards of living for all seems to a much better goal. And that can be done with small or big companies - so long as economic production is high enough and distributed well enough.
Because tech innovation requires tons of R&D and you can't afford to do that otherwise. Europeans use American laptops running an American operating system to watch American movies in an American browser. European economic production is nowhere near high enough and now Europe is struggling to provide for its aging population and doesn't have enough good jobs for younger people. I support redistribution generally, but…
.... while all of the content is hosted on Linux servers, and you're listening to music through a Swedish app. That is running on silicone made in Taiwan. Using equipment that can currently only be manufactured in Belgium and Germany. On a Mac you are using a British instruction set.
Also in terms of tech innovation: What part of the US-based tech innovation couldn't have been (and actually were) achieved with open-source solutions many many years earlier for a fraction of the cost, if we didn't have copyright?
Honestly, a significant chunk of the "innovation" seems to relate directly to maximizing advertisement opportunities and inducing increased consumption. Who cares if a website takes a second to load rather than 0.1 seconds? If it has content I want, 1 second isn't a big deal. If I don't care about the content, I lose nothing by being distracted by something else in that 1 second.
---
More importantly:
European Economic production isn't high enough... by what standard?
https://data.oecd.org/lprdty/gdp-per-hour-worked.htm
GDP per hour worked is 74 in the US vs 69 in Germany and 54 in the EU. And the EU includes many large countries that emerged from communist dictatorship only 35 years ago, and are very much still in the process of catching up. Incidentally, the German economy is the result of the West German economy with 63 million people absorbing a failing economy hosting 16 million people in 1990.
The idea that the US is some promised land of economic prosperity while Europe is falling is entirely absurd. It's a narrative built on small relative differences and a US system that pressures people into working a lot more than Europeans do.
More importantly, even GDP per Capita wise:
https://data.oecd.org/gdp/gross-domestic-product-gdp.htm
EU per Capita GDP in 2022 is the same as USA 2016. Was the USA in 2016 struggling but now isn't?
This is all bullshit. Economic output is more than high enough and rising steadily. The problem remains solely in the distribution of
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#120Earlier quoted context omitted.
As easy as getting yourself 8 H100 and 8 MI300X. Fun weekend project for anybody.
You can rent them online for ~ 4-5 $ per hour per GPU. Not cheap, but definitely feasible as a weekend project.
where can I rent a H100 for 4-5 dollars an hour?
AWS doesn't let you use p5 instances (not getting a quota as a private person), lambda cloud is sold out.