Earlier quoted context omitted.
.... while all of the content is hosted on Linux servers, and you're listening to music through a Swedish app. That is running on silicone made in Taiwan. Using equipment that can currently only be manufactured in Belgium and Germany. On a Mac you are using a British instruction set. Also in terms of tech innovation: What part of the US-based tech innovation couldn't have been (and actually were) achieved with open-s…
>EU per Capita GDP in 2022 is the same as USA 2016. Was the USA in 2016 struggling but now isn't? Absolutely the US would be struggling if the GDP was still 2016 values with today’s costs.
AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
231–240 of 273 posts
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#232Earlier quoted context omitted.
>EU per Capita GDP in 2022 is the same as USA 2016. Was the USA in 2016 struggling but now isn't? Absolutely the US would be struggling if the GDP was still 2016 values with today’s costs.
Pretty much. I'd also love it (European) if we stil had 2017's rent prices and consumer prices, but we don't. Housing, energy, food, healthcare and everything has gone up like crazy. We're definitely poorer than before, just sweeping it under the rug pretending everyting's fine. So it boggles my mind that your parent tried to make a point by equating USA 2016 with EU 2022 GDP/capita as if nothing's wrong with that. A…
Nominal: https://data.oecd.org/gdp/gross-domestic-product-gdp.htm
PPP/Inflation adjusted not so much: https://ourworldindata.org/grapher/gdp-per-capita-worldbank?...
But pretty much all countries are still well ahead of where we were in 2017. If you feel poorer than in 2017 it's because you're getting less of a larger pie, not because the economy is producing less than it did then.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#233Earlier quoted context omitted.
Because tech innovation requires tons of R&D and you can't afford to do that otherwise. Europeans use American laptops running an American operating system to watch American movies in an American browser. European economic production is nowhere near high enough and now Europe is struggling to provide for its aging population and doesn't have enough good jobs for younger people. I support redistribution generally, but…
.... while all of the content is hosted on Linux servers, and you're listening to music through a Swedish app. That is running on silicone made in Taiwan. Using equipment that can currently only be manufactured in Belgium and Germany. On a Mac you are using a British instruction set. Also in terms of tech innovation: What part of the US-based tech innovation couldn't have been (and actually were) achieved with open-s…
> More importantly, even GDP per Capita wise: > > https://data.oecd.org/gdp/gross-domestic-product-gdp.htm > > EU per Capita GDP in 2022 is the same as USA 2016. Was the USA in 2016 struggling but now isn't? > > This is all bullshit. Economic output is more than high enough and rising steadily. The problem remains solely in the distribution of the ouptut.
The conclusion that economic output has been rising steadily, even in the last couple of years, is true. But the 2016 vs 2022 numbers are nominal, thus useless. There is a much more significant difference over time when working in PPP/Inflation adjusted numbers:
https://ourworldindata.org/grapher/gdp-per-capita-worldbank?...
The overall point holds though: The economic output of the EU is at 45K per capita today, the level of the US in 1997. The US was not a poor country in 1997. Germany is at the economic output per capita of 2009.
Did the US in 1997 suffer from the problem that it didn't produce enough economic output? Of course not.
And given that, adjusting for inflation, GDP per capita is at an all-time high, the conclusion that you're poorer because economic production is distributed to others is necessarily true. And it tracks, too. Corporate profits and the Dow Jones are not down. The already extremely wealthy have accumulated nearly two thirds of the new wealth being created since 2020:
https://www.oxfam.org/en/press-releases/richest-1-bag-nearly...
> Billionaire wealth surged in 2022 with rapidly rising food and energy profits. The report shows that 95 food and energy corporations have more than doubled their profits in 2022. They made $306 billion in windfall profits, and paid out $257 billion (84 percent) of that to rich shareholders. The Walton dynasty, which owns half of Walmart, received $8.5 billion over the last year. Indian billionaire Gautam Adani, owner of major energy corporations, has seen this wealth soar by $42 billion (46 percent) in 2022 alone.
Given these facts, if we have to accept lower economic production in the name of a fairer distribution of economic production, that seems more than acceptable to me.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#234Earlier quoted context omitted.
It's definitely successful, and I was probably too harsh there. But I genuinely think the barriers that are left are damn near insurmountable. An awful lot has to change before a Greek tech workers can move to Sweden as easily as a Virginian can move to California.
I had two Greeks on my team at a large tech company in Sweden.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#235Earlier quoted context omitted.
Europe is much much poorer than the US, actually. https://pbs.twimg.com/media/F3PGpsrWEAEiplB?format=jpg&name=...
EU GDP per capita 2022 is the same as US GDP per capita 2017. Unless you want to say that the US was much poorer in 2017 than it was in 2022 that's a fairly ridiculous statement. Also, the highest productivity places in the EU have much lower hours worked per capita than the US, with Germans on average working 25% less than Americans and the EU as a whole working 13% less than the US. https://data.oecd.org/emp/hours-…
Can't edit anymore, but: That were nominal numbers, and thus useless. See here:
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#236Earlier quoted context omitted.
IMO the relevant benchmark for now is a mixed stream of requests with 50 (20%), 500 (50%), 2000 (10%) and 50k (20%) input tokens, ignore EOS and decode until you get around 300 output tokens.
I'm really interested, do you have a source for those percentages ? I tried to look for some service provider to publish this kind of metrics, but haven't found any.
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#237The market (and selling price) is reflecting the perceived value of nvidia's solution vs AMDs - comprehensively including tooling, software, TCO and managability. Also curious how many companies are dropping that much money on those kind of accelerators just to run 8x 7B param models in parallel... You're also talking about being able to train a 14B model on a single accelerator. I'd be curious to see how "full-accel…
mi300x win in some inference workloads, h100 win in training and some others inference workloads ( fp8 inference with tensorRT-llm , rocm is young but is growing fast ) in a single system ( 8x accelerators ) LLMs, mi300x has very competitive inference TCO vs h100 . also : AMD Instinct MI300X Offers The Best Price To Performance on GPT-4 According To Microsoft, Red Team On-Track For 100x Perf/Watt By 2027 https://wccf…
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#238Earlier quoted context omitted.
Amazon doesn’t make anything anyone wants to buy? Not to be snarky, but if AWS counts as “nothing” I’d sure like a slice of nothing please.
AWS is a (collection of) service(s) which can scale and have low marginal cost of reproduction. Fire Tablets are a real thing you can buy, but they are shit so nobody does.
If I pay for a database server in Virginia, how is that not real?
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#239Earlier quoted context omitted.
I had two Greeks on my team at a large tech company in Sweden.
Do you think there are more Greeks in Sweden, or Virginians in Washington?
Re: AMD's MI300X Outperforms Nvidia's H100 for LLM Inference
#240Earlier quoted context omitted.
Maybe I'm a naive fanboy, but I would put my money on Apple catching Nvidia before AMD or Intel.
Apple doesn't have any hardware SIMD technology that I'm aware of. At best, Apple has Metal API which iOS video games use. I guess there's a level of SIMD-compute expertise here, but it'd take a lot of investment to turn that into a full scale GPU that tangos with Supercomputers. Software is a bit piece of the puzzle for sure, but Metal isn't ready for prime time. I'd say Apple is ahead of Intel (Intel keeps wasting…
M3 Max's GPU is significantly more efficient in perf/watt than RDNA3, already has better ray tracing performance, and is even faster than a 7900XT desktop GPU in Blender.[0]
[0]https://opendata.blender.org/benchmarks/query/?compute_type=...