Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

501–510 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#501

Post by bitmasher9 »

unglaublich wrote: post #468
bitmasher9 wrote: post #365

Earlier quoted context omitted.

The future is here now, it’s just not evenly distributed. China will mass produce something to the point that it is widely distributed. That is how China acts as a great equalizer on a global scale. Another way China is a great equalizer is their willingness to do business with anyone that can pay.

Isn't any other significant economy willing to do business with anyone that can pay?

Right, go ahead and import Cuban cigars, Iranian oil, and Chinese electric cars.

Nations often impose trade barriers for various reasons. This is a very old tactic.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#502

Post by wmf »

cubefox wrote: post #433
ifwinterco wrote: post #426

Earlier quoted context omitted.

Not necessarily in a notoriously cyclical industry where everyone has already been burned by doing exactly that multiple times

What would they be doing with their enormous profits instead?

For one thing they're paying enormous employee bonuses.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#503

Post by doom2 »

aunty_helen wrote: post #66
oceansky wrote: post #6

Awful time for gamers and PC hobbyists not fully into AI.

This is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.

It might kill the console gaming market, too. Typically consoles get cheaper over time post-release. Instead, all the latest gen consoles are getting price hikes and at least one company is potentially pushing back the next gen release (PS6). A PlayStation 5 for $900? I'll just wait and be happy with my perfectly usable Switch 1 (since the 2 is also more expensive than it should be).

Re: Memory has grown to nearly two-thirds of AI chip component costs

#504

Post by alexwwang »

viccis wrote: post #471
alexwwang wrote: post #416

Earlier quoted context omitted.

You just cannot see your privileges.

It's not that I cannot see them; it's that I don't care. And I certainly don't entertain sanctimonious "le China bad" nonsense like that. You live in a country where every 5 years has been substantially better than the previous 5 for a while now. You don't understand your own "privileges" if you can't understand what it's like to live in a country on a free fall that has foreclosed on the possibility of ever building…

I don’t object what you said. But I think it’s just naive to think the country which seems to keep on being better is a life saver. Surely I have my privilege, compared to my fellow citizens. But that’s just what makes me more disappointed if someone simply praise this country “oh your country is great. You gonna be the first place” or so. Because I know that does none business to me or most of the people like me on this land.

And I know you don’t care about it. It’s understandable because you have your own concerns to devote to. Never mind. But just don’t put all these stuff in a simple way. Please remember the problems beneath those achievements are much more and make a larger numbers of people suffered than those who are benefited through those achievements.

I guess we might have more similarities than disagreements if those issues are abstracted out the country level.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#505

Post by alexwwang »

roenxi wrote: post #447
alexwwang wrote: post #415

Earlier quoted context omitted.

I am living in China and I think you cannot just talk with imagination.

And are you going to enlighten us? What is life like working in a Chinese mine?

Not good. You may look for news online. If you can’t find anything, you might reflect why this happens.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#506

Post by alexwwang »

Curosinono wrote: post #427
alexwwang wrote: post #369

Earlier quoted context omitted.

It’s a horrible thought. Really horrible. You should come to China and work in those factories and mines for some years by yourself.

You should try to see how poor US Americans have to life. Normal people who have to wait for an event were doctors do free health care in a sport gym for a handful of days.

I cannot say more details or I will face the risk of being charged as leaking the nation’s secrets. No kidding. I regretted to start this issue.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#507

Post by cubefox »

wmf wrote: post #502
cubefox wrote: post #433

Earlier quoted context omitted.

What would they be doing with their enormous profits instead?

For one thing they're paying enormous employee bonuses.

This was simply a lie:

> No new DRAM fabs are being built.

See https://manufacturing.economictimes.indiatimes.com/news/hi-t...

Re: Memory has grown to nearly two-thirds of AI chip component costs

#508

Post by andrepd »

Dylan16807 wrote: post #359
andrepd wrote: post #151

Earlier quoted context omitted.

I wonder if we will see an adoption of alternative floating point formats. IEEE floats are notoriously terrible at lower widths (<= 16 bits). Floating point formats such as posits do much better at 16 or 8 bits. If you could train at 16 bits per value instead of 32, and suffer a much smaller inaccuracy penalty than you would from IEEE32 to IEEE16...

Notoriously terrible? Posits do a little better if your numbers are biased enough toward 1, but not much better. A 16 bit posit in a near-ideal situation matches an 18 bit IEEE float, and in a pretty wide range of situations loses to either fp16 or bf16. Training anything at 8 bits is going to be tough, and it's hard to say if the flexible exponent is worth the precision tradeoffs.

> A 16 bit posit in a near-ideal situation matches an 18 bit IEEE float

Unsure what you mean by this... A posit16 has up to 11 bits of precision. There's no such thing as an 18 bit IEEE float.

> and in a pretty wide range of situations loses to either fp16 or bf16

Many papers have compared neural networks at 16 bits or 8 bits, and posits beat the hell out of floats and it's not even close. Which is very much expected. As they're particularly suited to this task. But also in other domains, like numerical weather simulations, where tests have shown 16-bit posits can replace 32-bit floats.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#509

Post by Ey7NFZ3P0nzAe »

radialstub wrote: post #161
gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

The memory makers will not expand demand drastically. It is in the nature of their business to keep the market under-supplied, otherwise the following oversupply will kill them. Instead, supply is just rerouted from less profitable segments such as mobile and personal computing.

> It is in the nature of their business to keep the market under-supplied

What?! If they did an anti competitive agreement sure. Otherwise no as each supplier is incentivized to produce more than its competitor and less than the demand, while divesting just enough to survive the oversupply risk.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#510

Post by roenxi »

Tuna-Fish wrote: post #486
roenxi wrote: post #449

Earlier quoted context omitted.

This sounds like a plan to sell less when prices are high and more when prices are low. That is one of the stupidest strategies a company could adopt. I assure you, the RAM makers are pumping out as much as they can and increasing capacity as fast as they think the market can handle. I'm not sure what world we live in when the scheming capitalists are all hunched around their table working out how to dodge selling th…

Building new capacity takes years. The idea is that the market is reliably cyclical, so you should expand when there is a downturn, when costs are low and you can afford the short-term capacity hits that expansion causes (fe. when you divide productive teams in two and fill both halves to full strength with new hires).

If you prefer. But we seem to have gone from "undersupply in a boom" to a strategy of oversupplying so aggressively that manufacturers would finish ramping up supply well before the boom before it even happens. And that would be a better strategy.
Post reply on HN