Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory has grown to nearly two-thirds of AI chip component costs

epoch.ai

Post reply on HN
Print view View original

261–270 of 524 posts

Re: Memory has grown to nearly two-thirds of AI chip component costs

#261

Post by xbmcuser »

overfeed wrote: post #216
shdh wrote: post #201

Earlier quoted context omitted.

Yeah, more global competition in DRAM would be great. SK Hynix and Samsung are South Korean.

> SK Hynix and Samsung are South Korean. The Korean memory makers are playing the same game as Micron and simply moving existing capacity up-market. GP was referring to upstart Chinese memory manufacturers like ChangXin, who - if their yields manage to catch the wave - could not have asked for a more favorable market after the big 3 have abandoned the consumer segment. Consumers who would have otherwise turned up the…

Chinese manufacturers will probably takeover consumer ram that most of us use as current manufacturing contracts expire and Samsung SK and micron move all their production to HBM for data centers. Corsair recently released chinese chips based DDR5 sticks.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#262

Post by dylan604 »

esseph wrote: post #257
ahartmetz wrote: post #231

Earlier quoted context omitted.

It is not a law of nature that Chinese products are lower quality (cf. electric cars) and I don't see why they would go for that. They can just bin what they produce like everyone else and sell their products for what they have been tested to deliver.

This has nothing to do about nationality, it has everything to do with building and running a brand new, highly technical, mass production facility.

And historical record of the lack of QA coming from Chinese manufacturing

Re: Memory has grown to nearly two-thirds of AI chip component costs

#263

Post by rcxdude »

bigbadfeline wrote: post #214
brookst wrote: post #172

Earlier quoted context omitted.

Not the person you’re replying to, but RAM has historically been a boom-or-bust business, and companies that invest to meet demand during a boom cycle usually have that new capacity come online just in time for the bust. If it was just variable costs and new capacity was available today they’d do it. But there are substantial fixed costs and delays to increasing capacity, and that uncertainty makes it risky.

That's such a nonsensical argument, it holds for every other business too and in this case it's just a lame excuse for monopolization. If you are that chicken and can't stomach competition you should not be in business anyway. The current RAM manufactures were convicted of conspiracy to manipulate prices back in the 2000s or thereabout, doing so is their modus operandi, but this time the government is participating i…

Chip manufacturing has unusually long spin-up times, high capital costs and relatively thin margins for anything but the latest and greatest processes, compared to most industries.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#264

Post by rcxdude »

Legend2440 wrote: post #31

I wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.

Fab margins are on average super thin compared to the margins of big tech companies, and come with a lot of risk because of that. It's not something they are likely to be keen to integrate.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#265

Post by DarkUranium »

IshKebab wrote: post #102
slicktux wrote: post #2

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

I bought a couple of used computers with 256 GB of DDR 4 (total) a year ago. The ram is worth more than I paid for the whole machines now.

Someone was selling an Epyc machine with 512GB RAM @ 500 EUR last year. I regret not buying it now ...

Re: Memory has grown to nearly two-thirds of AI chip component costs

#266

Post by paulmist »

baq wrote: post #148
paulmist wrote: post #112

Earlier quoted context omitted.

I think it's the opposite. Sure in short term hobbyists are getting squeezed, but the amount of capital that they can put into pushing the edge is small compared to Fortune 500. Sooner or later hobbyists will benefit, especially if the market crashes.

If it crashes after it kills the PC we’ll be left with… nothing? Path matters as much as destination

Why would it kill PC? There will always be hobbyists, e.g. I can't imagine pro e-sports players running on a Mac. Personally, half of the reason I moved away from Windows is Microsoft stalling/degrading Windows experience.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#267

Post by flykespice »

Since memory is becoming an expensive commodity, I guess the old ways of being precious on the efficient memory usage of your program (like it running on the constrained 1mb memory back then) are making a comeback.

I only feel sorrow for the electron devs, they will have a hard time.

Re: Memory has grown to nearly two-thirds of AI chip component costs

#268

Post by NikolaNovak »

cogman10 wrote: post #141
ksec wrote: post #29

Earlier quoted context omitted.

It is one of the thing with consumer when they remember they brought it at the absolutely lowest price point when DRAM maker were bleeding money. Those are not normal pricing. Before the pricing collapse in early 2020, 96GB DDR5 would have cost about $450 to $500. And I will need to restate again the cost of DRAM hasn't really changed much in the past 20 years. Its price just goes up and down in cycles. So in reality…

> Its price just goes up and down in cycles. Historically the price has always trended downward. When I first got into computing $200 could buy you 128 MB (yes M) of ram. Really nice systems had 512 MB. That's obviously changed over the decades as process shrinks have lead to higher memory density. We should generally expect that ram will cheaper up and until the point where process shrinks stop happening. They've de…

Exactly. My first computer had 48k, yes K of ram :-). My first PC has 2MB and made all my friends jealous as they had 1MB. Amiga 500 at the time had half.

I am keeping a piece of paper that came with my Tex Murphy game which stated that one could get 32MB of RAM for as little as $700 (1990s dollars) which would drastically improve the game!

Re: Memory has grown to nearly two-thirds of AI chip component costs

#269

Post by DoctorOetker »

sowbug wrote: post #167
DoctorOetker wrote: post #55

Earlier quoted context omitted.

I know the differences between SRAM, DRAM, ... I asked for evidence different people keep feeding me opposite stories: one insists its not fab capacity but wafer competition, with a recent article claiming HBM3E takes 3 times as much wafer area per bit than LPDDR5X. Others tell me the complete opposite: its fab capacity, not wafer shortage. Do we have citable references to ground either set of claims?

I believe those are two ways of describing the same thing. If you're able to book some fab capacity, that means you get to decide what the fab does with the next wafers in the queue. From your sibling comment, I think you're interpreting the 3x HBM stat as contributing to making wafers scarce. It's more that the next wafer to be processed in a fab is especially precious, making the opportunity cost larger. The beach…

so the bottleneck is the fab, again

Re: Memory has grown to nearly two-thirds of AI chip component costs

#270

Post by weitendorf »

gpm wrote: post #100

An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

If you factor in Nvidia’s profit margin due to the scarcity of the current bleeding-edge chips there is a path to a much larger cost reduction still.

There’s a lot to criticize Sam Altman for saying or popularizing culturally but I’ve come to think his “this is the worst it will ever be” is, in the long run, actually a very intriguing and underrated point.

In a decade training LLMs to the current level of sophistication, which is in my opinion rather advanced and probably has lots of additional upside just from constructing better RL training regime independently of hardware advancement, will become just as table stakes as running a database is now. I highly recommend everyone look into the Allen Institute’s projects in GitHub and HF because they have open source training materials (including an LLM from scratch off common crawl, and some quite interesting tunes of qwen) to get a taste for what will be in the near future afternoon projects or educational material. The future is going to be wild

Post reply on HN