Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Micron Kicks Off Production of HBM3E Memory

anandtech.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Micron Kicks Off Production of HBM3E Memory

#21

Post by usrusr »

ianbutler wrote: post #7
Brananarchy wrote: post #5

Earlier quoted context omitted.

You over estimate the 'semi-pro' market for graphics cards. Gamers are barely willing to pay for 20GB. There's no market for consumer cards with an order of magnitude more RAM until games are built to use that memory.

This is for ML, not gamers. There is an entirely different market here.

But "basement ML" is a thing, the market of people who are interested in PC gaming but not to the point of being lifestyle gamers who throw every cent they can spare at that altar. The GPU they bought long before the pandemic is still running every game they throw at it, but they never completely stop eyeing the new stuff. Dipping their toes in ML, even if it's just getting through 80% of some stable diffusion setup tutorial, can be a very welcome excuse to upgrade their gaming. A card sold for gaming but with generously overprovisioned VRAM (ideally in the range of the lowest bin of the biggest or second-biggest chip I think) could match that market segment very well - and it would not only compete with other price points, it would actually increase the market by some buyers (those who would not upgrade without the "ML excuse").

Re: Micron Kicks Off Production of HBM3E Memory

#22

Post by omneity »

AnthonyMouse wrote: post #17
omneity wrote: post #15

Earlier quoted context omitted.

I agree with you, that’s how it should be, but that’s not how it currently is. Looking at what’s available right now. You need 3 A100 40GB to get this amount of VRAM which will cost you way north of 20000$. Doing it with A6000s is still about 15k$. There’s not that many high VRAM options out there you know..

If all you want is VRAM you can get old P40s with 24GB for $175 and it's 144GB for $1050. Then you need a big machine to put six of them in but that doesn't cost $4000. But all of this is kludges. The Radeon RX 7900 XTX has more than twice the memory bandwidth of the M3 Max with much better performance per watt than an array of P40s. What you want is that with more VRAM, not any of this misery.

That checks out in principle, but given that P40 doesn't support NVLink, I wouldn't count too much on using six of them together in a performant manner.

But yeah the best option remains an MI300 if you can afford that.

Re: Micron Kicks Off Production of HBM3E Memory

#26

Post by xadhominemx »

hkgirjenk wrote: post #18
brucethemoose2 wrote: post #2

How about higher capacity GDDR6X? 48GB consumer cards (or 96GB pro cards) would sell like hotcakes if AMD/Intel dare to break the artificial VRAM segmentation status quo.

Don't you think Nvidia analyzed, sliced and diced the market to figure out how to maximize profits? AMD is doing the same thing, the only high memory cards they put out (MI300) are for data centers.

NVIDIA has actually not really sliced and diced the market. They only sell Cadillacs, which is fine for now because they're the only game in town.

Re: Micron Kicks Off Production of HBM3E Memory

#27

Post by xadhominemx »

AnthonyMouse wrote: post #9
cinntaile wrote: post #6

Earlier quoted context omitted.

48GB cards would sell like hotcakes. The problem is that they would sell way less cards aimed at professionals, where they have much higher margins.

Intel doesn't sell a lot of graphics cards whatsoever though. Be the first to offer 64GB of VRAM for under $1000 and that could change pretty fast.

Not without CUDA unfortunately.

Re: Micron Kicks Off Production of HBM3E Memory

#28

Post by KeplerBoy »

xadhominemx wrote: post #27
AnthonyMouse wrote: post #9

Earlier quoted context omitted.

Intel doesn't sell a lot of graphics cards whatsoever though. Be the first to offer 64GB of VRAM for under $1000 and that could change pretty fast.

Not without CUDA unfortunately.

Don't underestimate the amount of shit people would be willing to deal with to make stuff work.

A capable GPU with 24+ GB would sell if it significantly undercuts Nvidia. Just look at geohot building his tinyboxes with AMD cards.

Re: Micron Kicks Off Production of HBM3E Memory

#29

Post by a_wild_dandan »

omneity wrote: post #22
AnthonyMouse wrote: post #17

Earlier quoted context omitted.

If all you want is VRAM you can get old P40s with 24GB for $175 and it's 144GB for $1050. Then you need a big machine to put six of them in but that doesn't cost $4000. But all of this is kludges. The Radeon RX 7900 XTX has more than twice the memory bandwidth of the M3 Max with much better performance per watt than an array of P40s. What you want is that with more VRAM, not any of this misery.

That checks out in principle, but given that P40 doesn't support NVLink, I wouldn't count too much on using six of them together in a performant manner. But yeah the best option remains an MI300 if you can afford that.

Yeah, my M2 MacBook has 96GB @400GB/s. For $4k or so, it feels like cheating. Does it beat 4x24GB NVIDIA cards? Absolutely not! It's slower and occasionally runs into CUDA-moat software issues. But the capability to daily drive Mixtral 8x7 locally, with great token speeds, is phenomenal.

Re: Micron Kicks Off Production of HBM3E Memory

#30

Post by xadhominemx »

KeplerBoy wrote: post #28
xadhominemx wrote: post #27

Earlier quoted context omitted.

Not without CUDA unfortunately.

Don't underestimate the amount of shit people would be willing to deal with to make stuff work. A capable GPU with 24+ GB would sell if it significantly undercuts Nvidia. Just look at geohot building his tinyboxes with AMD cards.

I would personally love that project but there are already so many versioning issues in the space it would be a nightmare if ROCm randomly broke things all the time.
Post reply on HN