Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Samsung's Processing-in-Memory (PIM)

chipsandcheese.com

Post reply on HN
Print view View original

1–10 of 118 posts

Re: Samsung's Processing-in-Memory (PIM)

#3

Post by reliabilityguy »

Interesting that Samsung still pursues PIM. IIRC they had a paper in ISCA21 or 22 where they showed HBM2 module with PIM, which back then impressed me quite a lot.

That being said, I am not sure what’s the killer application for this technology, and without such application adoption is unlikely.

Re: Samsung's Processing-in-Memory (PIM)

#5

Post by xyzzy123 »

reliabilityguy wrote: post #3

Interesting that Samsung still pursues PIM. IIRC they had a paper in ISCA21 or 22 where they showed HBM2 module with PIM, which back then impressed me quite a lot. That being said, I am not sure what’s the killer application for this technology, and without such application adoption is unlikely.

As I understand it, the killer app is llms. You could run MACs directly in RAM, offloading a lot of work from CPU and cutting down on insane (external) memory bandwidth required.

Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.

Re: Samsung's Processing-in-Memory (PIM)

#6

Post by netfortius »

reliabilityguy wrote: post #3

Interesting that Samsung still pursues PIM. IIRC they had a paper in ISCA21 or 22 where they showed HBM2 module with PIM, which back then impressed me quite a lot. That being said, I am not sure what’s the killer application for this technology, and without such application adoption is unlikely.

> That being said, I am not sure what's the killer application for this technology...

Build it, and they will come ;)

Re: Samsung's Processing-in-Memory (PIM)

#7

Post by consp »

So you basically dispose of cache for the memory region used? I wonder what the offsets of the cache misses is going to be in practice (the article addresses it but there is no solution/impact given by samsung).

Re: Samsung's Processing-in-Memory (PIM)

#9

Post by embedding-shape »

xyzzy123 wrote: post #5
reliabilityguy wrote: post #3

Interesting that Samsung still pursues PIM. IIRC they had a paper in ISCA21 or 22 where they showed HBM2 module with PIM, which back then impressed me quite a lot. That being said, I am not sure what’s the killer application for this technology, and without such application adoption is unlikely.

As I understand it, the killer app is llms. You could run MACs directly in RAM, offloading a lot of work from CPU and cutting down on insane (external) memory bandwidth required. Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.

> Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.

Isn't this how it works today already? Granted you wanted to run it on RAM rather than VRAM.

Re: Samsung's Processing-in-Memory (PIM)

#10

Post by userbinator »

In-memory computation was already possible with regular DRAM: https://news.ycombinator.com/item?id=22712811

Add a new set of CPU instructions like “rep macb”

...and it's been long enough now, that I can say there was an effort to implement this on standard x86 memory controllers and have the existing string instructions do so, back in the days of SDR SDRAM, but the tradeoffs weren't (yet) in favour.

Post reply on HN