It's funny how the article just assumes PIM be for computing weights of a model without even mentioning it in the introduction.
Samsung's Processing-in-Memory (PIM)
121–123 of 123 posts
Re: Samsung's Processing-in-Memory (PIM)
#122Earlier quoted context omitted.
I think you could do many performant things without any involvement of software. For example you could do AVX on RAM. The CPU would recognize PID RAM and offload AVX instructions to the module. Then, by simply asking for a special memory address you could have access to registers/regions within PID RAM that serve as a result region. Let's say you would need to run a sum over megabytes of data like for accounting. You…
> run a sum over megabytes of data like for accounting It's been many decades since the last time somebody ran a sum over megabytes of data for accounting and though "damn that's a bottleneck I need to optimize". > Personally I'm a big fan of the "in-ram accelerator" idea especially for server space. The operations this model supports are so extremely limited that you would be hard pressed to find applications where…
Well I do. I run blockchain analysis. The RAM is very much a bottleneck in many cases even with DDR5.
Re: Samsung's Processing-in-Memory (PIM)
#123Earlier quoted context omitted.
I think you could do many performant things without any involvement of software. For example you could do AVX on RAM. The CPU would recognize PID RAM and offload AVX instructions to the module. Then, by simply asking for a special memory address you could have access to registers/regions within PID RAM that serve as a result region. Let's say you would need to run a sum over megabytes of data like for accounting. You…
> Having accelerators on RAM like for jpeg compression, audio decoding or mass data operations could be beneficial but you would need to be careful with heat dissipation. I think we are essentially reinventing SSE, AVX & friends from first principles. This is already being done. Compare the speed of libjpegturbo to a non-vectorized implementation and you'll find a 2-4x difference in throughput.
Isn't any external accelerator just reinventing SSE and AVX? CPU is a general purpose processor it could do GPU's job but we don't force it to, because it's faster to do it on external accelerator.