CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
1–10 of 32 posts
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#2For GPUs, memory compression isn't theoretical: they use it to compress frame, depth and stencil buffers to save bandwidth. Overall performance (i.e. frame rate) improvement is typically 10-20% AFAIK.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#3Sadly you have to remember that transparent compression is both defeated by transparent encryption (as more and more data is encrypted) and is also susceptible to timing attacks when that is not the case. I really wish this weren't the case, since I really think compression is awesome, but this is something you have to keep in mind whenever you introduce compression.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#4I wonder how it would fare compared to "software-only" solution, that is re-architecturing your algorithm to work with compressed data directly.
For example, consider an analytical engine going over values in a column of a database to sum it together. To increase the bandwidth, you can consider compressing the values (for example, with some form of Huffman encoding), and you have two options:
1. Decompress the values before passing them onto calculation, then do the calculation. This is what the paper proposes (transparently in HW as the data are being used in the cache).
2. Do the "addition" directly on the compressed values. This requires more logic. (But it doesn't have to be just implemented in software, it could be hardware assisted by some FPGA or something like that.)
So I wonder, instead of compressing the cache, wouldn't it be better to implement the 2nd solution?
The disadvantage of the 1st solution is that since it has to work in the general case, the compression is more likely to be bad in special cases, where the 2nd solution with a specialized compression/processing combo could be superior.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#5I wonder how it would fare compared to "software-only" solution, that is re-architecturing your algorithm to work with compressed data directly. For example, consider an analytical engine going over values in a column of a database to sum it together. To increase the bandwidth, you can consider compressing the values (for example, with some form of Huffman encoding), and you have two options: 1. Decompress the values…
How does one work with compressed data directly? If you are dealing with the data in blocks, then your compression is only as good as the repetition in each block (perhaps the dictionary can be shared).
But even in this case, you don't deal with compressed data directly, but decompress only 1 block at a time.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#6I wonder how it would fare compared to "software-only" solution, that is re-architecturing your algorithm to work with compressed data directly. For example, consider an analytical engine going over values in a column of a database to sum it together. To increase the bandwidth, you can consider compressing the values (for example, with some form of Huffman encoding), and you have two options: 1. Decompress the values…
How does one work with compressed data directly? If you are dealing with the data in blocks, then your compression is only as good as the repetition in each block (perhaps the dictionary can be shared). But even in this case, you don't deal with compressed data directly, but decompress only 1 block at a time.
There are some operations that can be efficiently done over the compressed data itself. Compressed bitmap is a good motivating example (and I really liked Roaring Bitmap [1]). But I guess that such efficiency is very specific to the exact operation and the viability of general compressed operation is unclear; as a related example, fully homomorphic encryption (FHE) that does the same over ciphertexts is still an active research problem.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#7I wonder how it would fare compared to "software-only" solution, that is re-architecturing your algorithm to work with compressed data directly. For example, consider an analytical engine going over values in a column of a database to sum it together. To increase the bandwidth, you can consider compressing the values (for example, with some form of Huffman encoding), and you have two options: 1. Decompress the values…
How does one work with compressed data directly? If you are dealing with the data in blocks, then your compression is only as good as the repetition in each block (perhaps the dictionary can be shared). But even in this case, you don't deal with compressed data directly, but decompress only 1 block at a time.
No, the quality of the compression depends on what is the probability distribution of your data blocks. If some blocks are much more likely to occur, then you can compress them better (put them in a smaller number of bits).
What I mean by "directly" is instead of having to do
y = y + c^-1 (x_i)
where c(x) is the compression function (and c^-1 is its inverse, decompression), we could define a function f such that
z = f(z, x_i)
and final y = c^-1(z). This f would entail both the "+" (operation to be done on the data) and c^-1 (the decompression).
In general, it's non-trivial to come up with f, but I do believe that it would be in many cases a better approach than to use the first method.
Floating-point numbers can be seen as a simple case of this. If we represent numbers in fixed decimal point, then we have a lot of 0s in the numbers (and also lot of digits of very low significance) that we mostly don't need to have there. Therefore, in practice, we use compression - floating-point representation. The operations on FP are more complicated (you have to align the mantissas before you do arithmetic operations), but it's a good trade-off.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#8For GPUs, memory compression isn't theoretical: they use it to compress frame, depth and stencil buffers to save bandwidth. Overall performance (i.e. frame rate) improvement is typically 10-20% AFAIK.
Yes it also only works well with GPUs. GPU-Ram communication is all about bandwidth.
General purpose CPUs are all about latency. Only very specialized problems would benefit from memory compression. On the other Hand compression will increase latency which is counterproductive to any traditional CPU tasks.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#9Earlier quoted context omitted.
How does one work with compressed data directly? If you are dealing with the data in blocks, then your compression is only as good as the repetition in each block (perhaps the dictionary can be shared). But even in this case, you don't deal with compressed data directly, but decompress only 1 block at a time.
No, the quality of the compression depends on what is the probability distribution of your data blocks. If some blocks are much more likely to occur, then you can compress them better (put them in a smaller number of bits). What I mean by "directly" is instead of having to do y = y + c^-1 (x_i) where c(x) is the compression function (and c^-1 is its inverse, decompression), we could define a function f such that z =…
In most, if not all compression schemes, your "compression function" depends on all x_j with j<i. So it's really a new function for every i.
Re: CRAM: Efficient Hardware-Based Memory Compression for Bandwidth Enhancement
#10Their assumption of 64 bytes cacheline is unrealistic. Most CPU architecture use 64 bit cacheline (1/8 of 64 bytes), for good reasons. 512 bit cacheline is insane.
Also, good compression requires large corpus / large model - thus page-based compression is much more useful in real life. For example Google's ChromeOS uses zram by default since 2013 [1].