Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neuralink Compression Challenge

content.neuralink.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Neuralink Compression Challenge

#11

Post by iamcreasy »

palaiologos wrote: post #10

they're looking for a compressor that can do more than 200MB/s on a 10mW machine (that's including radio, so it has to run on a CPU clocked like original 8086) and yield 200x size improvement. speaking from the perspective of a data compression person, this is completely unrealistic. the best statistical models that i have on hand yield ~7x compression ratio after some tweaking, but they won't run under these constra…

I thought 200x is too extreme as well. In compression literature, is there a way to estimate the upper limit on lossless compressibility of a given data set?

Re: Neuralink Compression Challenge

#14

Post by Sesse__ »

iamcreasy wrote: post #11
palaiologos wrote: post #10

they're looking for a compressor that can do more than 200MB/s on a 10mW machine (that's including radio, so it has to run on a CPU clocked like original 8086) and yield 200x size improvement. speaking from the perspective of a data compression person, this is completely unrealistic. the best statistical models that i have on hand yield ~7x compression ratio after some tweaking, but they won't run under these constra…

I thought 200x is too extreme as well. In compression literature, is there a way to estimate the upper limit on lossless compressibility of a given data set?

There is not, because there could always be some underlying tricky generator that you just haven't discovered, and discovering that pattern is basically equivalent to solving the halting problem. (See https://en.wikipedia.org/wiki/Kolmogorov_complexity#Uncomput...)

As a trivial example, if your dataset is one trillion binary digits of pi, it is essentially incompressible by any regular compressor, but you can fit a generator well under 1 kB.

Re: Neuralink Compression Challenge

#15

Post by iamcreasy »

Sesse__ wrote: post #14
iamcreasy wrote: post #11

Earlier quoted context omitted.

I thought 200x is too extreme as well. In compression literature, is there a way to estimate the upper limit on lossless compressibility of a given data set?

There is not, because there could always be some underlying tricky generator that you just haven't discovered, and discovering that pattern is basically equivalent to solving the halting problem. (See https://en.wikipedia.org/wiki/Kolmogorov_complexity#Uncomput... ) As a trivial example, if your dataset is one trillion binary digits of pi, it is essentially incompressible by any regular compressor, but you can fit a…

Cool. Thanks. How about lossy compression?

Re: Neuralink Compression Challenge

#16

Post by Sesse__ »

iamcreasy wrote: post #15
Sesse__ wrote: post #14

Earlier quoted context omitted.

There is not, because there could always be some underlying tricky generator that you just haven't discovered, and discovering that pattern is basically equivalent to solving the halting problem. (See https://en.wikipedia.org/wiki/Kolmogorov_complexity#Uncomput... ) As a trivial example, if your dataset is one trillion binary digits of pi, it is essentially incompressible by any regular compressor, but you can fit a…

Cool. Thanks. How about lossy compression?

The same, since lossy compression can never be worse than lossless compression. (Also, it is more complex since you have to define your loss somehow. These Neuralink samples seemingly come as .wav files, but you probably wouldn't want to encode them with MP3!)

Re: Neuralink Compression Challenge

#18

Post by p0nce »

jappgar wrote: post #12

why should it be lossless when presumably there is a lot of noise you don't really need to preserve

exactly, when you look at the data it looks entirely like noise without any signal, why transmit that in the first place. And why losslessly.

Re: Neuralink Compression Challenge

#20

Post by [deleted] »

palaiologos wrote: post #10

they're looking for a compressor that can do more than 200MB/s on a 10mW machine (that's including radio, so it has to run on a CPU clocked like original 8086) and yield 200x size improvement. speaking from the perspective of a data compression person, this is completely unrealistic. the best statistical models that i have on hand yield ~7x compression ratio after some tweaking, but they won't run under these constra…

[deleted]
Post reply on HN