Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Text Compression as a Test for Artificial Intelligence (1999) [pdf]

aaai.org

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#11

Post by elevaet »

unnah wrote: post #7
isaacfrond wrote: post #5

I'd be interested to see lossy text compression. That would also give much more scope for AI methods. It turned out that recently Stable Diffusion gives excellent compression https://pub.towardsai.net/stable-diffusion-based-image-compr... Funny thing is, on a per pixel basis, Stable Diffusion and jpeg are just as good. Stable diffusion looks much better, though. The reason is that when Stable diffusion lacks the info…

A lossy text compressor can be converted into a lossless compressor quite easily by encoding the remaining differences between the lossy reproduction and the original text. The more accurate your lossy compressor is, the less additional information you need to encode the differences. You can get even better results if your lossy compressor is probabilistic, and can compute approximate probabilities for different text…

But if you're not careful, the delta between the compressed version and the original could be bigger than the savings from the compression.

In my armchair speculation for the case of severe lossy text compression, the delta/diff could easily get there, making a strong compression algo based on something like GPT-3 not really practical for doing lossless.

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#12

Post by PeterisP »

fxtentacle wrote: post #4

I wonder how the use of static world-prior information is considered here. Or said in a different way: What stops me from supplying a hard-coded pre-compressed file to circumvent their RAM and HDD limits?

It's stopped by the fact that technically quantity that's being measured in the specific rules (http://prize.hutter1.net/hrules.htm) is size of the compression program plus the size of the compressed output ("zero-input decompressor" which is code+data that can produce the uncompressed output), so you can use static world-prior information but that counts against you twice as it has to be contained both in the "compressor" and the "decompressor".

For most cases the size of the "compressor" is irrelevantly tiny compared to the data, but if your "compressor" is a hard-coded pre-compressed file which it copies to the "decompressor" then the few percent of gains don't outweigh the fact that you've just doubled the size that's scored. It could be useful to include hardcoded priors iff they are very slow to compute but can be expressed in a small amount of storage.

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#13

Post by MultiverseUI »

isaacfrond wrote: post #5

I'd be interested to see lossy text compression. That would also give much more scope for AI methods. It turned out that recently Stable Diffusion gives excellent compression https://pub.towardsai.net/stable-diffusion-based-image-compr... Funny thing is, on a per pixel basis, Stable Diffusion and jpeg are just as good. Stable diffusion looks much better, though. The reason is that when Stable diffusion lacks the info…

For classical compression benchmarking, you need to include the size of the decoder; GPT can already recite most lyrics and speeches given a single line prompt.

The state of the art in neural language models was evaluated ˜5 years ago, and it was found that standard LSTM's do very well on text compression, when properly architectured and parametrized.

The main reason for excluding lossy text compression in these tests, is that there is no clear path around requiring a panel of human judges, and that evaluation now is subjective (instead of objective).

Perhaps a different route would be to task an AI to compress Wikipedia into a (graph) knowledge base, and then test these AIs on correctly answering "multiple choice"-questions. But then intelligence becomes a proxy for measuring compression, instead of here, where compression is chosen as a proxy for measuring intelligence.

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#14

Post by abriosi »

isaacfrond wrote: post #5

I'd be interested to see lossy text compression. That would also give much more scope for AI methods. It turned out that recently Stable Diffusion gives excellent compression https://pub.towardsai.net/stable-diffusion-based-image-compr... Funny thing is, on a per pixel basis, Stable Diffusion and jpeg are just as good. Stable diffusion looks much better, though. The reason is that when Stable diffusion lacks the info…

It has been tried

https://bellard.org/libnc/gpt2tc.html

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#15

Post by bigyikes »

acadapter wrote: post #10
isaacfrond wrote: post #5

I'd be interested to see lossy text compression. That would also give much more scope for AI methods. It turned out that recently Stable Diffusion gives excellent compression https://pub.towardsai.net/stable-diffusion-based-image-compr... Funny thing is, on a per pixel basis, Stable Diffusion and jpeg are just as good. Stable diffusion looks much better, though. The reason is that when Stable diffusion lacks the info…

I hope that lossy text compression never becomes popular, it should ideally remain a technological curiosity. Think of all the damage that can be caused by incompetence combined with usage of lossy text compression in the wrong places.

Human brains have been performing lossy text compression for thousands of years. I think we’ll be ok.

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#16

Post by woliveirajr »

unnah wrote: post #7
isaacfrond wrote: post #5

I'd be interested to see lossy text compression. That would also give much more scope for AI methods. It turned out that recently Stable Diffusion gives excellent compression https://pub.towardsai.net/stable-diffusion-based-image-compr... Funny thing is, on a per pixel basis, Stable Diffusion and jpeg are just as good. Stable diffusion looks much better, though. The reason is that when Stable diffusion lacks the info…

A lossy text compressor can be converted into a lossless compressor quite easily by encoding the remaining differences between the lossy reproduction and the original text. The more accurate your lossy compressor is, the less additional information you need to encode the differences. You can get even better results if your lossy compressor is probabilistic, and can compute approximate probabilities for different text…

Instead of storing the differences, use a spell checker. The better written the text with proper grammar rules, the easier for getting it right with a spell checker. Yes, patented.

Re: Text Compression as a Test for Artificial Intelligence (1999) [pdf]

#17

Post by fxtentacle »

keithalewis wrote: post #8
fxtentacle wrote: post #4

I wonder how the use of static world-prior information is considered here. Or said in a different way: What stops me from supplying a hard-coded pre-compressed file to circumvent their RAM and HDD limits?

Quoted post unavailable.

This reads a bit unfriendly.

But anyway I read their FAQ and it's allowed to hand in offline-prepared data but size*2 will be added to your result so its typically a disadvantage unless your statically prepared data has an exceptionally high knowledge content.

Post reply on HN