78% MNIST accuracy using GZIP in under 10 lines of code • Hacker News Board

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#81

Post by burnished » Wed, Sep 20, 2023, 6:24 PM UTC

hinkley wrote: post #5

seeknotfind wrote: post #3

Earlier quoted context omitted.

original -> orthogonal? I like short code! It's like a joke "this is so easy, 10 loc" :)

The internet has turned into a game of “is it him or is it autocorrect?” I can’t count how many times I’ve looked up at a line of text to spot check it, type in a couple more letters, hit save without looking, and find later that I sound like I’m having a stroke. Never mind the times I’ve simply forgotten to look. Yes that’s a word, stop trying to replace it. (Five minutes ago I had to type “laissez-faire” three time…

I ended up turning auto correct off due to iOS being more aggressive with its corrections than my previous android phones (both of which frequently fail to recognize common words, and maybe both but certainly the android was absurdly brand aware).

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#82

Post by burnished » Wed, Sep 20, 2023, 6:25 PM UTC

Detrytus wrote: post #74

simonw wrote: post #65

Earlier quoted context omitted.

MNIST is a classic image classification exercise - a dataset of 60,000 training images and 10,000 testing images where each image is a handwritten numeral as a 28x28 pixel grayscale image. The challenge is to build a computer vision model that can tell which numeral each handwritten digit represents. https://en.wikipedia.org/wiki/MNIST_database 78% accuracy on a solution is pretty bad, but achieving it just using GZI…

What's the average human performance for this task?

I don't know off hand, but go take a look at the images - I would expect near 100%.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#83

Post by westurner » Wed, Sep 20, 2023, 6:29 PM UTC

GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

So there is a more optimal compression algorithm for the relation between the MNIST inputs and outputs!

Other models tend to add noise somewhere in there; what about feature engineering before the gzip? Maybe a gaussian blur and a convolution first and then deep learning for feature selection

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#84

Post by Frotag » Wed, Sep 20, 2023, 6:33 PM UTC

yvdriess wrote: post #42

p1esk wrote: post #28

Earlier quoted context omitted.

What do you mean by “size”? Gzipped size? If you simply look at how dark a Mnist image is (count the percentage of dark pixels) you’ll get about 20% accuracy, which is twice better than random guess but a long way from 90’ish %.

What do you mean with accuracy here? Usually 50% accuracy means cointoss, meaning 20% accuracy is equal to 80% accuracy, which is better than the article's 78% and not that far from 90%.

> meaning 20% accuracy is equal to 80% accuracy,

Only if your model is outputting a yes/no answer right? And that your definition of accuracy is "class with highest probability" (and not "N classes with highest prob")

If your dataset has more than 2 classes like MNIST, a super low accuracy only tells you to ignore the class the model guesses. It doesn't tell you which of the remaining classes is correct

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#85

Post by esafak » Wed, Sep 20, 2023, 6:47 PM UTC

a_wild_dandan wrote: post #73

esafak wrote: post #44

Earlier quoted context omitted.

The article emphasizes the wrong thing, in my view. The interesting part is that compression -- without learning a model -- can be used for classification. This raises the question of what other information-theoretic measures can be used; cheaper, lossy ones. To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review https://arxiv.org/abs/2304.09355\ *

An increasingly common refrain in machine learning is “intelligence is compression.” Folks who believe that might bristle at the distinction between learning and compression.

I doubt it because learned features already constitute lossy compression. The question is what kind of compression; lossy vs. lossless, learned vs. unlearned.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#86

Post by 0xDEF » Wed, Sep 20, 2023, 6:51 PM UTC

Has anyone tried how different compression algorithms compare when doing NCD classification?

gzip first does LZ77 and then Huffman coding. ANS is a more modern alternative to Huffman coding that can achieve higher compression ratios than Huffman coding.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#87

Post by tobeyey » Wed, Sep 20, 2023, 6:52 PM UTC

Replacing the NCD

  distances = [(compute_ncd(x1, x), label) for x, _, label in compressed_lengths]

with the Euclidean distance

  distances = [(np.sqrt(np.sum(np.square(x1-x))), label) for x, _, label in compressed_lengths]

gives you +15% test accuracy and saves you a lot of compute.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#88

Post by hinkley » Wed, Sep 20, 2023, 6:57 PM UTC

burnished wrote: post #81

hinkley wrote: post #5

Earlier quoted context omitted.

The internet has turned into a game of “is it him or is it autocorrect?” I can’t count how many times I’ve looked up at a line of text to spot check it, type in a couple more letters, hit save without looking, and find later that I sound like I’m having a stroke. Never mind the times I’ve simply forgotten to look. Yes that’s a word, stop trying to replace it. (Five minutes ago I had to type “laissez-faire” three time…

I ended up turning auto correct off due to iOS being more aggressive with its corrections than my previous android phones (both of which frequently fail to recognize common words, and maybe both but certainly the android was absurdly brand aware).

> brand aware

I got so tired of Mathematica italicizing its own name when I was in college so I figured out how to type it in two phases so it wouldn’t. It was my tiny little petty victory over Stephen Wolfram ruining math for me.

I know that struggle.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#89

Post by 0xfffafaCrash » Wed, Sep 20, 2023, 7:03 PM UTC

3abiton wrote: post #13

Didn't it turn out that authors of that paper have made mistakes that catapulted their results to the top of the benchmark charts? I thought the theory was inconsistent after that incident. 78% accuracy from just GZIP is impressive.

haven’t looked through this post yet but I think this is what you have in mind: https://kenschutte.com/gzip-knn-paper/

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#90

Post by Lerc » Wed, Sep 20, 2023, 7:06 PM UTC

I'm sure there was something like this mentioned in "Managing Gigabytes" or thereabouts. It might have been using bitwise arithmetic compression which makes sense for the problem at hand.