Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

78% MNIST accuracy using GZIP in under 10 lines of code

jakobs.dev

Post reply on HN
Print view View original

21–30 of 141 posts

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#21

Post by unoti »

grandma_tea wrote: post #20
GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.

MNIST is pretty good as a proof of concept. While I agree it's trivial, I see MNIST as being more like how when you're making a toy programming language, the first thing you do with it is writing a recursive Fibonacci function.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#22

Post by dmazzoni »

GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

That blog page is not showing state-of-the-art results, it's just taking relatively naive SVM implementations and comparing them.

The original research paper that introduced the MNIST data set was getting around 98% accuracy, and today neural nets are getting 99.87% accuracy.

https://paperswithcode.com/sota/image-classification-on-mnis...

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#23

Post by [deleted] »

tasty_freeze wrote: post #18
hinkley wrote: post #12

Earlier quoted context omitted.

If you can’t hear my eyes rolling from here I’ll do it again, harder… how bout now? Optical Character Recognition. Very old, and very infamous attempts at replacing humans with software. Secretaries and executives know what OCR means.

I believe you missed the point, though I could be wrong. Person A complains about MNIST not being defined. Person B was sending them up for complaining about not defining the meaning of acroynms and then using one they don't define.

[deleted]

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#24

Post by dmazzoni »

grandma_tea wrote: post #20
GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.

It's still a great data set because it's real-world, useful, high-quality, and an excellent size (~60k examples), and the "correct" classification isn't very subjective at all - humans can easily get 99% accuracy.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#25

Post by jszymborski »

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days.

https://github.com/lmcinnes/umap_paper_notebooks/blob/master...

EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#26

Post by sundarurfriend »

tdr2d wrote: post #11

Obviously, the code may be elegant and compact, 78% accuracy is considered very very bad for MNIST. A dummy model written with Tensorflow easilly reaches 90% accuracy. The best models ranked at 99,87%, see the benchmark : https://paperswithcode.com/sota/image-classification-on-mnis...

The point is not to have "elegant and compact" code, this is meant to be a fun curiosity, and doing it in 10 lines is just an additional layer of challenge for the heck of it.

The interesting thing is not in whether GZip can achieve SOTA, it's that it can do a decent job at all. (The interesting thing is not in whether the bear can recreate Mozart exactly, it's that it can play the piano at all.)

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#28

Post by p1esk »

tjungblut wrote: post #15

I don't immediately find it, but couple of years back there was a "meta-feature" which was the size of the MNIST image. I think that scored about 90'ish % accurate results on its own - without even looking at the image.

What do you mean by “size”? Gzipped size? If you simply look at how dark a Mnist image is (count the percentage of dark pixels) you’ll get about 20% accuracy, which is twice better than random guess but a long way from 90’ish %.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#29

Post by ipython »

hinkley wrote: post #19
vren wrote: post #16

Earlier quoted context omitted.

Optical character recognition... Has been pretty widespread tech since ~2016 or so.

Oh my ex and some acquaintances were bitching about it in 1999. I recall a guy with good transcription skills starting a long thread on… must have been slashdot? About how it was faster for him to transcribe the numbers again than to verify the OCR was correct, but he couldn’t tell his boss that and what should he do? There’s also a famous case where the compression algorithm in a copy/fax machine had a bug in a pred…

you're referring to the Xerox Workcenter: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#30

Post by hinkley »

tasty_freeze wrote: post #18
hinkley wrote: post #12

Earlier quoted context omitted.

If you can’t hear my eyes rolling from here I’ll do it again, harder… how bout now? Optical Character Recognition. Very old, and very infamous attempts at replacing humans with software. Secretaries and executives know what OCR means.

I believe you missed the point, though I could be wrong. Person A complains about MNIST not being defined. Person B was sending them up for complaining about not defining the meaning of acroynms and then using one they don't define.

No I got it. I just don’t want it.
Post reply on HN