78% MNIST accuracy using GZIP in under 10 lines of code • Hacker News Board

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#11

Post by tdr2d » Wed, Sep 20, 2023, 3:14 PM UTC

Obviously, the code may be elegant and compact, 78% accuracy is considered very very bad for MNIST.

A dummy model written with Tensorflow easilly reaches 90% accuracy. The best models ranked at 99,87%, see the benchmark : https://paperswithcode.com/sota/image-classification-on-mnis...

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#12

Post by hinkley » Wed, Sep 20, 2023, 3:15 PM UTC

morelisp wrote: post #10

hinkley wrote: post #4

[flagged]

> Such as OCR. Such as what?

If you can’t hear my eyes rolling from here I’ll do it again, harder… how bout now?

Optical Character Recognition. Very old, and very infamous attempts at replacing humans with software. Secretaries and executives know what OCR means.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#13

Post by 3abiton » Wed, Sep 20, 2023, 3:15 PM UTC

Didn't it turn out that authors of that paper have made mistakes that catapulted their results to the top of the benchmark charts? I thought the theory was inconsistent after that incident. 78% accuracy from just GZIP is impressive.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#14

Post by hinkley » Wed, Sep 20, 2023, 3:17 PM UTC

telotortium wrote: post #6

hinkley wrote: post #4

[flagged]

MNIST (and OCR) are acronyms that are so well-known to anyone who has taken any kind of intro to ML class that there's no more need to define them in a short blog post than there would be for us to define HTML. I learned about MNIST in 2008, and I was just taking a class on numerical methods as a physics major in Matlab where MNIST was just one project.

This isn’t a forum about machine learning, though. It’s a general forum of geek news. I could talk to you all day about compression. Gzip plus MNIST rings no bells.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#15

Post by tjungblut » Wed, Sep 20, 2023, 3:18 PM UTC

I don't immediately find it, but couple of years back there was a "meta-feature" which was the size of the MNIST image. I think that scored about 90'ish % accurate results on its own - without even looking at the image.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#16

Post by vren » Wed, Sep 20, 2023, 3:18 PM UTC

morelisp wrote: post #10

hinkley wrote: post #4

[flagged]

> Such as OCR. Such as what?

Optical character recognition... Has been pretty widespread tech since ~2016 or so.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#17

Post by GaggiX » Wed, Sep 20, 2023, 3:21 PM UTC

For a comparison with others techniques:

Linear SVC (best performance): 92 %

SVC rbf (best performance): 96.4 %

SVC poly (best performance): 94.5 %

Logistic regression (prev assignment): 89 %

Naive Bayes (prev assignment): 81 %

From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi...

Also it seems from reading online articles that people are able to obtain much better results just by using K-NN, so I imagine that the author just made his job harder by using gzip but I could be wrong about this.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#18

Post by tasty_freeze » Wed, Sep 20, 2023, 3:24 PM UTC

hinkley wrote: post #12

morelisp wrote: post #10

Earlier quoted context omitted.

> Such as OCR. Such as what?

If you can’t hear my eyes rolling from here I’ll do it again, harder… how bout now? Optical Character Recognition. Very old, and very infamous attempts at replacing humans with software. Secretaries and executives know what OCR means.

I believe you missed the point, though I could be wrong.

Person A complains about MNIST not being defined. Person B was sending them up for complaining about not defining the meaning of acroynms and then using one they don't define.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#19

Post by hinkley » Wed, Sep 20, 2023, 3:27 PM UTC

vren wrote: post #16

morelisp wrote: post #10

Earlier quoted context omitted.

> Such as OCR. Such as what?

Optical character recognition... Has been pretty widespread tech since ~2016 or so.

Oh my ex and some acquaintances were bitching about it in 1999.

I recall a guy with good transcription skills starting a long thread on… must have been slashdot? About how it was faster for him to transcribe the numbers again than to verify the OCR was correct, but he couldn’t tell his boss that and what should he do?

There’s also a famous case where the compression algorithm in a copy/fax machine had a bug in a predefined dictionary that resulted in changing digits to other digits when making copies or faxes. Because they weren’t just compressing, they were trying to use a specialized OCR to do aggressive compression. I believe it was spitting out perfectly formed zeroes where another digit was in the original. Yikes.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#20

Post by grandma_tea » Wed, Sep 20, 2023, 3:29 PM UTC

GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.