Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

78% MNIST accuracy using GZIP in under 10 lines of code

jakobs.dev

Post reply on HN
Print view View original

31–40 of 141 posts

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#31

Post by GaggiX »

dmazzoni wrote: post #22
GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

That blog page is not showing state-of-the-art results, it's just taking relatively naive SVM implementations and comparing them. The original research paper that introduced the MNIST data set was getting around 98% accuracy, and today neural nets are getting 99.87% accuracy. https://paperswithcode.com/sota/image-classification-on-mnis...

Yep I was showing others simple techniques, of couse a neural network would be much better at it.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#32

Post by sundarurfriend »

jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Y'all are making the (rude) person below complaining about acronyms look reasonable. The repository doesn't define UMAP either, but if you believe ChatGPT it is:

> UMAP, which stands for Uniform Manifold Approximation and Projection, is a dimensionality reduction technique and data visualization method commonly used in machine learning and data analysis.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#33

Post by telotortium »

hinkley wrote: post #14
telotortium wrote: post #6

Earlier quoted context omitted.

MNIST (and OCR) are acronyms that are so well-known to anyone who has taken any kind of intro to ML class that there's no more need to define them in a short blog post than there would be for us to define HTML. I learned about MNIST in 2008, and I was just taking a class on numerical methods as a physics major in Matlab where MNIST was just one project.

This isn’t a forum about machine learning, though. It’s a general forum of geek news. I could talk to you all day about compression. Gzip plus MNIST rings no bells.

Fortunately MNIST is a pretty distinctive Google search term. At most you would need to search for (MNIST machine learning).

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#34

Post by ElectricalUnion »

hinkley wrote: post #19
vren wrote: post #16

Earlier quoted context omitted.

Optical character recognition... Has been pretty widespread tech since ~2016 or so.

Oh my ex and some acquaintances were bitching about it in 1999. I recall a guy with good transcription skills starting a long thread on… must have been slashdot? About how it was faster for him to transcribe the numbers again than to verify the OCR was correct, but he couldn’t tell his boss that and what should he do? There’s also a famous case where the compression algorithm in a copy/fax machine had a bug in a pred…

> I believe it was spitting out perfectly formed zeroes where another digit was in the original. Yikes.

JBIG2 lossy compression. Covered in another hn story: https://news.ycombinator.com/item?id=29223815

From the story and the comments:

> This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned images look correct at first glance, even though numbers may actually be incorrect.

It's lossy compression turned to 11 that looks convincingly non-lossy.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#35

Post by tomrod »

sundarurfriend wrote: post #32
jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Y'all are making the (rude) person below complaining about acronyms look reasonable. The repository doesn't define UMAP either, but if you believe ChatGPT it is: > UMAP, which stands for Uniform Manifold Approximation and Projection, is a dimensionality reduction technique and data visualization method commonly used in machine learning and data analysis.

UMAP is as ubiquitous as T-SNE these days, though to be fair those on HN that focus on JVM, LLVM, NPM, or TLS may not know about UMAP/T-SNE.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#36

Post by js98 »

jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Author here: completely agree. I think it is not much of an achievement by itself, but it is interesting to see that it works!

I will add a comment/edit to the post once I am home to clarify the relative ease of solving MNIST

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#37

Post by jszymborski »

sundarurfriend wrote: post #32
jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Y'all are making the (rude) person below complaining about acronyms look reasonable. The repository doesn't define UMAP either, but if you believe ChatGPT it is: > UMAP, which stands for Uniform Manifold Approximation and Projection, is a dimensionality reduction technique and data visualization method commonly used in machine learning and data analysis.

It's[0] a non-linear dimensionality reduction technique in the vein of t-SNE[1] that is very well known in the field.

My two cents is that if your problem can be solved with something like UMAP + kNN[2], then you really shouldn't be using Deep Learning to solve it.

[0] https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduc...

[1] https://en.wikipedia.org/wiki/T-distributed_stochastic_neigh...

[2] https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#38

Post by jszymborski »

js98 wrote: post #36
jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Author here: completely agree. I think it is not much of an achievement by itself, but it is interesting to see that it works! I will add a comment/edit to the post once I am home to clarify the relative ease of solving MNIST

Sorry if my comment came off as dismissive of the post! I was just trying to frame the results for those unfamiliar with the properties of MNIST.

I think it is indeed interesting as it shows that Gzip can capture the same things that e.g. UMAP et co. are capturing when they acheive good scores on MNIST.

Also, I'll add, that even despite some of the suspicions people have cast on the Gzip results in other experiments, I'm bullish on the utility of reducing entropy for classification problems.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#40

Post by hinkley »

telotortium wrote: post #33
hinkley wrote: post #14

Earlier quoted context omitted.

This isn’t a forum about machine learning, though. It’s a general forum of geek news. I could talk to you all day about compression. Gzip plus MNIST rings no bells.

Fortunately MNIST is a pretty distinctive Google search term. At most you would need to search for (MNIST machine learning).

Are you running google searches on https://news.ycombinator.com/news before clicking on stuff? Or expecting an executive summary. I think you can guess which one I’m doing. And which one I believe is normal human behavior.
Post reply on HN