Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

78% MNIST accuracy using GZIP in under 10 lines of code

jakobs.dev

Post reply on HN
Print view View original

51–60 of 141 posts

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#51

Post by IKantRead »

GaggiX wrote: post #17

For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…

Thanks for posting this.

Most people don't realize that Logistic regression can get ~90% accuracy on MNIST.

As a big fan of starting with simple models first and adding complexity later, I've frequently been told that "logistic regression won't work!" for problems where it can in fact perform excellent.

When faced with this resistance to logistic regression I'll often ask what they think the baseline performance of it would be on MNIST. The guesses I hear most often are 20-30%.

People, even machine learning people, often don't realize the rapidly diminishing returns you get for adding a lot of complexity in your models. Sometimes it's worth it, but it's always good to start simple first.

It's also been my experience that if you don't get good performance with a simple model, you are very unlikely to get great performance from a more complex one.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#52

Post by hinkley »

ElectricalUnion wrote: post #34
hinkley wrote: post #19

Earlier quoted context omitted.

Oh my ex and some acquaintances were bitching about it in 1999. I recall a guy with good transcription skills starting a long thread on… must have been slashdot? About how it was faster for him to transcribe the numbers again than to verify the OCR was correct, but he couldn’t tell his boss that and what should he do? There’s also a famous case where the compression algorithm in a copy/fax machine had a bug in a pred…

> I believe it was spitting out perfectly formed zeroes where another digit was in the original. Yikes. JBIG2 lossy compression. Covered in another hn story: https://news.ycombinator.com/item?id=29223815 From the story and the comments: > This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned i…

I know technically you are right, but I feel like if you sat down and described the high level features and goals of that image compression (to compress text to a ridiculous degree) and OCR , you’d be hard pressed to say which list is which unless they gave it away with jargon words. My brain compressed it to “OCR” and filed it there.

It’s like Jim Gaffigan’s joke about working in a TexMex restaurant in the northern Midwest. It’s a tortilla, cheese, meat and beans… I tell you what I’ll just bring you something.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#53

Post by hinkley »

ipython wrote: post #29
hinkley wrote: post #19

Earlier quoted context omitted.

Oh my ex and some acquaintances were bitching about it in 1999. I recall a guy with good transcription skills starting a long thread on… must have been slashdot? About how it was faster for him to transcribe the numbers again than to verify the OCR was correct, but he couldn’t tell his boss that and what should he do? There’s also a famous case where the compression algorithm in a copy/fax machine had a bug in a pred…

you're referring to the Xerox Workcenter: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

YES. God what a nightmare. Can you imagine all of the arguments and threats of lawsuits that bug caused for small and medium businesses? You changed the contract after we had a verbal agreement and tricked me into signing!

I wonder if any executive assistants got fired over that. Surely.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#54

Post by tysam_and »

If you'd like to play around with MNIST yourself, I wrote a PyTorch training implementation that gets ~99.45%+ val accuracy in https://github.com/tysam-code/hlb-CIFAR10

It's originally kitted for CIFAR10, but I've found the parameters to be quite general. The code is very easy to read and well-commented, and is a great starting place for exploration.

Min-cut deltas to run MNIST:

  .datasets.CIFAR10(' -> .datasets.MNIST(' (both occurences)

  'whiten': Conv(3, -> 'whiten': Conv(1,

  crop_size = 28 - > `crop_size = 28
Compute for the project funded by Carter Brown and Daniel Gross, my appreciation to them both for helping make this possible. Their support via encouragement has been very helpful as well. <3 :)

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#55

Post by itishappy »

hinkley wrote: post #40
telotortium wrote: post #33

Earlier quoted context omitted.

Fortunately MNIST is a pretty distinctive Google search term. At most you would need to search for (MNIST machine learning).

Are you running google searches on https://news.ycombinator.com/news before clicking on stuff? Or expecting an executive summary. I think you can guess which one I’m doing. And which one I believe is normal human behavior.

No need to guess, you requested a summary:

> you’ve missed the most important hyperlink: what the fuck is a MNIST?

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#56

Post by itishappy »

hinkley wrote: post #14
telotortium wrote: post #6

Earlier quoted context omitted.

MNIST (and OCR) are acronyms that are so well-known to anyone who has taken any kind of intro to ML class that there's no more need to define them in a short blog post than there would be for us to define HTML. I learned about MNIST in 2008, and I was just taking a class on numerical methods as a physics major in Matlab where MNIST was just one project.

This isn’t a forum about machine learning, though. It’s a general forum of geek news. I could talk to you all day about compression. Gzip plus MNIST rings no bells.

The fuck is a GZIP? ;)

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#57

Post by wenc »

sundarurfriend wrote: post #32
jszymborski wrote: post #25

In fairness you can run MNIST through UMAP and get near perfect seperation. I'm of the belief that you have to try pretty hard not to do well on MNIST these days. https://github.com/lmcinnes/umap_paper_notebooks/blob/master... EDIT: I should add, unless it isn't clear, that we really should retire the dataset. Something like the QuickDraw dataset makes a lot more sense to me.

Y'all are making the (rude) person below complaining about acronyms look reasonable. The repository doesn't define UMAP either, but if you believe ChatGPT it is: > UMAP, which stands for Uniform Manifold Approximation and Projection, is a dimensionality reduction technique and data visualization method commonly used in machine learning and data analysis.

We work in our own ecological niches which sometimes entails having a specialized language that allows us to be precise when referring to things.

I sometimes see HN comments complaining that acronyms and terms are dropped with no explanation. These comments come across as entitled to me — instead of complaining, why not be curious and ask “what does that mean in this context?”

Imagine someone dropping in here and complaining, “the article sucks because it uses the word ‘compiler’ and assumes we all know what a ‘compiler’ is.” This is what it sounds like to those of us to work in specialized fields.

Instead if someone said, “I’m new to this, could someone help me understand what a ‘compiler’ is?”, it demonstrates curiosity and people are more inclined to explain.

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#58

Post by tysam_and »

13of40 wrote: post #50
tjungblut wrote: post #15

I don't immediately find it, but couple of years back there was a "meta-feature" which was the size of the MNIST image. I think that scored about 90'ish % accurate results on its own - without even looking at the image.

A few years back I worked on a project that involved fingerprinting screenshots of web pages, and compressed image size was pretty much as good as any fingerprinting method we could come up with for comparing the similarity between them.

Makes sense, considering that's not too terribly far off from what the KL divergence does

Re: 78% MNIST accuracy using GZIP in under 10 lines of code

#59

Post by tysam_and »

yvdriess wrote: post #42
p1esk wrote: post #28

Earlier quoted context omitted.

What do you mean by “size”? Gzipped size? If you simply look at how dark a Mnist image is (count the percentage of dark pixels) you’ll get about 20% accuracy, which is twice better than random guess but a long way from 90’ish %.

What do you mean with accuracy here? Usually 50% accuracy means cointoss, meaning 20% accuracy is equal to 80% accuracy, which is better than the article's 78% and not that far from 90%.

"One simple trick to beat the statistical odds...."
Post reply on HN