Additionally, try flipping your images and averaging the size before comparing the distance between them. I'd expect a boost of about 78% -> 84% or so, based on how this typically works as TTA.
78% MNIST accuracy using GZIP in under 10 lines of code
61–70 of 141 posts
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#62Earlier quoted context omitted.
This isn’t a forum about machine learning, though. It’s a general forum of geek news. I could talk to you all day about compression. Gzip plus MNIST rings no bells.
The fuck is a GZIP? ;)
Goddamnit where is the spray bottle?
Psst psst psst. NO. Bad kitty.
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#63General purpose compressors and information distance measures have become super interesting to me while I've been investigating alternative language models.
I've been playing around with an attention mechanism that combines the idea of using normalized compression distance (gzip) with discrete convolution between candidate sequences (sliding window of N bytes over each). Another round of normalization over the convolution outputs - accommodating varying lengths - allows for us to compare candidate sequences for relevant information on equal grounds.
The NCD formula I am using right now:
NCD(x,y) = (C(xy) - MIN(C(x),C(y))) / MAX(C(x),C(y))
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#64For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…
While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.
Take a look at the EMNIST - Extended MNIST dataset. It has both digits and letters of the alphabet.
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#65"MNIST"? accuracy - but of what? what's this about?
MNIST is a classic image classification exercise - a dataset of 60,000 training images and 10,000 testing images where each image is a handwritten numeral as a 28x28 pixel grayscale image.
The challenge is to build a computer vision model that can tell which numeral each handwritten digit represents.
https://en.wikipedia.org/wiki/MNIST_database
78% accuracy on a solution is pretty bad, but achieving it just using GZIP is a very neat hack.
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#66For a comparison with others techniques: Linear SVC (best performance): 92 % SVC rbf (best performance): 96.4 % SVC poly (best performance): 94.5 % Logistic regression (prev assignment): 89 % Naive Bayes (prev assignment): 81 % From this blog page: https://dmkothari.github.io/Machine-Learning-Projects/SVM_wi... Also it seems from reading online articles that people are able to obtain much better results just by using…
While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.
It's a good benchmark because it's so trivial.
Sure it's not great at differentiating between SotA techniques, but it's very useful for sanity checks like this one.
Even for SotA models, it's still useful to verify that you can get greater than 98% accuracy on MNIST, before exploring larger, more complex bench marks.
It certainly shouldn't be the only benchmark but it's a great place to start iterating on ideas.
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#67"MNIST"? accuracy - but of what? what's this about?
MNIST is a classic image classification exercise - a dataset of 60,000 training images and 10,000 testing images where each image is a handwritten numeral as a 28x28 pixel grayscale image. The challenge is to build a computer vision model that can tell which numeral each handwritten digit represents. https://en.wikipedia.org/wiki/MNIST_database 78% accuracy on a solution is pretty bad, but achieving it just using GZI…
thank you
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#68Earlier quoted context omitted.
Fortunately MNIST is a pretty distinctive Google search term. At most you would need to search for (MNIST machine learning).
Are you running google searches on https://news.ycombinator.com/news before clicking on stuff? Or expecting an executive summary. I think you can guess which one I’m doing. And which one I believe is normal human behavior.
No, but if the article headline interested me enough to click into the article, I would Google MNIST if I didn't know it already (ML is full of acronyms for datasets, methods, etc. - can't expect blog posts to define all of them. If it were a paper that didn't define the term, I would be annoyed.)
Also, just because an acronym isn't used doesn't mean you'll understand the jargon. For example, also on the front page is a paper titled "Neurons in Large Language Models: Dead, N-Gram, Positional". No acronyms, but I would certainly need to Google (or actually read the paper, not just the abstract) to know what the 3 terms at the end means (well, I do know what a dead neuron is, but not the other two).
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#69Earlier quoted context omitted.
While it's cool that this works at all, I wish we would stop using MNIST as a benchmark given how trivial it is.
It's a good benchmark because it's so trivial. Sure it's not great at differentiating between SotA techniques, but it's very useful for sanity checks like this one. Even for SotA models, it's still useful to verify that you can get greater than 98% accuracy on MNIST, before exploring larger, more complex bench marks. It certainly shouldn't be the only benchmark but it's a great place to start iterating on ideas.
It's a bad benchmark because it's artificially clean. It's effectively a 2d dataset with no occlusions. So nearly everything you try on it will work, and many things you try on it won't scale to typical image problems. There are good 3d datasets with more realistic examples that are still fairly simplistic compared to the state of the art large datasets, but at least give you signal that your technique is robust to common problems in vision. MNIST is so simplistic that you encounter none of the typical problems in computer vision settings so it doesn't give you a good prediction of how good your technique is.
Re: 78% MNIST accuracy using GZIP in under 10 lines of code
#70Earlier quoted context omitted.
Are you running google searches on https://news.ycombinator.com/news before clicking on stuff? Or expecting an executive summary. I think you can guess which one I’m doing. And which one I believe is normal human behavior.
No need to guess, you requested a summary: > you’ve missed the most important hyperlink: what the fuck is a MNIST?
Two questions and you answered the more rhetorical one. Is that a no?
Anyway my main beef isn’t with this GitHub repo or the author, it’s that nobody remembers how to write a goddamned overview anymore. The point of tree structured data is making it cheap to backtrack when doing a semi random search. The overview is an important part of making hypertext work, particularly when it feeds through catch-all lists like a news site or a landing page for a wiki.
When a hyperlink is organically embedded into a paragraph, you can usually guess what it’s for, from the sentence and the context. When it’s just a title that is lost.