Viewing profile — liliumregale

I'm sorry, this article reads like AI slop. It has all the hallmarks: grandiose writing ("everything changed"), the classic "It wasn't X, it was Y" (about five times in the first m…

Regularization as a concept is taught in introductory ML classes. A simple example is called L2 regularization: you include in your loss function the sum of squares of the paramete…

We absolutely should treat it just like any other software tool.

Let's distinguish between papers and preprints, please. arXiv has contributed to a blurring of the distinction. The arXiv preprints are useful but should always be taken with a gra…

This repo is a joke, right? I'd be embarrassed peddling this as AGI. We're a long way from AGI existing at all. Even if you disagree, it's agreed upon that we're not there yet. For…

Google absolutely has their own internal models that do exactly this. It wouldn't surprise me if Microsoft indeed does have an internal Copilot that is trained on their data, but e…

Well…one that peeks at the test set labels. https://kenschutte.com/gzip-knn-paper2/

Further analysis shows that it doesn’t perform well at all—successes are tied to things like test set leakage. https://kenschutte.com/gzip-knn-paper2/ This paper isn’t any surprisi…

Yes - it's mentioned, but doesn't the framing below make it sound like they're still advocating for this paper? > In essence, it's advisable to take the paper’s reported figures wi…

The paper has recently been called into question for overestimating their performance relative to BERT: https://news.ycombinator.com/item?id=36758433 . Might be good for the blog's…

The title wordplay dates back to at least Drew McDermott's 1976 essay "Artificial Intelligence Meets Natural Stupidity" [0]. The intro is phenomenal. --- > As a field, artificial i…

By your first paragraph's argument, the semantics are in the Transformer, not the tokenizer. And yes, what they do helps on their two test tasks. I'm not disputing that. It's the f…

I was being generous - stemming is poor man's morphology. Empirically useful (ask the IR folks) but incredibly heuristic.

I'm going to add a contrarian take here: this preprint is not a research paper. While it's nice to see that there is an improvement here on their one task, this is not "semanticall…

Yep! Percy Liang in an interview with Chris Potts said he sees BERT and ELMo as foundation models.

This is a God (language)-of-the-gaps argument: we can't figure out this rarely language, but maybe we can figure out an entirely unattested language instead, and also learn the cor…

Is that a pun, because this was done by Pearson?