A cartel of influential datasets are dominating machine learning research • Hacker News Board

Re: A cartel of influential datasets are dominating machine learning research

#61

Post by YeGoblynQueenne » Mon, Dec 06, 2021, 7:25 PM UTC

durovo wrote: post #56

lumost wrote: post #35

Earlier quoted context omitted.

It depends, feedback is rarely as polite as "I noticed you used this other dataset". The feedback would probably look like. - "Nice paper, however the results are not relevant to current research due to the use of X dataset rather than Y or Z datasets score 2/5 do not accept." - "Nice paper, however the results are of unknown quality due to the use of X dataset 3/5 recommend poster track". In fact I'd generally say t…

If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper. Especially if a proposed model is being compared against models that were trained on those standard datasets. If you are establishing new baselines using those same models on your non-standard dataset, then one would expect you to put in a good amount of ef…

>> If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper.

Unfortunately that just adds more work for the reviewer, which is a motive for many reviewers to scrap the paper so they don't have to do the extra work.

That sounds mean, so I will quote (yet again) Geoff Hinton on things that "make the brain hurt":

GH: One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas.

Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad.

https://www.wired.com/story/googles-ai-guru-computers-think-...

Basically a new dataset is like a new idea: it makes the brain hurt, for the overburdened experienced researcher or inexperienced younger researcher alike. Testing a new approach on a new dataset? That makes brain go boom.

Which is a funny state of affairs. Not so long ago it used to be that one sure-fire way to make a significant contribution that would give your paper a leg up over the competition was to create a new dataset. I was advised as much at the start of my PhD (four ish years ago). Seems like this has already changed.

Re: A cartel of influential datasets are dominating machine learning research

#62

Post by Derbasti » Mon, Dec 06, 2021, 7:37 PM UTC

fock wrote: post #44

Derbasti wrote: post #41

This is what my dissertation was about, in a different field: Everyone using the same datasets, even though they are severely flawed. As it turned out, differences between datasets proved significantly larger (by a big margin) than differences between algorithms. And the most popular datasets in fact included biases and eccentricities that were bound to cause such problems. https://bastibe.github.io/Dissertation-Webs…

thank you! I also found the same thing during my PhD and it took a basic-pay-topublish paper and 2 years to get my supervisor to agree that this is not really going anywhere with publications and switch to a field, where I am more comfortable with producing publications. Essentially my field has 1 big dataset, and then domain experts "improve" models by creating their own data (200 samples) and then claim a "novel" m…

Funny that you would mention trying to publish such findings in journals. I tried... more than two times, as well. Rejected with the most spurious of claims, despite my presenting evidence disproving them. Frankly, these journal submissions were the most frustrating and trying experiences in my professional life.

Oh well. I'm glad I got my degree, and could leave academia relatively unharmed.

Now I work in AI (engineering!), where most science is highschool-level "hey, I tried stuff, and things happened. Dunno why, though". It's just ridiculous.

Re: A cartel of influential datasets are dominating machine learning research

#63

Post by xiaolingxiao » Mon, Dec 06, 2021, 8:38 PM UTC

YeGoblynQueenne wrote: post #60

>> According to the paper, Computer Vision research is notably more affected by the syndrome it outlines than other sectors, with the authors noting that Natural Language Processing (NLP) research is far less affected. The authors suggest that this could be because NLP communities are ‘more coherent’ and larger in size, and because NLP datasets are more accessible and easier to curate, as well as being smaller and le…

Your suspicion is correct. I worked on such a dataset paper and worked to "beat" other methods on well-accepted benchmarks with dubious accuracy scores. One fundamental issue is that outside of POS tagging, there isn't a notion of empirical truth to measure against, only a small sample of what a "normal person would think." This stands in contrast to computer vision, whereby in a task such as monocular depth perception from a single frame, you can always measure against a Lidar-scanned depth map. The system can still overfit on the benchmark and they do, but at least the baseline truth itself is not in dispute. But questions such as: is this the "appropriate" response to a query is too open to interpretation.

Re: A cartel of influential datasets are dominating machine learning research

#64

Post by 908B64B197 » Mon, Dec 06, 2021, 8:55 PM UTC

michaelt wrote: post #47

omarhaneef wrote: post #25

A little confused: Are these big "dominant" institutions charging for this data? No, the spend a lot of resources putting them together and give them away free. Are they preventing others from giving away data? No, but it costs a lot and they bear that cost. Are they forcing smaller institutions to use their data? No. Its just a free resource they offer. Do they get the grants themselves because they have some kind o…

> So they collect this data, vet it, propose their use for benchmarks and give it away free. What is the complaint? It's well known that neural networks can easily inherit biases from their training data. It's also well known that datasets generated by western universities are widely used in training and evaluating neural networks. If my training set is full of pictures of Stanford CS undergraduates, I could end up w…

> It's well known that neural networks can easily inherit biases from their training data. It's also well known that datasets generated by western universities are widely used in training and evaluating neural networks. If my training set is full of pictures of Stanford CS undergraduates, I could end up with a computational photography system that makes everyone look like Stanford CS undergraduates, or a historical photo colourisation system that makes everyone look like Stanford CS undergraduates, or a self driving car pedestrian tracking system that expects 90% of pedestrians to look like Stanford CS undergraduates.

Why then aren't foreign universities/companies simply... building their own datasets?

Re: A cartel of influential datasets are dominating machine learning research

#65

Post by durovo » Mon, Dec 06, 2021, 9:28 PM UTC

YeGoblynQueenne wrote: post #61

durovo wrote: post #56

Earlier quoted context omitted.

If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper. Especially if a proposed model is being compared against models that were trained on those standard datasets. If you are establishing new baselines using those same models on your non-standard dataset, then one would expect you to put in a good amount of ef…

>> If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper. Unfortunately that just adds more work for the reviewer, which is a motive for many reviewers to scrap the paper so they don't have to do the extra work. That sounds mean, so I will quote (yet again) Geoff Hinton on things that "make the brain hurt": GH:…

You might be right here. My comment was more of my expectation as a reader on what should be present in such a paper.

Re: A cartel of influential datasets are dominating machine learning research

#66

Post by gopher_space » Mon, Dec 06, 2021, 9:42 PM UTC

sea-shunned wrote: post #38

carschno wrote: post #10

Too bad they don't cite the paper "The Benchmark Lottery" (M. Dehghani et al, 2021) ( https://arxiv.org/abs/2107.07002 ) > The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods. This paper proposes the notion of "a benchmark lottery" that describes the overall fragility of the ML benchmarking process. The benchma…

It's also a shame that this paper in turn doesn't cite "Testing Heuristics: We Have It All Wrong" (J. N. Hooker, 1995) ( http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.71.... ), which discusses these issues in much the same way. It's good to see in "The Benchmark Lottery", however, that they look more into specific tasks and their algorithmic rankings and provide some sound recommendations. One thing that I…

I've been looking at parsing paragraph structure and have started thinking about a conceptual mechanical turk/e e cummings line in the sand where it's just going to be easier to pay some kid with a cell phone to read words for you. The working implementations I've seen are heavily tied to domain and need to nail down language, which isn't really a thing.

Quantification is fascinating, it seems to be something I take for granted until I actually want to make decisions. It's like I'm constantly trying to forget that analog and digital are two totally separate concepts. I wouldn't really recommend reading Castaneda to anyone but he describes people living comfortably with mutually exclusive ideas in their head walled off by context, and I'd like that sort of understanding.

Re: A cartel of influential datasets are dominating machine learning research

#67

Post by ansgri » Mon, Dec 06, 2021, 10:28 PM UTC

whimsicalism wrote: post #43

> Among their findings – based on core data from the Facebook-led community project Papers With Code (PWC) Oh boy. PWC is not even close to a representative sample of what datasets are being used in papers. It's also often out of date.

When you cannot analyze every paper ever published you need to have some relevant criterion for inclusion -- and PWC is somewhat of a GitHub of AI science (at least in early days of it) -- there may be multiple collections, but PWC is by far the most accessible and thus reasonably captures current and emerging trends some dataset-heavy research fields.

Re: A cartel of influential datasets are dominating machine learning research

#68

Post by dontreact » Mon, Dec 06, 2021, 11:39 PM UTC

sdenton4 wrote: post #57

dontreact wrote: post #54

Earlier quoted context omitted.

I think it's super important to separate the following two situations, both of which I suppose are fair to call overfitting Situation A: Models are slightly overfit to some portions of the test set. But the following holds IF PerformanceOnBenchmark(Model A) > PerformanceOnBenchmark(Model B) Then PerformanceOnSimilarDaset(Model A) > PerformanceOnSimilarDataset(Model B) Therefore progress on the benchmark is predictive…

>> "This almost always happen if you train a deep neural network long enough: train performance goes up, but test performance goes down." This is a problem that is more common for classification problems, I think. Generative and self-supervised models (trained with augmentation) tend to just get better forever (with some asymptote) because memorization isn't a viable strategy. I personally think image classification…

Nearly all useful machine learning is supervised, still. And if you are using a neural network, it will eventually memorize. This is fine though, we have early stopping :)

Re: A cartel of influential datasets are dominating machine learning research

#69

Post by cs702 » Mon, Dec 06, 2021, 11:51 PM UTC

The title is linkbait: There is no cartel.

The word cartel implies collusion. There is no collusion.

Using the word "cartel" in this case is not only wrong, but also... insulting to all the hardworking individuals who have toiled away in obscurity to compile, clean-up, label, package, and publish influential datasets.

Despite the terrible title, the OP is worth a read. It summarizes a recent research paper, also worth a read, about the disproportionate popularity in ML research of a relatively small number of datasets sponsored by and produced at elite institutions (e.g., ImageNet for visual recognition):

https://openreview.net/forum?id=zNQBIBKJRkd

Surely we can find ways to address the winner-take-all dynamics of benchmark popularity without unfairly accusing anyone of running a cartel.

Re: A cartel of influential datasets are dominating machine learning research

#70

Post by mrdrozdov » Tue, Dec 07, 2021, 12:03 AM UTC

This paper the article refers to is fantastic! I think it's a work most in ML research should become familiar with. And if you believe in the power of benchmarks and data, then this holds even more true. Investing in diversity in datasets is likely an impactful way to make progress in AI/ML.

Minor typo in this article...

ARTICLE: Among their findings – based on core data from the Facebook-led community project Papers With Code (PWC) – the authors contend that ‘widely-used datasets are introduced by only a handful of elite institutions’, and that this ‘consolidation’ has increased to 80% in recent years.

...but right after, they quote the paper and clearly it is 50% not 80%. See the quote from the paper:

PAPER: ‘[We] find that there is increasing inequality in dataset usage globally, and that more than 50% of all dataset usages in our sample of 43,140 corresponded to datasets introduced by twelve elite, primarily Western, institutions.’

...and the article is leaving out this relevant quote from the paper:

PAPER: Moreover, this concentration on elite institutions as measured through Gini has increased to over 0.80 in recent years (Figure 3 right red). This trend is also observed in Gini concentration on datasets in PWC more generally (Figure 3 right black).

...and in general the article is right that inequality is increasing over time, but Gini is a specific metric to measure inequality, and 0.80 is not the same as 80% inequality.