No "Zero-Shot" Without Exponential Data

We are very clearly on that same path again. What leads to the conclusion that another winter is coming. But even the fact that people are talking about it is evidence it's not here yet, and as always with political phenomena, there's no guarantee history will repeat.

Anyway, none of it means people will stop applying their knowledge or studying AI. The entire thing happens on funding and PR, and the world is not entirely controlled by those two.

1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps.

2. Create (simulated) environmental data.

3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair.

There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse.

Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)

If I understand it correctly, that seems to be exactly what this paper is suggesting.

Scoring high on the bar exam is pretty trivial for AI - the data needed for that is fairly generic and widely available on the internet. It requires you to demonstrate a relatively basic understanding of the concepts by answering a bunch of multiple-choice answers. If anything, I'd expect AI to have a perfect score.

Like I said, such an AI is not entirely useless. You can replace quite a few legal assistants with that, and I bet it could be used to create first drafts or to expand a core concept into a full legal argument. There is plenty of money to be made there, and it's going to make an awful lot of people jobless. But that's just replacing more-trivial jobs with automation, it doesn't add anything novel to society.

On the other hand, the actual difficult work involves being able to come up with completely novel concepts, and being able to expand upon some obscure but crucial stuff few people have ever heard about. Current models simply aren't capable of that, and the results achieved here with multimodal models suggests that they never will. We risk getting stuck with models which can do some trivial work, but silently produce complete garbage when you ask them to do anything providing substantial value.

Eventually AGI's will need to learn by experimentation as we do, but in the meantime ability to well predict a potential training sample could be used to decide whether to add it to training set or not. At the moment it seems the main emphasis is on a combination of multi-modality (esp. video) and synthetic data where one generation of LLM generates tailored training samples for the next generation. I guess this synthetic data allows a more selective acquisition of knowledge than just adding surprising texts found in the wild.