151–160 of 351 posts
Earlier quoted context omitted.
Did you even try the puzzles? They’re not particularly “unsolvable”.
ARC-AGI: "here are some pretty simple puzzles, we'll give you a million dollars to solve them!" Human: "They're quite challenging, this might be a trick to engage activity for the purpose of training models." skrebbel: "You're stupid".
This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…
Earlier quoted context omitted.
ARC-AGI: "here are some pretty simple puzzles, we'll give you a million dollars to solve them!" Human: "They're quite challenging, this might be a trick to engage activity for the purpose of training models." skrebbel: "You're stupid".
Did you try the puzzles?
Any details on how these tests were created? I.e. which kind of program was used for generation.
I think the ARC-AGI tasks was manually drawn with an early version of fchollet's editor. Recently Michael Hodel has reverse engineered 400 of the tasks, so more tasks can be generated. Interestingly it can generate python programs that solves the tasks too. https://github.com/michaelhodel/re-arc
Earlier quoted context omitted.
> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…
> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. I have kids so I'm presuming I'm allowed to have an opinion here. This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc. Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new fa…
Until they encounter a similar animal and get confused, at which point you understand the implicit heuristic they were relying on. (Eg. They confused a dairy cow as a zebra, which means their heuristic was a black-and-white quadrupedal)
Doesn't this seem remarkably close to how LLMs behave with one-shot or few-shot learning? I think there are a lot more similarities here than you give it credit for.
Also, I grew up in South Korea where early math education is highly prioritized (for better or for worse). I remember having to solve 2 dozen arithmetic problems every week after school with a private tutor. Yes, it was torture and I was miserable, but it did expose me to thousands more arithmetic questions than my American peers. All that misery paid off when I moved to the U.S. at the age of 12 and realized that my math level was 3-4 years above my peers. So yes, I think human intelligence accuracy also does improve with more training data.
This is amazing, and much needed. Thanks for organizing this. Makes me want to flex the programming muscle again.
This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…
> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…
This reminds of the story of Adam learning names, or how some languages can express a lot more in fewer words. And it makes sense that LLMs look intelligent to us.
My kid loves repeating the names of things he learned recently. For past few weeks, after learning 'spider' and 'snake' and 'dangerous' he keeps finding spiders around, no snakes so makes up snakes from curly drawn lines and tells us they are dangerous.
I think we learn fast because of stereo (3d) vision. I have no idea how these models learn and don't know if 3d vision will make multi model LLMs better and require exponentially less examples.
https://manifold.markets/JacobPfau/will-the-arcagi-grand-pri...
Chollet's argument is that LLMs just imitate and recombine patterns. This might be true if you're looking at LLMs in isolation, but when they chat with people something different happens. The system made of humans+LLMs is an AGI. It is no longer just a parrot, it ingests new information, gets guidance, feedback and is basically embodied in a chat room with human and tools. This scales for 200M users and 1 billion ses…
Pay no attention to the man behind the curtain.
This type of thinking would claim that mechanical turk is AGI, or perhaps that human+pen and paper is AGI. While they are great tools, that's not how I'd characterize them.