Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

71–80 of 307 posts

Re: There are no new ideas in AI, only new datasets

#71

Post by beefnugs »

NBJack wrote: post #40
tough wrote: post #39

Earlier quoted context omitted.

I think the problem is we train models to pattern match, not to learn or reason about world models

In other words, they learn the game, not how to play games .

yeahhhh why isnt there a training structure where you play 5000 games, and the reward function is based on doing well in all of them?

I guess its a totaly different level of control: instead of immediately choosing a certain button to press, you need to set longer term goals. "press whatever sequence over this time i need to do to end up closer to this result"

There is some kind of nested multidimensional thing to train on here instead of immediate limited choices

Re: There are no new ideas in AI, only new datasets

#72

Post by alganet »

Dataset? That's so 2000s.

Each crawl on the internet is actually a discrete chunk of a more abstractly defined, constant influx of information streams. Let's call them rivers (it's a big stream).

These rivers can dry up, present seasonal shifts, be poisoned, be barraged.

It will never "get there" and gather enough data to "be done".

--

Regarding "new ideas in AI", I think there could be. But this whole thing is not about AI anymore.

Re: There are no new ideas in AI, only new datasets

#73

Post by varjag »

justanotherjoe wrote: post #68
EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

I don't get why people are so invested in framing it this way. I'm sure there are ways to do the stated objective. John Carmack isn't even an AI guy why is he suddenly the standard.

What in your opinion constitutes an AI guy?

Re: There are no new ideas in AI, only new datasets

#74

Post by nomel »

Swizec wrote: post #19
voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

> The real frontier of AI lies in the messy, rich, sensory world where people live. We’ll need new hardware (sensors), new data representations (beyond tokens), and new ways to train models that grow understanding from experience, not just patterns. Like Dr. Who said: DALEKs aren't brains in a machine, they are the machine! Same is true for humans. We really are the whole body, we're not just driving it around.

There are many people who mentally developed while paralyzed that literally drive around their bodies via motorized wheelchair. I don't think there's any evidence that a brain couldn't exist or develop in a jar, given only the inputs modern AI now has (text, video, audio).

Re: There are no new ideas in AI, only new datasets

#75

Post by wizzwizz4 »

voxleone wrote: post #55
dinfinity wrote: post #36

Earlier quoted context omitted.

> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I respectfully disagree. Touch gives pretty cool skills, but language, video and audio are all that are needed for all online interactions . We use touch for typing and pointing, but that is only because we don't have a more efficient and effective interface. Now I'm not saying t…

> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I probably made a mistake when i asserted that -- should have thought it over. Vision is evolutionarily older and more “primitive”, while language is uniquely human [or maybe, more broadly, primate, cetacean, cephalopod, avian...] symbolic, and abstract — arguably a different ord…

People who lack one of those senses, or even two of them, tend to do just fine.

Re: There are no new ideas in AI, only new datasets

#76

Post by nomel »

nyrulez wrote: post #47

Things haven't changed much in terms of truly new ideas since electricity was invented. Everything else is just applications on top of that. Make the electrons flow in a different way and you get a different outcome.

> Make the electrons flow in a different way and you get a different outcome.

This happens to be the basis of every aspect of our biology.

Re: There are no new ideas in AI, only new datasets

#77

Post by qaq »

justanotherjoe wrote: post #68
EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

I don't get why people are so invested in framing it this way. I'm sure there are ways to do the stated objective. John Carmack isn't even an AI guy why is he suddenly the standard.

Keen includes researchers like Richard Sutton, Joseph Modayil etc. Also John has being doing it full time for almost 5 years now so given his background and aptitude for learning I would imaging by this time he is more of an AI guy then a fairly large percentage of AI PhDs.

Re: There are no new ideas in AI, only new datasets

#78

Post by sieabahlpark »

justanotherjoe wrote: post #68
EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

I don't get why people are so invested in framing it this way. I'm sure there are ways to do the stated objective. John Carmack isn't even an AI guy why is he suddenly the standard.

[dead]

Re: There are no new ideas in AI, only new datasets

#79

Post by veqz »

ferguess_k wrote: post #66
tough wrote: post #39

Earlier quoted context omitted.

I think the problem is we train models to pattern match, not to learn or reason about world models

I kinda think I'm more or less the same...OK maybe we have different definitions of "pattern matching".

It's Plato's cave:

We train the models on what are basically shadows, and they learn how to pattern match the shadows.

But the shadows are only depictions of the real world, and the LLMs never learn about that.

Re: There are no new ideas in AI, only new datasets

#80

Post by nomel »

chasd00 wrote: post #52
voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

> Language and vision are just the beginning.. Based on the architectures we have they may also be the ending. There’s been a lot of news in the past couple years about LLMs but has there been any breakthroughs making headlines anywhere else in AI?

Two Nobel prizes in chemistry: https://www.nature.com/articles/s41746-024-01345-9
Post reply on HN