Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

21–30 of 307 posts

Re: There are no new ideas in AI, only new datasets

#21

Post by bcrosby95 »

jschveibinz wrote: post #7

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now. Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants. So AI can be an express elevator up to an army of giant's shoulders? It all de…

The article is discussing working in AI innovation vs focusing on getting more and better data. And while there have been key breakthroughs in new ideas, one of the best ways to increase the performance of these systems is getting more and better data. And how many people think data is the primary avenue to improvement.

It reminds me of an AI talk a few decades ago, about how the cycle goes: more data -> more layers -> repeat...

Anyways, I'm not sure how your comment relates to these two avenues of improvement.

Re: There are no new ideas in AI, only new datasets

#22

Post by nyrikki »

ctoth wrote: post #3

Reinforcement learning from self-play/AlphaWhatever? Nah must just be datasets. :)

Big difference between a perfect information, completely specified zero sum game and the real world.

As a simple analogy, read out the following sentence multiple times, stressing a different word each time.

"I never said she stole my money"

Note how the meaning changes and is often unique?

That is a lens I to the frame problem and it's inverse, the specification problem.

The above problem quickly becomes tower-complete, and recent studies suggest that RL is reinforcing or increasing the weight of existing patterns.

As the open domain frame problem and similar challenges are equivalent to HALT, finding new ways to extract useful information will be important for generalization IMHO.

Synthetic data is useful, but not a complete solution, especially for tower problems.

Re: There are no new ideas in AI, only new datasets

#24

Post by skydhash »

voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

Yeah, but are there new ideas or only wishes?

Re: There are no new ideas in AI, only new datasets

#25

Post by rar00 »

disagree, there are a few organisations exploring novel paths. It's just that throwing new data at an "old" algorithm is much easier and has been a winning strategy. And, also, there's no incentive for a private org to advertise a new idea that seems to be working (mine's a notable exception :D).

Re: There are no new ideas in AI, only new datasets

#26

Post by jjtheblunt »

jschveibinz wrote: post #7

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now. Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants. So AI can be an express elevator up to an army of giant's shoulders? It all de…

> I will respectfully disagree. All "new" ideas come from old ideas.

The insight into the structure of the benzene ring famously came in a dream, hadn't been seen before, but was imagined as a snake bitings its own tail.

Re: There are no new ideas in AI, only new datasets

#28

Post by piinbinary »

AI training is currently a process of making the AI remember the dataset. It doesn't involve the AI thinking about the dataset and drawing (and remembering) conclusions.

It can probably remember more facts about a topic than a PhD in that topic, but the PhD will be better at thinking about that topic.

Re: There are no new ideas in AI, only new datasets

#29

Post by FeepingCreature »

Y_Y wrote: post #13
grumpopotamus wrote: post #5

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/TD-Gammon

You raise a really interesting point. I'm sure it's just missed my notice, but I'm not familiar with any projects from antediluvian AI that have been resurrected to run on modern hardware and see where they'd really asymptote if they'd had the compute they deserved.

To be fair, usually those projects would need considerable work to be ported to modern multicore machines, let alone GPUs.

Re: There are no new ideas in AI, only new datasets

#30

Post by EternalFury »

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks.

It’s apparently much easier to scare the masses with visions of ASI, than to build a general intelligence that can pick up a new 2D video game faster than a human being.

Post reply on HN