There are no new ideas in AI, only new datasets

To be fair, if you imagine a system that successfully reproduced human intelligence, then 'changing datasets' would probably be a fair summary of what it would take to have different models. After all, our own memories, training, education, background, etc are a very large component of our own problem solving abilities.

Re: There are no new ideas in AI, only new datasets

#5

Post by grumpopotamus » Mon, Jun 30, 2025, 5:17 PM UTC

ctoth wrote: post #3

Reinforcement learning from self-play/AlphaWhatever? Nah must just be datasets. :)

https://en.wikipedia.org/wiki/TD-Gammon

Re: There are no new ideas in AI, only new datasets

#6

Post by NitpickLawyer » Mon, Jun 30, 2025, 5:21 PM UTC

ctoth wrote: post #3

Reinforcement learning from self-play/AlphaWhatever? Nah must just be datasets. :)

And architecture stuff like actually useful long context. Whatever they did with gemini 2.5 is miles ahead in long context useful results compared to the previous models. I'd be very surprised if gemini 2.5 is "just" gemini 1 w/ better data.

Re: There are no new ideas in AI, only new datasets

#7

Post by jschveibinz » Mon, Jun 30, 2025, 5:24 PM UTC

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now.

Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants.

So AI can be an express elevator up to an army of giant's shoulders? It all depends on how you use the tools.

Re: There are no new ideas in AI, only new datasets

#8

Post by HeatrayEnjoyer » Mon, Jun 30, 2025, 5:26 PM UTC

code_for_monkey wrote: post #2

[flagged]

Everyone knows we'll always need horseshoes /s

Re: There are no new ideas in AI, only new datasets

#9

Post by saltserv » Mon, Jun 30, 2025, 5:29 PM UTC

[dead]

Re: There are no new ideas in AI, only new datasets

#10

Post by tippytippytango » Mon, Jun 30, 2025, 5:33 PM UTC

Sometimes we get confused by the difference between technological and scientific progress. When science makes progress it unlocks new S-curves that progress at an incredible pace until you get into the diminishing returns region. People complain of slowing progress but it was always slow, you just didn’t notice that nothing new was happening during the exponential take off of the S-curve, just furious optimization.