Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

121–130 of 307 posts

Re: There are no new ideas in AI, only new datasets

#121

Post by troupo »

jjtheblunt wrote: post #26
jschveibinz wrote: post #7

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now. Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants. So AI can be an express elevator up to an army of giant's shoulders? It all de…

> I will respectfully disagree. All "new" ideas come from old ideas. The insight into the structure of the benzene ring famously came in a dream, hadn't been seen before, but was imagined as a snake bitings its own tail.

And as we all know, it came in a dream to a complete novice in chemistry with zero knowledge of any old ideas in chemistry: https://en.wikipedia.org/wiki/August_Kekul%C3%A9

--- start quote ---

The empirical formula for benzene had been long known, but its highly unsaturated structure was a challenge to determine. Archibald Scott Couper in 1858 and Joseph Loschmidt in 1861 suggested possible structures that contained multiple double bonds or multiple rings, but the study of aromatic compounds was in its earliest years, and too little evidence was then available to help chemists decide on any particular structure.

More evidence was available by 1865, especially regarding the relationships of aromatic isomers.

[ Kekule claimed to have had the dream in 1865 ]

--- end quote ---

The dream claim came from Kekule himself 25 years after his proposal that he had to modify 10 years after he proposed it.

Re: There are no new ideas in AI, only new datasets

#122

Post by vladimirralev »

keerthiko wrote: post #102
vladimirralev wrote: post #48

Earlier quoted context omitted.

He is not using appropriate models for this conclusion and neither is he using state of the art models in this research and moreover he doesn't have an expensive foundational model to build upon for 2d games. It's just a fun project. A serious attempt at video/vision would involve some probabilistic latent space that can be noised in ways that make sense for games in general. I think veo3 proves that ai can generaliz…

> generating a video under prompt constraints is basically playing a game Besides static puzzles (like a maze or jigsaw) I don't believe this analogy holds? A model working with prompt constraints that aren't evolving or being added over the course of "navigating" the generation of the model's output means it needs to process 0 new information that it didn't come up with itself — playing a game is different from othe…

It's not ideal, but you can prompt it with an image of a game frame, explain the objects and physics in text and let it generate a few frames of gameplay as a substitute for controller input as well as what it expects as an outcome. I am not talking about real interactive gameplay.

I am just saying we have proof that it can understand complex worlds and sets of rules, and then abide by them. It doesn't know how to use a controller and it doesn't know how to explore the game physics on its own, but those steps are much easier to implement based on how coding agents are able to iterate and explore solutions.

Re: There are no new ideas in AI, only new datasets

#123

Post by imiric »

IshKebab wrote: post #110
0xWTF wrote: post #65

Earlier quoted context omitted.

> You can trivially prove this yourself. Given the long list of dead philosophers of mind, if you have a trivial proof, would you mind providing a link?

Just go and ask ChatGPT or Claude something that can't possibly be in its training set. Make something up. If it is only memorising answers then it will be impossible for it to get the correct result. A simple nonsense programming task would suffice. For example "write a Python function to erase every character from a string unless either of its adjacent characters are also adjacent to it in the alphabet. The string…

LLMs don't "memorize" concepts like humans do. They generate output based on token patterns in their training data. So instead of having to be trained on every possible problem, they can still generate output that solves it by referencing the most probable combination of tokens for the specified input tokens. To humans this seems like they're truly solving novel problems, but it's merely a trick of statistics. These tools can reference and generate patterns that no human ever could. This is what makes them useful and powerful, but I would argue not intelligent.

Re: There are no new ideas in AI, only new datasets

#124

Post by thenaturalist »

strangescript wrote: post #91

If you work with model architecture and read papers, how could not know there are a flood of new ideas? Only few yield interesting results though. I kind of wonder if libraries like pytorch have hurt experimental development. So many basic concepts no one thinks about anymore because they just use the out of the box solutions. And maybe those solutions are great and those parts are "solved", but I am not sure. How ma…

That's been the very normal way of the human world.

When the foundation layer at a given moment doesn't yield an ROI on intellectual exploration - say because you can overcompensate with VC funded raw compute and make more progess elsewhere -, few(er) will go there.

But inevitably, as other domains reach diminishing returns, bright minds will take a look around where significant gains for their effort can be found.

And so will the next generation of PyTorch or foundational technologies evolve.

Re: There are no new ideas in AI, only new datasets

#125

Post by thenaturalist »

ks2048 wrote: post #14

The latest LLMs are simply multiplying and adding various numbers together... Babylonians were doing that 4000 years ago.

But on clay tables, not in semi-conductive electron prisons separated by one-atom-thick walls.

Slight difference to those methods, wouldn't you agree?

Re: There are no new ideas in AI, only new datasets

#126

Post by tsimionescu »

raincole wrote: post #99
goatlover wrote: post #87

Earlier quoted context omitted.

I've wondered about the claim that the models played those Atari/2D video games at superhuman levels, because I clearly recall some humans achieving superhuman levels before models were capable of it. Must have been superhuman compared to average human player, not someone who spent an inordinate amount of time mastering the game.

I'm not sure why you think so. AI outperforms humans in many games already. Basically all the games we care to put money to train a model. AI has beat the best human players in Chess, Go, Mahjong, Texas hold'em, Dota, Starcraft, etc. It would be really, really surprising that some Atari game is the holy grail of human performance that AI cannot beat.

I recall this not being true at all for Dota and Starcraft. I recall AlphaStar performed much better than the top non-pro players, but it couldn't consistently beat the pro players with the budget that Google was willing to spend, and I believe the same was true of Dota II (and there they were even playing a limited form of the game, with fewer heroes and without the hero choice part, I believe).

Re: There are no new ideas in AI, only new datasets

#127

Post by nightpool »

EternalFury wrote: post #104
fsmv wrote: post #45

Earlier quoted context omitted.

They memorize the answers not the process to arrive at answers

They learn the value of specific actions in specific contexts based on the rewards they received during their play time. Specific actions and specific contexts are not transferable for various reasons. John quoted that varying frame rates and variable latency between action and effect really confuse the models.

Okay, so fuzz the frame rate and latency? That feels very easy to fix.

Re: There are no new ideas in AI, only new datasets

#128

Post by vladimirralev »

troupo wrote: post #116
vladimirralev wrote: post #48

Earlier quoted context omitted.

He is not using appropriate models for this conclusion and neither is he using state of the art models in this research and moreover he doesn't have an expensive foundational model to build upon for 2d games. It's just a fun project. A serious attempt at video/vision would involve some probabilistic latent space that can be noised in ways that make sense for games in general. I think veo3 proves that ai can generaliz…

> I think veo3 proves that ai can generalize 2d and even 3d games It doesn't. And you said it yourself: > generating a video under prompt constraints is basically playing a game. No. It's neither generating a game (that people can play) nor is it playing a game (it's generating a video ). Since it's not a model of the world in any sense of the word, there are issues with even the most basic object permanenece. E.g. h…

It is still doing a great job for a few frames, you could keep it more anchored to the state of the game if you prompt it. Much like you can prompt coding agents to keep a log of all decisions previously made. Permanenece is excellent, it slips often but it mostly because it is not grounded to specific game state by the prompt or by the decision log.

Re: There are no new ideas in AI, only new datasets

#129

Post by shwouchk »

NitpickLawyer wrote: post #6
ctoth wrote: post #3

Reinforcement learning from self-play/AlphaWhatever? Nah must just be datasets. :)

And architecture stuff like actually useful long context. Whatever they did with gemini 2.5 is miles ahead in long context useful results compared to the previous models. I'd be very surprised if gemini 2.5 is "just" gemini 1 w/ better data.

i dont know what all the hype is with gemini 2.5, at least the currently running instance. from my experience at least in conversation mode, it cannot remember my instructions to avoid apologies and similar platitudes from either the “persona”, personal instructions, or from ine message to the next.

Re: There are no new ideas in AI, only new datasets

#130

Post by pshc »

vladimirralev wrote: post #48
EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

He is not using appropriate models for this conclusion and neither is he using state of the art models in this research and moreover he doesn't have an expensive foundational model to build upon for 2d games. It's just a fun project. A serious attempt at video/vision would involve some probabilistic latent space that can be noised in ways that make sense for games in general. I think veo3 proves that ai can generaliz…

I think we need a spatial/physics model handling movement and tactics watched over by a high level strategy model (maybe an LLM).
Post reply on HN