There are no new ideas in AI, only new datasets • Hacker News Board

Re: There are no new ideas in AI, only new datasets

#191

Post by PeeMcGee » Tue, Jul 01, 2025, 5:26 AM UTC

amelius wrote: post #153

GuB-42 wrote: post #150

Earlier quoted context omitted.

Who is an "AI guy"? The field as we know it is fairly new. Sure, neural nets are old hat, but a lot has happened in the last few years. John Carmack founded Keen technology in 2022 and has been working seriously on AI since 2019. From his experience in the video game industry, he knows a thing or two about linear algebra and GPUs, that is the underlying maths and the underlying hardware. So, for all intent and purpos…

But the logic seems flawed. He has built an AI system that fails to do X. That does not mean there isn't an AI system that can do X. Especially considering that a lot is happening in AI, as you say. Anyway, Carmack knows a lot about optimizing computations on modern hardware. In practice, that happens to be also necessary for AI. However, it is not __sufficient__ for AI.

> That does not mean there isn't an AI system that can do X.

You are holding the burden of proof here...

Re: There are no new ideas in AI, only new datasets

#192

Post by geysersam » Tue, Jul 01, 2025, 5:29 AM UTC

thenaturalist wrote: post #125

ks2048 wrote: post #14

The latest LLMs are simply multiplying and adding various numbers together... Babylonians were doing that 4000 years ago.

But on clay tables, not in semi-conductive electron prisons separated by one-atom-thick walls. Slight difference to those methods, wouldn't you agree?

No it's exactly the same. Everything old is new again...

Re: There are no new ideas in AI, only new datasets

#193

Post by [deleted] » Tue, Jul 01, 2025, 5:37 AM UTC

ozgrakkurt wrote: post #175

EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

Seeing comments here saying “this problem is already solved”, “he is just bad at this” etc. feels bad. He has given a long time to this problem by now. He is trying to solve this to advance the field. And needless to say, he is a legend in computer engineering or w/e you call it. It should be required to point to the “solution” and maybe how it works to say “he just sucks” or “this was solved before”. IMO the problem…

[deleted]

Re: There are no new ideas in AI, only new datasets

#194

Post by nkmnz » Tue, Jul 01, 2025, 5:37 AM UTC

amelius wrote: post #153

GuB-42 wrote: post #150

Earlier quoted context omitted.

Who is an "AI guy"? The field as we know it is fairly new. Sure, neural nets are old hat, but a lot has happened in the last few years. John Carmack founded Keen technology in 2022 and has been working seriously on AI since 2019. From his experience in the video game industry, he knows a thing or two about linear algebra and GPUs, that is the underlying maths and the underlying hardware. So, for all intent and purpos…

But the logic seems flawed. He has built an AI system that fails to do X. That does not mean there isn't an AI system that can do X. Especially considering that a lot is happening in AI, as you say. Anyway, Carmack knows a lot about optimizing computations on modern hardware. In practice, that happens to be also necessary for AI. However, it is not __sufficient__ for AI.

This is exactly how Science works. He’s right until proven wrong. And so are you.

Re: There are no new ideas in AI, only new datasets

#195

Post by e2021 » Tue, Jul 01, 2025, 5:45 AM UTC

IshKebab wrote: post #50

NBJack wrote: post #40

Earlier quoted context omitted.

In other words, they learn the game, not how to play games .

Well yeah... If you only ever played one game in your life you would probably be pretty shit at other games too. This does not seem very revealing to me.

I don't think thats true. If you'd only ever played Doom, I think you could play, say, counterstrike or half-life and be pretty good at it, and i think Carmack is right that its pretty interesting that this doesn't seem to be the case for ai models

Re: There are no new ideas in AI, only new datasets

#196

Post by Centigonal » Tue, Jul 01, 2025, 5:54 AM UTC

cadamsdotcom wrote: post #145

Centigonal wrote: post #143

Earlier quoted context omitted.

Simulation and embodied AI (putting the AI in a robotic arm or a car so it can try stuff and gather information about the results) are very actively being explored.

What about at inference time? ie. in response to a query. We let models write code and run it. Which gives them a high chance of getting arithmetic right. Solving the “crossing the river” problem by letting the model create and run a simulation would give a pretty high chance of getting it right.

The newest Claude update comes with a python sandbox built right into the API for exactly this reason.

https://docs.anthropic.com/en/docs/agents-and-tools/tool-use...

Re: There are no new ideas in AI, only new datasets

#197

Post by slashdave » Tue, Jul 01, 2025, 5:59 AM UTC

voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

> modeled human intelligence

That's not what these models do

Re: There are no new ideas in AI, only new datasets

#198

Post by mym1990 » Tue, Jul 01, 2025, 6:07 AM UTC

sigmoid10 wrote: post #93

vladimirralev wrote: post #48

Earlier quoted context omitted.

He is not using appropriate models for this conclusion and neither is he using state of the art models in this research and moreover he doesn't have an expensive foundational model to build upon for 2d games. It's just a fun project. A serious attempt at video/vision would involve some probabilistic latent space that can be noised in ways that make sense for games in general. I think veo3 proves that ai can generaliz…

Veo3's world model is still pretty limited. That becomes obvious very fast once you prompt out of distribution video content (i.e. stuff that you are unlikely to find on youtube). It's extremely good at creating photorealistic surfaces and lighting. It even has some reasonably solid understanding of fluid dynamics for simulating water. But for complex human behaviour (in particular certain motions) it simply lacks th…

What is the basis for it having a reasonable understanding of fluid dynamics? Why don’t you think it’s just regurgitating some water scenes derived from its training data, rather than generating actual fluid dynamics?

Re: There are no new ideas in AI, only new datasets

#199

Post by cadamsdotcom » Tue, Jul 01, 2025, 6:13 AM UTC

kevmo314 wrote: post #186

cadamsdotcom wrote: post #146

Earlier quoted context omitted.

Agreed for most cases. Each Cursor rule is a byproduct of tons of work and probably contains lots that can be unpacked. Any research on that?

Yeah, at a very high level it's similar to an actor-critic reinforcement learning algorithm. The rule text is a value function and one could build a critic model that takes as input the rule text and the main model's (the actor's) output to produce a reward. This is easier said than done though because this value function is so noisy it's often hard to learn from it. And also whether or not a response (the model outp…

Suggestion was even simpler - feed a reasoning model a prompt like “tell me a few reasons a user might’ve created this Cursor rule: {RULE_TEXT}”

Do that for a bunch of rules scraped from a bunch of repos - and you’ve got yourself a dataset for training a new model with - or maybe for fine tuning.

Re: There are no new ideas in AI, only new datasets

#200

Post by IIAOPSW » Tue, Jul 01, 2025, 6:14 AM UTC

EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

There's something fascinating about this, because the human ability to "transfer knowledge" (eg pick up some other never before seen video game and quickly understand it) isn't really that general. There's a very particular "overtone window" of the sort of degrees of difference where it is possible.

If I were to hand you a version of a 2d platformer (lets say Mario) where the gimmick is that you're actually playing the fourier transform of the normal game, it would be hopeless. You might not ever catch on that the images on screen are completely isomorphic to a game you're quite familiar with and possibly even good at.

But some range of spatial transform gimmicks are cleanly intuitive. We've seen this with games like vvvvvv and braid.

So the general rule seems to be that intelligence is transferable to situations that are isomorphic up to certain "natural" transforms, but not to "matching any possible embedding of the same game in a different representation".

Our failure to produce anything more than hyper-specialists forces us to question exactly is meant by the ability to generalize other than just "mimicking an ability humans seem to have".