Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

131–140 of 307 posts

Re: There are no new ideas in AI, only new datasets

#131

Post by refulgentis »

dvfjsdhgfv wrote: post #115
refulgentis wrote: post #89

Earlier quoted context omitted.

Names >> all, and increasingly so. One phenomena that bared this to me, in a substantive way, was noticing an increasing # of reverent comments re: Geohot in odd places here, that are just as quickly replied to by people with a sense of how he works , as opposed to the keywords he associates himself with . But that only happens here AFAIK. Yapping, or, inducing people to yap about me, unfortunately, is much more sali…

> Now I have to rework my site to not use "prompt engineering" and have a Take™ on "context engineering". Because of a couple tweets + a blog reverberating over 2-3 days. The difference here is that your example shows a trivial statement and a change period of 3 days, whereas what Carmack is doing is taking years.

Right. Nothing against Carmack. Grew up on the guy. I haven't looked into, at all, into any of the disputed stuff, and should actively proclaim I'm a yuge Carmack fanboy.

Re: There are no new ideas in AI, only new datasets

#132

Post by outofpaper »

altairprime wrote: post #106
vladimirralev wrote: post #48

Earlier quoted context omitted.

He is not using appropriate models for this conclusion and neither is he using state of the art models in this research and moreover he doesn't have an expensive foundational model to build upon for 2d games. It's just a fun project. A serious attempt at video/vision would involve some probabilistic latent space that can be noised in ways that make sense for games in general. I think veo3 proves that ai can generaliz…

Is any model currently known to succeed in the scenario that Carmack’s inappropriate model failed?

No monolithic models but us ng hybrid approaches we've been able to beet humans for some time now.

Re: There are no new ideas in AI, only new datasets

#133

Post by [deleted] »

voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

[deleted]

Re: There are no new ideas in AI, only new datasets

#134

Post by layer8 »

snapcaster wrote: post #81
layer8 wrote: post #63

Earlier quoted context omitted.

This needs to be generalized however. For example, if you present an AI with a drawing of some directed graph (a state diagram, for example), it should be able to answer questions based on the precise set of all possible paths in that graph, without someone having to write tooling for diagram or graph processing and traversal. Or, given a photo of a dropped box of matches, an AI should be able to precisely count the…

What you're describing sounds like agentic tool usage. Have you kept up with the latest developments on that? it's already solved depending on how strict you define your criteria above

My understanding is that you need to provide and configure task-specific tools. You can’t combine the AI with just a general-purpose computer and have the AI figure out on its own how to make use of it to achieve with reliability and precision whatever task it is given. In other words, the current tool usage isn’t general-purpose in the way the LLM itself is, and also the LLM doesn’t reason about its own capabilities in order to decide how to incorporate computer use to compensate for its own weaknesses. Instead you have to tell the LLM what it should apply the tooling for.

Re: There are no new ideas in AI, only new datasets

#135

Post by somebodythere »

I don't know if it matters. Even if the best we can do is get really good at interpolating between solutions to cognitive tasks on the data manifold, the only economically useful human labor left asymptotes toward frontier work; work that only a single-digit percentage of people can actually perform.

Re: There are no new ideas in AI, only new datasets

#136

Post by MichaelZuo »

IshKebab wrote: post #111
MichaelZuo wrote: post #105

Earlier quoted context omitted.

How do you know it’s a novel question?

It's not exactly difficult to come up with a question that's so unusual the chance of it being in the training set is effectively zero.

Can you provide some examples of these genuinely unique questions?

Re: There are no new ideas in AI, only new datasets

#137

Post by tim333 »

jdgoesmarching wrote: post #44
skydhash wrote: post #24

Earlier quoted context omitted.

Yeah, but are there new ideas or only wishes?

It’s pure magical thinking that would be correctly dismissed if it didn’t have AI attached to it. Imagine talking this way about anything else. “We’ve barely scratched the surface with Rust, so far we’re only focused on code and haven’t even explored building mansions or ending world hunger”

AI has some real possibilities of building mansions and ending hunger in a way that Rust doesn't.

Re: There are no new ideas in AI, only new datasets

#138

Post by hluska »

keerthiko wrote: post #120
hluska wrote: post #112

Earlier quoted context omitted.

[flagged]

fair, and I edited my choice of words, but if you're reading that much aggression from my initial comment (which contains topical discussion) to say what you did, you must find the internet a far more savage place than it really is :/

[flagged]

Re: There are no new ideas in AI, only new datasets

#140

Post by cadamsdotcom »

What about actively obtained data - models seeking data, rather than being fed. Human babies put things in their mouths, they try to stand and fall over. They “do stuff” to learn what works. Right now we’re just telling models what works.

What about simulation: models can make 3D objects so why not give them a physics simulator? We have amazing high fidelity (and low cost!) game engines that would be a great building block.

What about rumination: behind every Cursor rule for example, is a whole story of why a user added it. Why not take the rule, ask a reasoning model to hypothesize about why that rule was created, and add that rumination (along with the rule) to the training data. Providing opportunities to reflect on the choices made by their users might deepen any insights, squeezing more juice out of the data.

Post reply on HN