Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

81–90 of 307 posts

Re: There are no new ideas in AI, only new datasets

#81

Post by snapcaster »

layer8 wrote: post #63
cheevly wrote: post #43

Earlier quoted context omitted.

Many of us have solved this with internal tooling that has not yet been shared or released to the public.

This needs to be generalized however. For example, if you present an AI with a drawing of some directed graph (a state diagram, for example), it should be able to answer questions based on the precise set of all possible paths in that graph, without someone having to write tooling for diagram or graph processing and traversal. Or, given a photo of a dropped box of matches, an AI should be able to precisely count the…

What you're describing sounds like agentic tool usage. Have you kept up with the latest developments on that? it's already solved depending on how strict you define your criteria above

Re: There are no new ideas in AI, only new datasets

#82

Post by pdabbadabba »

0xWTF wrote: post #65
IshKebab wrote: post #53

Earlier quoted context omitted.

This has been disproven so many times... They clearly do both. You can trivially prove this yourself.

> You can trivially prove this yourself. Given the long list of dead philosophers of mind, if you have a trivial proof, would you mind providing a link?

It’s really easy: go to Claude and ask it a novel question. It will generally reason its way to a perfectly good answer even if there is no direct example of it in the training data.

Re: There are no new ideas in AI, only new datasets

#84

Post by baxtr »

tippytippytango wrote: post #10

Sometimes we get confused by the difference between technological and scientific progress. When science makes progress it unlocks new S-curves that progress at an incredible pace until you get into the diminishing returns region. People complain of slowing progress but it was always slow, you just didn’t notice that nothing new was happening during the exponential take off of the S-curve, just furious optimization.

Fully agree.

And at the same time I have noticed that people don’t understand the difference between an S-curve and an exponential function. They can look almost identical at certain intervals.

Re: There are no new ideas in AI, only new datasets

#85

Post by Swizec »

nomel wrote: post #74
Swizec wrote: post #19

Earlier quoted context omitted.

> The real frontier of AI lies in the messy, rich, sensory world where people live. We’ll need new hardware (sensors), new data representations (beyond tokens), and new ways to train models that grow understanding from experience, not just patterns. Like Dr. Who said: DALEKs aren't brains in a machine, they are the machine! Same is true for humans. We really are the whole body, we're not just driving it around.

There are many people who mentally developed while paralyzed that literally drive around their bodies via motorized wheelchair. I don't think there's any evidence that a brain couldn't exist or develop in a jar, given only the inputs modern AI now has (text, video, audio).

> any evidence that a brain couldn't exist or develop in a jar

The brain could. Of course it could. It's just a signals processing machine.

But would it be missing anything we consider core to the way humans think? Would it struggle with parts of cognition?

For example: experiments were done with cats growing up in environments with vertical lines only. They were then put in a normal room and had a hard time understanding flat surfaces.

https://computervisionblog.wordpress.com/2013/06/01/cats-and...

Re: There are no new ideas in AI, only new datasets

#86

Post by baxtr »

jschveibinz wrote: post #7

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now. Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants. So AI can be an express elevator up to an army of giant's shoulders? It all de…

Imagine a human had read every book/publication in every field of knowledge that mankind has ever produced AND couldn’t come up with anything entirely new. Hard to imagine.

Re: There are no new ideas in AI, only new datasets

#87

Post by goatlover »

EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

I've wondered about the claim that the models played those Atari/2D video games at superhuman levels, because I clearly recall some humans achieving superhuman levels before models were capable of it. Must have been superhuman compared to average human player, not someone who spent an inordinate amount of time mastering the game.

Re: There are no new ideas in AI, only new datasets

#88

Post by edanm »

chasd00 wrote: post #52
voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

> Language and vision are just the beginning.. Based on the architectures we have they may also be the ending. There’s been a lot of news in the past couple years about LLMs but has there been any breakthroughs making headlines anywhere else in AI?

Sure. In physics, math, chemistry, biology. To name a few.

Re: There are no new ideas in AI, only new datasets

#89

Post by refulgentis »

justanotherjoe wrote: post #68
EternalFury wrote: post #30

What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…

I don't get why people are so invested in framing it this way. I'm sure there are ways to do the stated objective. John Carmack isn't even an AI guy why is he suddenly the standard.

Names >> all, and increasingly so.

One phenomena that bared this to me, in a substantive way, was noticing an increasing # of reverent comments re: Geohot in odd places here, that are just as quickly replied to by people with a sense of how he works, as opposed to the keywords he associates himself with. But that only happens here AFAIK.

Yapping, or, inducing people to yap about me, unfortunately, is much more salient to my expected mindshare than the work I do.

It's getting claustrophobic intellectually, as a result.

Example from the last week is the phrase "context engineering" - Shopify CEO says he likes it better than prompt engineering, Karpathy QTs to affirm, SimonW writes it up as fait accompli. Now I have to rework my site to not use "prompt engineering" and have a Take™ on "context engineering". Because of a couple tweets + a blog reverberating over 2-3 days.

Nothing against Carmack, or anyone else named, at all. i.e. in the context engineering case, they're just sharing their thoughts in realtime. (i.e. I don't wanna get rolled up into a downvote brigade because it seems like I'm affirming the loose assertion Carmack is "not an AI guy", or, that it seems I'm criticizing anyone's conduct at all)

EDIT: The context engineering example was not in reference to another post at the time of writing, now one is the top of front page.

Re: There are no new ideas in AI, only new datasets

#90

Post by mr_world »

voxleone wrote: post #16

I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…

Organic adaption and persistence of memory I would say are the two major advancements that need to happen.

Human neural networks are dynamic, they change and rearrange, grow and sever. An LLM is fixed and relies on context, if you give it the right answer it won't "learn" that is the correct answer unless it is fed back into the system and trained over months. What if it's only the right answer for a limited period of time?

To build an intelligent machine, it must be able train itself in real time and remember.

Post reply on HN