Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

291–300 of 307 posts

Re: There are no new ideas in AI, only new datasets

#291

Post by BlueTemplar »

halfcat wrote: post #149
antisthenes wrote: post #54

Earlier quoted context omitted.

Where do you draw the line between pattern matching and reasoning about world models? A lot of intelligence is just pattern matching and being quick about it.

The line is: building an internal world model requires interfacing with the world, not a model of it, and subsequent failing (including death and survivorship over generations) and adaptation. Plus pattern matching. Current AI only does one of those (pattern matching, not evolution), and the prospects of simulating evolution is kind of bleak, given I don’t think we can simulate a full living cell yet from scratch? Bu…

You don't need to simulate a full living cell to have evolution. In fact, isn't using evolving programs a decades-old technic ?

Re: There are no new ideas in AI, only new datasets

#292

Post by mr_toad »

timewizard wrote: post #234
petesergeant wrote: post #185

Earlier quoted context omitted.

> They took 1970s dead tech and deployed it on machines 1 million times more powerful. I’m not sure I’d qualify this as progress If this isn’t meant to be sarcasm or irony, you’ve got some really exciting research and learning ahead of you! At the moment it reads very “computers are just addition and multiplication and we’ve had that for thousands of years!”

> you’ve got some really exciting research and learning ahead of you I've done the research. Which is why I made the point I did. You're being dismissive and rude instead of putting forth any sort of argument. It's the paper hat of fake intellect. Yawn. > At the moment it reads very “computers are just addition and multiplication and we’ve had that for thousands of years!” Let's be specific then. The problem with the…

A linear increase in technology can easily lead to a greater than linear increase in economic gain. Sometimes even small performance gains can overturn whole industries.

Re: There are no new ideas in AI, only new datasets

#293

Post by MichaelZuo »

hackinthebochs wrote: post #165
MichaelZuo wrote: post #158

Earlier quoted context omitted.

Elephant mirror tests existed, so it doesn’t seem all that novel when the word “elephant” could just be substituted for the word “LLM”?

The question isn't about universal novelty, but whether the prompt/context is novel enough such that the LLM answering competently demonstrates understanding. The claim of parroting is that the dataset contains a near exact duplicate of any prompt and so the LLM demonstrating what appears to be competence is really just memorization. But if an LLM can generalize from an elephant mirror test to an LLM mirror test in a…

How do you know it’s the one generalizing?

Likely there has been at least one text that already does that for say dolphin mirror tests or chimpanzee mirror teats.

Re: There are no new ideas in AI, only new datasets

#294

Post by MichaelZuo »

pdabbadabba wrote: post #287
MichaelZuo wrote: post #136

Earlier quoted context omitted.

Can you provide some examples of these genuinely unique questions?

I'm not sure what you mean by "genuinely." But in the coding context LLMs answer novel questions all the time. My codebase uses components and follows patterns that an LLM will have seen before, but the actual codebase is unique. Yet, the LLM can provide detailed explanations about how it works, what bugs or vulnerabilities it might have, modify it, or add features to it.

It must not have existed prior in any text database whatsoever.

Re: There are no new ideas in AI, only new datasets

#295

Post by pdabbadabba »

MichaelZuo wrote: post #294
pdabbadabba wrote: post #287

Earlier quoted context omitted.

I'm not sure what you mean by "genuinely." But in the coding context LLMs answer novel questions all the time. My codebase uses components and follows patterns that an LLM will have seen before, but the actual codebase is unique. Yet, the LLM can provide detailed explanations about how it works, what bugs or vulnerabilities it might have, modify it, or add features to it.

It must not have existed prior in any text database whatsoever.

It certainly wasn't. The codebase is thousands of lines of bespoke code that I just wrote.

Re: There are no new ideas in AI, only new datasets

#296

Post by drw85 »

dbspin wrote: post #256
cma wrote: post #255

Earlier quoted context omitted.

You can be born without hands and have zero cognitive deficits. Sensory info and action-feedback from hands, vision, hearing, isn't key to intelligence, but if you are born without vision and hearing it can cause developmental issues, but even if you lose vision and hearing before 2 years you can develop normally, like Hellen Keller.

Actually this is wrong. There's a connection between bodily sensation and emotion so profound that quadriplegics can develop flat affect which in turn leads to decision paralysis and cognitive deficit. Emotions are regulated somatically, and inform decision making and other aspects of motivation and reasoning. Source: https://pmc.ncbi.nlm.nih.gov/articles/PMC2633768/

Spinal cord injury implies that you were born with those senses and then abruptly lost them.

In that case, all the processes and pathways in your brain are relying on and tied to those senses, so losing them might also disrupt or affect those pathways and how they function.

Re: There are no new ideas in AI, only new datasets

#297

Post by drw85 »

tim333 wrote: post #137
jdgoesmarching wrote: post #44

Earlier quoted context omitted.

It’s pure magical thinking that would be correctly dismissed if it didn’t have AI attached to it. Imagine talking this way about anything else. “We’ve barely scratched the surface with Rust, so far we’re only focused on code and haven’t even explored building mansions or ending world hunger”

AI has some real possibilities of building mansions and ending hunger in a way that Rust doesn't.

How is this ending anyones hunger? As long as humans are steering the ship, the commodity will be limited to those that control it and they will make all the money. If anything, it has a big potential to cause more hunger.

Re: There are no new ideas in AI, only new datasets

#298

Post by drw85 »

pdabbadabba wrote: post #295
MichaelZuo wrote: post #294

Earlier quoted context omitted.

It must not have existed prior in any text database whatsoever.

It certainly wasn't. The codebase is thousands of lines of bespoke code that I just wrote.

Which pretty much every line in it was written similarly somewhere else before, including an explanation and is somehow included in the massive data set it was trained on.

So far i have asked the AI some novel questions and it came up with novel answers full of hallucinated nonsense, since it copied some similarly named setting or library function and replaced a part of it's name with something i was looking for.

Re: There are no new ideas in AI, only new datasets

#299

Post by drw85 »

trainerxr50 wrote: post #142
IshKebab wrote: post #50

Earlier quoted context omitted.

Well yeah... If you only ever played one game in your life you would probably be pretty shit at other games too. This does not seem very revealing to me.

I am decent at chess but barely know how the pieces in Go move. Of course, this because I have spent a lot of time TRAINING to play chess and basically none training to play go. I am good on guitar because I started training young but can't play the flute or piano to save my life. Most complicated skills have basically no transfer or carry over other than knowing how to train on a new skill.

But the point here is, if i gave you a guitar with a string more or less. Or a different shaped guitar, you could play it.

If i give you a chess set with dwarf themed pieces and different colored squares, you could play immediately.

Re: There are no new ideas in AI, only new datasets

#300

Post by specialist »

8n4vidtmkvmk wrote: post #205
specialist wrote: post #144

Earlier quoted context omitted.

Yes and: and forget.

Why is forgetting important? Things can either have an end time where they are no longer applicable or things we thought were true turn out to be false but it's still useful to see where we went wrong. I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep mem…

Do you remove dead code? Get rid of clutter? Ever try to change a habit?
Post reply on HN