Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Will scaling work?

dwarkeshpatel.com

Post reply on HN
Print view View original

191–200 of 289 posts

Re: Will scaling work?

#191

Post by yashap »

slibhb wrote: post #49

The best analogy for LLMs (up to and including AGI) is the internet + google search. Imagine explaining the internet/google to someone in 1950. That person might say "Oh my god, everything will change! Instantaneous, cheap communication! The world's information available at light speed! Science will accelerate, productivity will explode!" And yet, 70 years later, things have certainly changed, but we're living in the…

Depends on the quality of the AGI. If it’s legitimately as good or better than humans at almost everything, while being cost effective, it will utterly and completely change society. Humans will be obsolete at almost every job - why pay a human if an AGI can do it as good or better, for free(-ish)? Best case scenario, the AGI is benevolent, traditional work is gone, but we find some post-capitalism system, and new ways to keep life interesting/meaningful. Worst case scenario, pure sci-fi dystopia.

If it’s closer to a midpoint between GPT-4 and true human intelligence, then sure, I agree with you, it’s a significant change to society but not an overhaul. But if it’s actually a human level (or better) general intelligence, it’ll be the biggest change to human society maybe ever.

Re: Will scaling work?

#192

Post by Jensson »

cortic wrote: post #184
lossolo wrote: post #120

Earlier quoted context omitted.

Reasoning and intelligence exists without language.

You know i assumed that was true until right now. But I can't think of a single example of reason and intelligence existing without any form of language. Even insects have rudimentary language, and in fact reasoning and intelligence seem to scale with the complexity of language, both by species and within species.

Do slime mold have a language? Slime mold can learn and adapt to environments, so it is intelligent and can do rudimentary reasoning, but I doubt it communicates that information to other slime molds.

It is a very different kind of life form though so many things that applies to other complex being doesn't apply to them. Being a large single cell means that they learn by changing its proteins and other internals, very hard for us humans to reason about and understand since it is so alien compared to just having nerve cells with physical connections.

Re: Will scaling work?

#193

Post by elktown »

lern_too_spel wrote: post #177
elktown wrote: post #106

Earlier quoted context omitted.

> There is no magic in the brain. The amount of hubris we have in our field is deeply embarrassing. Imagine a neuroscientists reading that. The thought makes me blush.

Neuro scientists would agree with GP, otherwise they would be neuro mystics instead of neuro scientists . There is no magic. It's all physical processes that we can eventually understand.

The entire point was that we do not understand it? That much of how the brain work is "magic" atm.

It's our field that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways, while stirring our ML-pots that we have never even tried to prove have a chance to be anymore successful than the alchemists.

Re: Will scaling work?

#194

Post by lern_too_spel »

elktown wrote: post #193
lern_too_spel wrote: post #177

Earlier quoted context omitted.

Neuro scientists would agree with GP, otherwise they would be neuro mystics instead of neuro scientists . There is no magic. It's all physical processes that we can eventually understand.

The entire point was that we do not understand it? That much of how the brain work is "magic" atm. It's our field that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways, while stirring our ML-pots that we have never even tried to prove have a chance to be anymore successful than the alchemists.

Just because we don't understand it doesn't mean it's magic. That's the whole point of science.

> It's _our field_ that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways,

These "ramblings" are what scientists call hypotheses. The people making these hypotheses have even proposed how to test them.

Re: Will scaling work?

#195

Post by cortic »

Jensson wrote: post #192
cortic wrote: post #184

Earlier quoted context omitted.

You know i assumed that was true until right now. But I can't think of a single example of reason and intelligence existing without any form of language. Even insects have rudimentary language, and in fact reasoning and intelligence seem to scale with the complexity of language, both by species and within species.

Do slime mold have a language? Slime mold can learn and adapt to environments, so it is intelligent and can do rudimentary reasoning, but I doubt it communicates that information to other slime molds. It is a very different kind of life form though so many things that applies to other complex being doesn't apply to them. Being a large single cell means that they learn by changing its proteins and other internals, ver…

Not sure i would say a slime mold has reason and intelligence .. Or if i would then so does a river. Also i think that how it changes its proteins could be considered a language, without stretching the definition of language any more than we have already stretched the definition of reason and intelligence.

Re: Will scaling work?

#196

Post by jeremyjh »

berniedurfee wrote: post #188
jeremyjh wrote: post #140

Earlier quoted context omitted.

I feel like an LLM would do a much better job than GP.

Lol, at least then your comment wouldn’t have bothered me so much!

I'm sorry I hurt your feelings, it wasn't my intention. For what its worth, I actually think there is a good chance that you are right - that there is something missing in LLMs that still won't be present in bigger LLMs. I mostly meant that an LLM would be more organized around the source material and address specific points.

I actually asked ChatGPT 4 to do so, and it produced the sort of reasonable but unremarkable stuff I've come to expect from it.

Re: Will scaling work?

#197

Post by elktown »

lern_too_spel wrote: post #194
elktown wrote: post #193

Earlier quoted context omitted.

The entire point was that we do not understand it? That much of how the brain work is "magic" atm. It's our field that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways, while stirring our ML-pots that we have never even tried to prove have a chance to be anymore successful than the alchemists.

Just because we don't understand it doesn't mean it's magic. That's the whole point of science. > It's _our field_ that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways, These "ramblings" are what scientists call hypotheses. The people making these hypotheses have even proposed how to test them.

Even with added quotes you can't stop reading it literally? The total lack of critical thinking due to confirmation bias is just as embarrassing.

Re: Will scaling work?

#198

Post by lewhoo »

whoami_nr wrote: post #41
berniedurfee wrote: post #31

I think there’s a huge assumption here that more LLM will lead to AGI. Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI. LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI. LLMs don’t “reason” in any sense of the word that I understand and I think the ab…

Why next-token prediction is enough for AGI - Ilya Sutskever - https://www.youtube.com/watch?v=YEUclZdj_Sc

I really don't think there's an explanation there. All Sutskever says is the idea is to ask a LLM to be the smartest being on the planet and it magically happens.

Re: Will scaling work?

#199

Post by Jensson »

cortic wrote: post #195
Jensson wrote: post #192

Earlier quoted context omitted.

Do slime mold have a language? Slime mold can learn and adapt to environments, so it is intelligent and can do rudimentary reasoning, but I doubt it communicates that information to other slime molds. It is a very different kind of life form though so many things that applies to other complex being doesn't apply to them. Being a large single cell means that they learn by changing its proteins and other internals, ver…

Not sure i would say a slime mold has reason and intelligence .. Or if i would then so does a river. Also i think that how it changes its proteins could be considered a language, without stretching the definition of language any more than we have already stretched the definition of reason and intelligence.

Why is a slime mold a river but a human isn't? Slime mold can predict temperature changes in its environment and react before it happens, that isn't something a river could do.

So your statement just seems to be your bias thinking that a slime mold couldn't possible do any reasoning. Cells are much smarter than most thinks.

Edit: Anyway, apparently slime molds can communicate what they learn by sharing those proteins. So they do have a language, it is like a primitive version of how human bodies cells communicate. So your point still stands, reasoning seems to go hand in hand with communication. If you can reason then it is worth it to share those conclusions with your friends and family.

They also taught slime molds to cross a bridge for food, and it learned to do it. Then they got the slime mold to tell other slime molds and now those also knew how to cross the bridge. It is pretty cool that slime molds can be that smart.

https://asknature.org/strategy/brainless-slime-molds-both-le...

Re: Will scaling work?

#200

Post by discreteevent »

gitfan86 wrote: post #164
discreteevent wrote: post #146

Earlier quoted context omitted.

Parent is saying that with something as sophisticated as intelligence it's not enough to say that if it behaves like a duck it's a duck (which is what your seem to be saying and which the parent calls a 0-day). There are some really good bulshitters who have led smart people into deep trouble. These bulshitters behaved really like ducks but they weren't ducks. The duck test just isn't good enough. The -1 day is where…

That is a new definition of intelligence that you are using. You are saying that even when something can outperform humans in the SAT or other tests of intelligence, it isn't actually intelligent due to it not being a carbon based lifeform

No. I gave the example of a bullshitter - who is usually a carbon based life form.
Post reply on HN