Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaCode as a dog speaking mediocre English

scottaaronson.blog

Post reply on HN
Print view View original

71–80 of 263 posts

Re: AlphaCode as a dog speaking mediocre English

#71

Post by mlyle »

sillysaurusx wrote: post #36
sanxiyn wrote: post #35

Earlier quoted context omitted.

Sure, it doesn't think, just as submarines don't swim, as EWD said.

People always reach for these analogies. "Planes don't fly like birds." "Submarines don't swim like fish." Backpropagation has zero creativity. It's an elaborate mechanical parrot, and nothing more. It can never relate to you on a personal level, because it never experiences the world. It has no conception of what the world is. At least a dog gets hungry. Not persuaded? Try https://news.ycombinator.com/item?id=233469…

> Backpropagation has zero creativity. It's an elaborate mechanical parrot, and nothing more. It can never relate to you on a personal level, because it never experiences the world. It has no conception of what the world is.

The problem is: it's not really clear how much creativity we have, and how much of it is better explained by highly constrained randomized search and optimization.

> It can never relate to you on a personal level

Well, sure. Even if/once we reach AGI, it's going to be a highly alien creature.

> because it never experiences the world.

Hard to put this on a rigorous setting.

> It has no conception of what the world is.

It has imperfect models of the world it is presented. So do we!

> At least a dog gets hungry.

I don't think "gets hungry" is a very meaningful way to put this. But, yes: higher living beings act with agency in their environment (and most deep learning AIs we build don't, instead having rigorous steps of interaction not forming any memory of the interaction) and have mechanisms to seek novelty in those interactions. I don't view these as impossible barriers to leap over.

Re: AlphaCode as a dog speaking mediocre English

#72

Post by visarga »

323 wrote: post #50
asdfasgasdgasdg wrote: post #5

Earlier quoted context omitted.

It's rarely productive to take internet criticism into account, but it feels like AI is an especially strong instance of this. It seems like a lot of folks just want to pooh pooh any possible outcome. I'm not sure why this is. Possibly because of animosity toward big tech, given big tech is driving a lot of the research and practical implementation in this area?

> It seems like a lot of folks just want to pooh pooh any possible outcome. I'm not sure why this is. Possibly because of animosity toward big tech It's much more simple, and more deep: most humans believe they are special/unique/non-machine like spiritual beings. Anything suggesting they could be as simple as the result of mechanical matrix multiplications is deeply disturbing and unacceptable. There is a rich recen…

There was a time when we were proud that Earth was the center of the universe, nobody dare say otherwise!

Re: AlphaCode as a dog speaking mediocre English

#73

Post by mistercheph »

swyx wrote: post #61
sabujp wrote: post #31

Earlier quoted context omitted.

All these deepmind and openai computer scientists don't pay themselves, you're not taking their pay into consideration.

this is the definition of a nit. they dont cost 1b. still tiny vs the other projects listed

google is undoubtedly spending at least 1 billion on ML probably closer to 10 billion per annum

Re: AlphaCode as a dog speaking mediocre English

#74

Post by visarga »

mrshadowgoose wrote: post #51
stephc_int13 wrote: post #10

Earlier quoted context omitted.

AI has been over-hyped, that's all. The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software. But people have different and irrational expectations about AI.

> The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software. You might be using a different definition of "magical" than what others are using in this context. Of course, when you break down ML techniques, it's all just math running on FETs. So no, it's not extra-dimensional hocus pocus, but absolutely nobody is using that particular defi…

If recent philosophy taught us anything it's that brains are special. The hard problem of consciousness shows science is insufficient to raise to the level of entitlement of humans, we're exceptions flying over the physical laws of nature, we have free will, first person POV, and other magical stuff like that. Or we have to believe in panpsychism or dualism, like in the middle ages. Anything to lift the human status.

Maybe we should start by "humans are the greatest thing ever" and then try to fit our world knowledge to that conclusion. We feel it right in our qualia that we are right, and qualia is ineffable.

Re: AlphaCode as a dog speaking mediocre English

#75

Post by hprotagonist »

visarga wrote: post #72
323 wrote: post #50

Earlier quoted context omitted.

> It seems like a lot of folks just want to pooh pooh any possible outcome. I'm not sure why this is. Possibly because of animosity toward big tech It's much more simple, and more deep: most humans believe they are special/unique/non-machine like spiritual beings. Anything suggesting they could be as simple as the result of mechanical matrix multiplications is deeply disturbing and unacceptable. There is a rich recen…

There was a time when we were proud that Earth was the center of the universe, nobody dare say otherwise!

though at the same time, and in the same population, the existence of other planets full of conscious beings was basically non-controversial:

Life, as it exists on Earth in the form of men, animals and plants, is to be found, let us suppose in a high form in the solar and stellar regions.

Rather than think that so many stars and parts of the heavens are uninhabited and that this earth of ours alone is peopled—and that with beings perhaps of an inferior type—we will suppose that in every region there are inhabitants, differing in nature by rank and all owing their origin to God, who is the center and circumference of all stellar regions.

Of the inhabitants then of worlds other than our own we can know still less having no standards by which to appraise them.

Nicholas of Cusa, c1440; Cardinal and Bishop

Re: AlphaCode as a dog speaking mediocre English

#76

Post by recursive »

sinuhe69 wrote: post #55

I like to see how AlphaCode will solve a problem no human has solved before (or very unlikely). For example, given 3 beads and 2 stacks for the 10er and 1er positions of a number. One can make 4 different numbers when stack all the beads like in an abacus. Without using all the beads, one can of course make more numbers. The question is how many different numbers one can make using n beads, giving full and partial us…

Why do you think that's more difficult than the backspace question? I see no reason to doubt AlphaCode can solve it.

Re: AlphaCode as a dog speaking mediocre English

#77

Post by PeterisP »

sillysaurusx wrote: post #33
sanxiyn wrote: post #25

Earlier quoted context omitted.

I mean, it is magical, in a sense that we are not sure how and why it works.

I'm not sure how it works, but I'm sure it doesn't think. Not till it can choose its own loss function.

It's not like people can arbitrarily choose their own loss function; our drivers, needs and desires are what they are, you don't get to just redefine what makes you happy (otherwise clinical depression would not be a thing); they change over time and can be affected by various factors (things like heroin or brain injury can adjust your loss function) but it's not something within our conscious control. So I would not put that as a distinguishing factor between us and machines.

Re: AlphaCode as a dog speaking mediocre English

#78

Post by sanxiyn »

sillysaurusx wrote: post #36
sanxiyn wrote: post #35

Earlier quoted context omitted.

Sure, it doesn't think, just as submarines don't swim, as EWD said.

People always reach for these analogies. "Planes don't fly like birds." "Submarines don't swim like fish." Backpropagation has zero creativity. It's an elaborate mechanical parrot, and nothing more. It can never relate to you on a personal level, because it never experiences the world. It has no conception of what the world is. At least a dog gets hungry. Not persuaded? Try https://news.ycombinator.com/item?id=233469…

I agree GPT isn't grounded and it is a problem, but that's a weird point to argue against AlphaCode. AlphaCode is ground by actual code execution: its coding experience is no less real than people's.

AlphaGo is grounded because it experienced Go, and has a very good conception of what Go is. I similarly expect OpenAI's formal math effort to succeed. Doing math (e.g. choosing a problem and posing a conjecture) benefits from real world experience, but proving a theorem really doesn't. Writing a proof does, but it's a separate problem.

I think software engineering requires real world experience, but competitive programming probably doesn't.

Re: AlphaCode as a dog speaking mediocre English

#79

Post by shagie »

pessimizer wrote: post #68
shagie wrote: post #45

Earlier quoted context omitted.

Bunny is a neat dog... https://www.youtube.com/channel/UCEa46rlHqEP6ClWitFd2QOQ The vocal apparatus is not there, but there is certainly more cognition than what people think dogs have (there's a question that I wonder if language enables thought or if thought enables language) And this is even something that a cat can do (not as much language skills, but there's thought going on) https://www.youtube.com/c/BilliSpeak…

I'm not so certain. Seems like the owner is doing a lot of work to make sense out of those utterances. I'd like to see Bunny say what he's about to do, then do it. Or watch his owner do something with something, then describe it. edit: or just have a conversation of any kind longer than a 2 minute video. Or one without the owner in the room, where she talks back with the dog using the same board. That would at least…

The larger word array is... yea, it allows for a lot more interpretation of variation of intent and interpetation.

Consider this older one where there weren't as many words - https://youtu.be/6MMGmRVal6M and https://youtu.be/FPC6ElzSdxM

Or for billie the cat - https://youtu.be/DiuQqgTw-jY

The point with these is that there is language related thought going on there and an attempt to communicate from the dog (or cat) to the human.

Re: AlphaCode as a dog speaking mediocre English

#80

Post by darepublic »

Did I read correctly that the AI generates millions of potential solutions per question and then filters these down? That's an interesting approach and one that had occurred to me. Have a critic evaluate AI generates content and then have it take many tried to come up with a sample that passes the evaluator
Post reply on HN