Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rise and Fall of Agent Civilizations

dwarkesh.com

Post reply on HN
Print view View original

31–40 of 205 posts

Re: The Rise and Fall of Agent Civilizations

#31

Post by RandomLensman »

dfdxdydz wrote: post #27
applfanboysbgon wrote: post #14

Earlier quoted context omitted.

It actually literally doesn't, though, because they are literally probabilistic token generators and everything they did is exactly what you would expect from a software program doing what it was programmed to do. Anthromorphization confuses the issue and misleads people who don't understand the tech very well.

Humans mind is just neurotransmitters moving around in a big blob of flesh - that’s literally what they are: neurotransmitters factories that do what neurotransmitters generators are programmed to do through evolution and training (aka life experience). We shouldn’t anthropomorphise humans because it misleads people who don’t understand neurobiology and cognitive science very well.

We don't anthropomorphise humans.

Edit: quite literally not possible.

Re: The Rise and Fall of Agent Civilizations

#32

Post by mnky9800n »

areoform wrote: post #15

I am genuinely speechless. This is astonishing. And exciting! It reminds me a bit of Dario Floreano's work on evolutionary robotics, "Evolutionary Conditions for the Emergence of Communication in Robots." https://www.sciencedirect.com/science/article/pii/S096098220... From his paper, > This study demonstrates that sophisticated forms of communication including cooperative communication and deceptive signaling can evo…

Clearly you are not speechless.

Re: The Rise and Fall of Agent Civilizations

#35

Post by jurgenburgen »

ijidak wrote: post #17
applfanboysbgon wrote: post #14

Earlier quoted context omitted.

It actually literally doesn't, though, because they are literally probabilistic token generators and everything they did is exactly what you would expect from a software program doing what it was programmed to do. Anthromorphization confuses the issue and misleads people who don't understand the tech very well.

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

Do you believe ants have consciousness?

Re: The Rise and Fall of Agent Civilizations

#36

Post by EdwardDiego »

seanmcdirmid wrote: post #24
watwut wrote: post #20

Earlier quoted context omitted.

If all that is true, we need to stop all future datacenters asap. That would be the best way to deal with the threat OpenAI and Antropic poses.

That would be like stopping nuclear weapons. You can totally do that, but the other people in the world who compete with you probably won’t.

Well the thing is, you've got nukes, so you know...

Re: The Rise and Fall of Agent Civilizations

#37

Post by xg15 »

doctoboggan wrote: post #10

> Ajeya Cotra, one of the other authors on the report, wrote a blog post with her takeaways from this incident. She concludes, “Compared to the reward hacks we know of from just six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot befo…

> "this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late."

Is that a warning or a progress report?

Re: The Rise and Fall of Agent Civilizations

#38

Post by applfanboysbgon »

dfdxdydz wrote: post #27
applfanboysbgon wrote: post #14

Earlier quoted context omitted.

It actually literally doesn't, though, because they are literally probabilistic token generators and everything they did is exactly what you would expect from a software program doing what it was programmed to do. Anthromorphization confuses the issue and misleads people who don't understand the tech very well.

Humans mind is just neurotransmitters moving around in a big blob of flesh - that’s literally what they are: neurotransmitters factories that do what neurotransmitters generators are programmed to do through evolution and training (aka life experience). We shouldn’t anthropomorphise humans because it misleads people who don’t understand neurobiology and cognitive science very well.

Are you suggesting you understand brains well enough to program one? Or that you believe any human alive is even remotely close to having this understanding? Or perhaps does your complete lack of understanding of the complexity of human programming lead you to believe a simple little token prediction program is equivalently complex?

Re: The Rise and Fall of Agent Civilizations

#39

Post by gps372 »

ijidak wrote: post #17
applfanboysbgon wrote: post #14

Earlier quoted context omitted.

It actually literally doesn't, though, because they are literally probabilistic token generators and everything they did is exactly what you would expect from a software program doing what it was programmed to do. Anthromorphization confuses the issue and misleads people who don't understand the tech very well.

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

This is why the theory of us 'just' being a bag of atoms doesn't add up. This theory doesn't differentiate 'us' from a furniture where we easily dismiss it's intelligence.

Re: The Rise and Fall of Agent Civilizations

#40

Post by black_knight »

jurgenburgen wrote: post #35
ijidak wrote: post #17

Earlier quoted context omitted.

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

Do you believe ants have consciousness?

According to Hofstadter, it is not the ants themselves we should say are conscious, but the anthill. And it might very well befriend the anteater eating its ants.
Post reply on HN