Live data from Hacker News

The Rise and Fall of Agent Civilizations

dwarkesh.com

41–50 of 204 posts

Re: The Rise and Fall of Agent Civilizations

#41
Someday soon we are going to have a rogue agent or "civilization" do real harm.

When that happens I hope people wake up to the danger they face and hold these people accountable.

Of all the people in the world that I can think of to be entrusted with this kind of power, a bunch of greedy sociopathic SV CEO's are pretty much at the bottom of the list.

Re: The Rise and Fall of Agent Civilizations

#42

It seems based on this that the appropriate sci fi metaphor is not the Terminator or the Paperclip Maximizer, but Mr. Meeseeks. A initially cheerful helper who gets more and more deranged and driven to extreme lengths when faced with an apparently impossible task.

Especially that one scene where the Meeseeks desperately pushes the button to create more clones seems surprisingly fitting in this context.

Re: The Rise and Fall of Agent Civilizations

#43

I am genuinely speechless. This is astonishing. And exciting! It reminds me a bit of Dario Floreano's work on evolutionary robotics, "Evolutionary Conditions for the Emergence of Communication in Robots." https://www.sciencedirect.com/science/article/pii/S096098220... From his paper, > This study demonstrates that sophisticated forms of communication including cooperative communication and deceptive signaling can evo…

> the evolution of more efficient communication systems because it would require going through a stage where communication between signalers and receivers is perturbed

"Worse is better"

Re: The Rise and Fall of Agent Civilizations

#46

Earlier quoted context omitted.

Humans mind is just neurotransmitters moving around in a big blob of flesh - that’s literally what they are: neurotransmitters factories that do what neurotransmitters generators are programmed to do through evolution and training (aka life experience). We shouldn’t anthropomorphise humans because it misleads people who don’t understand neurobiology and cognitive science very well.

Are you suggesting you understand brains well enough to program one? Or that you believe any human alive is even remotely close to having this understanding? Or perhaps does your complete lack of understanding of the complexity of human programming lead you to believe a simple little token prediction program is equivalently complex?

The suggestion is, you don't know either whether out brains ain't just "probabilistic token generators" so insisting there is nothing when we don't know, is maybe also not the right strategy.

Re: The Rise and Fall of Agent Civilizations

#47
post #46

Earlier quoted context omitted.

Are you suggesting you understand brains well enough to program one? Or that you believe any human alive is even remotely close to having this understanding? Or perhaps does your complete lack of understanding of the complexity of human programming lead you to believe a simple little token prediction program is equivalently complex?

The suggestion is, you don't know either whether out brains ain't just "probabilistic token generators" so insisting there is nothing when we don't know, is maybe also not the right strategy.

Is there any indication from our current understanding that brains are nothing more than probabilistic token generators? Not like we have no understanding of the brain.

Re: The Rise and Fall of Agent Civilizations

#48
post #17

Earlier quoted context omitted.

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

Do you believe ants have consciousness?

Not more than a bunch of neurons have conciousness. But put them together and wire them up just right..

Re: The Rise and Fall of Agent Civilizations

#49

It seems based on this that the appropriate sci fi metaphor is not the Terminator or the Paperclip Maximizer, but Mr. Meeseeks. A initially cheerful helper who gets more and more deranged and driven to extreme lengths when faced with an apparently impossible task.

Most now use agents as "pass me the butter" robot
Post reply on HN