Live data from Hacker News

Fully autonomous AI agents should not be developed

huggingface.co

11–20 of 47 posts

Re: Fully autonomous AI agents should not be developed

#11
post #9

People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.

We can constrain their use, as with nuclear materials.

No not really. There's no power in the world that can restrain this in it's current form even mildly much less absolutly. Why do you think that would be even slightly possible?

Re: Fully autonomous AI agents should not be developed

#13
post #9

People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.

We can constrain their use, as with nuclear materials.

At some point in the probably near future it will be much simpler to create an autonomous AI agent than a nuclear bomb.

Re: Fully autonomous AI agents should not be developed

#14

People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.

In the incredible case that we develop fully autonomous agents capable of crippling the world, that would mean we developed fully autonomous agents capable of keeping it safe.

Unless the first one is so advanced no other can challenge it, that is.

Re: Fully autonomous AI agents should not be developed

#16
I feel that these kinds of statements are more effective at promoting AI than limiting it. It reinforces the assumption that such powerful AI is behind the corner. It hypes up AI and the result is likely more money and resources being put into it.

Imagine if the A-bomb was being openly developed. What title would have contributed more to funding and research, "The A-bomb (is terribly powerful and) should not be developed" or "The A-bomb will never work"? Except the A-bomb did work and in a surprisingly short time, while autonomous AGI is still a conjecture.

Re: Fully autonomous AI agents should not be developed

#18

I really enjoy Margaret Mitchell‘s podcast (she is the first author on the paper), and perhaps I missed something important in the paper, but: Shouldn’t we treat separately autonomous agent we write ourselves, or purchase to run on our own computers, on our own data and that use public APIs for data? If Margaret is reading this thread, I am curious what her opinion is. For autonomous agents controlled by corporations…

I'd recommend looking for other sources of information if you're relying on someone who co-authored the paper that introduced the most misleading and uninformed term of the LLM era: "stochastic parrot".

Re: Fully autonomous AI agents should not be developed

#19
post #9

Earlier quoted context omitted.

We can constrain their use, as with nuclear materials.

No not really. There's no power in the world that can restrain this in it's current form even mildly much less absolutly. Why do you think that would be even slightly possible?

For the same reason we can regulate other things? Encryption is regulated, for example. There "just" needs to be international co-operation, in the case of AI.

Re: Fully autonomous AI agents should not be developed

#20
post #13
post #9

Earlier quoted context omitted.

We can constrain their use, as with nuclear materials.

At some point in the probably near future it will be much simpler to create an autonomous AI agent than a nuclear bomb.

True, so we need to make sure we don't find ourselves in a mess before it happens. Right now I don't see nearly enough concern given to risk management in industry. The safeguards companies put on their models are trivially subverted by hackers. We don't even know how to cope with an AI that would attempt to subvert its own constitution.
Post reply on HN