21–30 of 47 posts
People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.
In the incredible case that we develop fully autonomous agents capable of crippling the world, that would mean we developed fully autonomous agents capable of keeping it safe. Unless the first one is so advanced no other can challenge it, that is.
> • Simple→Tool Call: Inaccuracy propagated to inappropriate tool selection.
> • Multi-step: Cascading errors compound risk of inaccurate or irrelevant outcomes.
> • Fully Autonomous: Unbounded inaccuracies may create outcomes wholly unaligned with human goals.
Just... lol
People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.
In the incredible case that we develop fully autonomous agents capable of crippling the world, that would mean we developed fully autonomous agents capable of keeping it safe. Unless the first one is so advanced no other can challenge it, that is.
Earlier quoted context omitted.
No not really. There's no power in the world that can restrain this in it's current form even mildly much less absolutly. Why do you think that would be even slightly possible?
For the same reason we can regulate other things? Encryption is regulated, for example. There "just" needs to be international co-operation, in the case of AI.
People are going to be developing these no matter what. Whether it wipes us out or not is just up to fate really.
We can constrain their use, as with nuclear materials.
If we only handle AI that well doom is probable. It has economic uses, unlike nuclear weapons, so there will be a thriving black market dodging the safety concerns.
I really enjoy Margaret Mitchell‘s podcast (she is the first author on the paper), and perhaps I missed something important in the paper, but: Shouldn’t we treat separately autonomous agent we write ourselves, or purchase to run on our own computers, on our own data and that use public APIs for data? If Margaret is reading this thread, I am curious what her opinion is. For autonomous agents controlled by corporations…
I'd recommend looking for other sources of information if you're relying on someone who co-authored the paper that introduced the most misleading and uninformed term of the LLM era: "stochastic parrot".
in this paper, it's clear that the authors don't think modern LLM-based systems are just stochastic parrots.
Earlier quoted context omitted.
We can constrain their use, as with nuclear materials.
It is a lot easier to detect illicit nuclear work compared to illicit AI work.
Also there are only a few companies that can fab the semiconductors needed for these training runs.