81–90 of 110 posts
At this point I feel like the boy who cried wolf is an ai researcher. Yes super intelligence could be dangerous. However, is the story a PR stunt or real? Well…
How is a researcher quitting a PR stunt?
To be clear, I don't believe this is what happened here, I'm typing this half jokingly, just I wouldn't be surprised.
An AI that generates text will never be scary to me. An autonomous AI with facial recognition on a flying drone with weapons (bombs/guns) with swarming capabilities will always be terrifying. I feel like we are ignoring the massive elephant in the room.
The killer robots are expensive and dependent on physical supply chains. While text is sufficient to radicalize humans into attacks.
The problem is they are too cheap, a drone that cost a couple thousand can wipe out millions of dollars of “defense”.
Earlier quoted context omitted.
How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it. I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.
Lab A reaches ASI, and is prompted the following: "Permanently nullify all other AI labs".If it's ASI it would succeed.
A lot of these scenarios seem to assume that nobody else gets a move. That there wouldn't be a human response.
What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.
China has a better track record of regulating their big tech than the USA.
The fundamental point I think is far too often confused is the difference between LLM and agentic system. An LLM can't do anything but generate tokens. You run your LLM in vLLM or whatever, and it generates output tokens based on your input tokens. That's it! Humans then build ~deterministic systems to take those tokens and do all sorts of things with the tokens, like take actions in the real world. And then we can f…
The real question then is: are human beings responsible enough, en masse, to wield these tools without restrictions? Would you let a toddler play with a loaded gun, even though without human action the gun is harmless? Maybe collectively we’re all little better, and eventually one of us is going to “pull the trigger” of ai just because it was shiny.
Ironically, I also don’t believe in regulation, unless that regulation is crushing and draconian. The only regulation that would seem to make sense is status quo anti ai, with only highly supervised research labs being able to interact with it. Clearly that’s unrealistic without the preceding calamity: millions had to die in WWI before we instilled a global taboo against chem weapons on ourselves. I don’t want either to happen (regulation or calamity), but I feel individually powerless to stop the relentless drive we (and I) seem to have in curiositying ourselves to death. So I’m stuck with all the rest of us, building SaaS apps and iPhone games.
I particularly like the last point he makes here: Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. It's interesting to see so much hate towards creators who use AI to make almost any type of creative work. At least these are humans using it as "controllable tools". Nuclear-powered b…
> Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. Is the idea here that no field needs experiments and data anymore (which can take a lot of time) to be revolutinized and just "thinking" would be enough? How would an AI itself own anything?
> Both OpenAI and Anthropic have recently flagged incidents in which agents powered by their models went rogue I may be biased and somewhat off topic, but I see these incidents as some of the most significant of the past century. I genuinely don't understand why these companies aren't taking a smarter approach to them. The latest analyses have been, at best, laughable: identify the vulnerability, patch it, and move o…
It does seem bizarre that major AI development (and autonomous robot development) hasn't been nationalised yet and treaties drafted up around producing it. Commercial incentives just seem totally at odds with the public good when it comes to controlling and regulating something like this. Even if not for the sake of avoiding a mitigable disaster, there's a huge benefit in simply pacing development so as to not comple…
The US absolutely could coerce the rest of the West in the short term, but everything that's happened in the last decade calls the long-term future of Atlanticism into serious question and such action would accelerate this process I think. If coordination came as an order from Uncle Sam, it would be seen not as a legitimate act of ensuring collective safety but as mere technological imperialism on America's part.
Why would AI wipe us out? We have not wiped out apes, ants, and most other species. We even have discussions about how to actively save them from extinction.
The classic thought experiment is the paper clip optimizer. Quoting Nick Bostrom: > Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into pa…
Earlier quoted context omitted.
Lab A reaches ASI, and is prompted the following: "Permanently nullify all other AI labs".If it's ASI it would succeed.
I am explicitly asking people to fill in the blank on how it would succeed. It's intelligence, not magic. A lot of these scenarios seem to assume that nobody else gets a move. That there wouldn't be a human response.