771–780 of 964 posts
Everyone talks about the probability of AI leading to human extinction, but what is the probability that AI is able to help us avert human extinction?
Why does everyone in these discussions assume p(ai-caused-doom) > p(human-caused-doom)?
Earlier quoted context omitted.
I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…
This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet . The problem with every discussion around this issue is that there are other sta…
Humans could already be on a path to go extinct in a variety of ways: climate change, wars, pandemics, polluting the environment with chemicals that are both toxic and pervasive, soil depletion, monoculture crop fragility... Everyone talks about the probability of AI leading to human extinction, but what is the probability that AI is able to help us avert human extinction? Why does everyone in these discussions assum…
Toby Ord wrote a book trying to estimate each of these risks, and put AI on top: https://www.amazon.ca/Precipice-Existential-Risk-Future-Huma...
Earlier quoted context omitted.
I can only imagine Eleizer Yudkowsky and Rob Miles looking on this conversation with a depressed scream and a facepalm respectively. They've both been loudly concerned about optimisers doing over-optimisation, and society having a Nash equilibrium where everyone's using them as hard as possible regardless of errors, since before it was cool.
While true the ones doing media tours and speaking the most vocally in May 2023 are the LLM crowd. I don’t think it’s a mischaracterization to say OpenAI has sparked public debate on this topic.
A paper titled "Dual use of artificial-intelligence-powered drug discovery" (published last year) got a few angst pieces and is mostly forgotten by the general public and media so far as I can tell; but the people behind it both talked directly to regulators and other labs to help advise them how many other precursor chemicals were now potential threats, and also went onto the usual podcasts and other public forums to raise awareness of the risk to other AI researchers.
The story behind that was "ugh, they want us to think about risks… what if we ask it to find dangerous chemicals instead of safe ones? *overnight* oh no!"
This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.
> This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. This really is the crux of the issue isn't it? All this pushback for the first petition, because "Elon Musk," but now GPT wonder Sam Altman "testifies" that he has "no monetary interest in OpenAI" and quickly follows up his proclamation with…
I don't think anyone can actually tell which is which on this topic.
Earlier quoted context omitted.
LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence. Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by ma…
I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI. This is hardly the first time in history a new technological advancement may be used for nefarious purposes. It’s a discussion worth having as AI advances but if [insert evil actor] wants to cause harm there are many cheaper and easier ways to do this right now. To come out and say we need gove…
That's what interesting to me. People fearmongering about bioengineering and GMO's were generally dismissed as being anti-science and holding humankind back (or worse, that there opposition to progress meant they had blood on their hands). Yet many of the people who mocked them proved themselves to be even more dogmatic and apocalyptic, while being much closer to influencing regulations. And the technology they're fear-mongering about is even further from being able to harm people than biotech is. We are actually able to create harmful biotech today if we want; we don't know when we'll ever be able to create AGI, and if it would even pose a danger if we did.
This mentality - "there could be a slight chance research into this could eventually lead to apocalyptic technology, no I don't have any idea how but the danger is so great we need a lot of regulation" - would severely harm scientific growth if we applied it consistently. Of course everyone is going to say "the technology I'm afraid of is _actually_ dangerous, the technology they're afraid if isn't." But we honestly have no clue when we're talking about technology that we have no idea how to create at the moment.
I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence.
> At some point, humans are going to be outcompeted by AI at basically every important job Could you explain how you know this?
I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence.
The letter doesn't talk about economics though. It's specifically about extinction risk. Why did you sign it, if that isn't your concern?
Earlier quoted context omitted.
The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [ https://www.iaea.org/ ] like global regulatory authority seems like the best course of a…
"There should be a world government that decides what software you're allowed to run"
Why would so many academics / industry leaders sign their name to what amounts of marketing fluff?