381–390 of 405 posts
Earlier quoted context omitted.
Thanks, that's not a nitpick at all. Can you provide a citation? It's all over the internet as a Dijkstra quote, and I'd like to be correct.
It seems I made some confusions, you were actually right. Apologies...
Earlier quoted context omitted.
The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). The argument starts with a hypothetical (…
> The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). As I understand it, Yud is actually pro…
I don't think anyone seriously believes this. It's very very clear to all humans that have ever played a game of any kind that they can be defeated in unexpected ways. I don't even think that anyone believes the claim "it's impossible for AGI to pose an existential risk to humanity".
The negation of the claim "AGI poses an existential risk to humanity" is "AGI doesn't necessarily pose an existential risk to humanity". This is what most people in the world believe, and it is the obvious "null theory" about any technology.
> https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk...
The argument here works just as much for single-minded humans, so it's quite moot.
> https://arxiv.org/abs/2206.13353
Too long, sorry. Maybe I will read it someday, but not today.
> https://aiadventures.net/summaries/agi-ruin-list-of-lethalit...
This seems to agree with my previously stated positions. It does try to establish a canonical argument, as you say, but then it goes on to explain why they don't think it's persuasive.
Earlier quoted context omitted.
Don’t care! The problem at hand is people trying to argue that laws should be written in ways that are entirely unenforceable or have enormous gaping loopholes that undermine their stated goals.
Lots of laws look like that to me. I don't much like that laws seem so messy either, but that just doesn't seem like much of a reason to think it won't shake out like that. The law may well start on the basis of what people feel , and work outwards from there to things that might, or might not, be actually enforceable. And the headline summary of that law may or may not have much in common with the details, which is…
Earlier quoted context omitted.
It is surely better to have regulation now than scramble to catch up if ago is possible.
What about nation states? Do you really think the US military will avoid working towards AGI if they think it would give them a tactical advantage? Or the CCP? Or North Korea? Personally, if AGI gets developed, I’d rather the first iteration be in the hands of someone who doesn’t also have access to nuclear weapons.
Earlier quoted context omitted.
No, I have no doubt they would but I never suggested they should be allowed to. I think the sale and quality should be heavily regulated, advertising still banned, and requirements made to be allowed to purchase it. Like age, and who knows, maybe going through some basic harm reduction training. Certainly not advocating doing anything the way the US does it, in almost any respect. But the pharma thing is fundamentall…
I think we might actually be overall in agreement, just using different language. I believed that by legalization you meant something similar to how weed was legalized in various places - fully legal with minimal restrictions akin to alcohol or tobacco. If you're for making it fully legal but heavily regulated, somewhat akin to a prescription drug, then I more or less agree.
https://www.jrf.org.uk/prescribing-heroin-what-is-the-eviden....
My elevator-pitch-summary understanding of UK drugs laws is that almost anything can be prescribed by a doctor, but almost everything is illegal unless a doctor does so, and there's 3 levels of severity if you have or supply something without that authorisation.
Earlier quoted context omitted.
> The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). As I understand it, Yud is actually pro…
> P1: If intelligent system A cannot give a detailed account of how it would be bested by a more intelligent system B, then A will not be bested by B. P2: Humans (so far) cannot give a detailed account of how a more intelligent AI system would best them. C: So, humans will not be bested by a more intelligent AI system. I don't think anyone seriously believes this. It's very very clear to all humans that have ever pla…
Okay. So we agree that (A) powerful systems can best weaker systems in ways that are unexpected to the weaker system, and (B) it is possible that AGI poses an existential risk to humanity.
> The negation of the claim "AGI poses an existential risk to humanity" is "AGI doesn't necessarily pose an existential risk to humanity".
It seems to me that the negation of your first claim is just "AGI doesn't pose an existential risk to humanity". Is "necessarily" doing some important work in your second claim?
>> https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk...
> The argument here works just as much for single-minded humans, so it's quite moot.
I don't understand why the argument being applicable to humans would make it moot. Please explain.
>> https://aiadventures.net/summaries/agi-ruin-list-of-lethalit...
> This seems to agree with my previously stated positions. It does try to establish a canonical argument, as you say, but then it goes on to explain why they don't think it's persuasive.
Is there a particular premise or inferential step in the blog's argument that you believe to be mistaken? (I've copied the argument below.)
P1: The current trajectory of AI research will lead to superhuman AGI.
P2: Superhuman AGI will be capable of escaping any human efforts to control it.
P3: Superhuman AGI will be misaligned by default, i.e. it will likely adopt values and/or set long-term goals that will lead to extinction-level outcomes, meaning outcomes that are as bad as human extinction.
P4: We do not know how to align superhuman AGI, i.e. reliably imbue it with values or define long-term goals that will ensure it does not ultimately lead to an extinction-level outcome, without some amount of trial & error (how nearly all of scientific research works).
C1: P2 + P3 In the case of superhuman AGI, since it will be able to escape human control and misaligned by default, the only survivable path to alignment cannot involve trial & error because the first failed try will result in an extinction-level outcome.
C2: P4 + C1 This means we will not survive superhuman AGI, because our survival would require alignment, towards which we have no survivable path: the only path we know of involves trial & error, which is not survivable.
C3: P1 + C2 Therefore the current trajectory of AI research which will produce superhuman AGI leads to an outcome where we do not survive.Earlier quoted context omitted.
> The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). As I understand it, Yud is actually pro…
Of the three links you posted: 1. States things like "Finding goals that are extinction-level bad and relatively useful appears to be easy: for example, advanced AI with the sole objective ‘increase company.com revenue’ might be highly valuable to company.com for a time, but risks longer term harms to society, if powerfully accruing resources and power toward this end with no regard for ethics beyond laws that are st…
Are you claiming that (A) nice behavior in current LLMs is good evidence that all future AI systems will behave nicely, or (B) nice behavior in current LLMs is good evidence that future LLMs will behave nicely?
> 3. Requires accepting that we will by default build a misaligned superhuman AI that will cause humanity to go extinct as the basic premises of the argument (P1-P3), which makes the conclusions not particularly convincing if you don't already believe that.
P3 from the argument says, "Superhuman AGI will be misaligned by default". I interpret that as meaning: if there isn't a highly resourced and focused effort to align superhuman AGI systems in advance of their creation, then the first systems we build will be misaligned.
Is that the some way you are interpreting it? If so, why do you believe it is probably false?
Earlier quoted context omitted.
You presuppose that intelligence is like flight in the ways you've outlined (so solutions are going to converge). Frankly I don't know whether that's true or not, but I want to suggest that it's a bad bet: I would have sworn blind that consciousness is an essential component of intelligence, but the chatbots are starting to make that look like a poor assumption on my part. When we know so little about intelligence, c…
I agree we don't know enough to know if I'm right! I tried to use a lot of hedgy-words. But it's not a presupposition, merely a line of argument why it's not a complete absurdity to think LLMs might be a step towards AGI. I do think consciousness is beside the point, as we have no way to test whether LLMs are conscious, just like we can't test anything else. We don't know what consciousness is, nor what it isn't. I d…
Yes and yes.
> it can identify plausible steps to achieve that objective;
It can predict.
> it can accurately predict human responses to its actions;
It can predict.
> it can decently model the environment, as we can;
It can predict.
> it can hide its objectives from interrogators, and convince them that its actions are in their interests;
It can predict.
> it can deliver enough value to be capable of earning money through its actions;
It can predict.
> it can propose ideas that can convince investors to part with $100 billion;
It can predict.
> it can design a chemical plant...
It can predict.
Earlier quoted context omitted.
Given history, it's infinitely more dubious to link our future safety to some idea of progress stopping for some reason. We'll have increasingly smart AI and your go-to position is that progress will...stop? It's literally helping us with thinking, which is the driver of innovation and progress. Very few things have slowed down over decades or centuries. If anything it's been a mad rush with AI recently. Of course th…
It's still very much possible that intelligence as we understand it is fundamentally limited. That is, it's possible that the smartest possible being is not that much smarter than a human, just like the speed of matter and energy is limited to c. Of course, it's also very much possible that it's not: we don't have any good evidence either way.
Which is notoriously "not that much faster than a human?"
Earlier quoted context omitted.
Given history, it's infinitely more dubious to link our future safety to some idea of progress stopping for some reason. We'll have increasingly smart AI and your go-to position is that progress will...stop? It's literally helping us with thinking, which is the driver of innovation and progress. Very few things have slowed down over decades or centuries. If anything it's been a mad rush with AI recently. Of course th…
It's still very much possible that intelligence as we understand it is fundamentally limited. That is, it's possible that the smartest possible being is not that much smarter than a human, just like the speed of matter and energy is limited to c. Of course, it's also very much possible that it's not: we don't have any good evidence either way.