311–320 of 405 posts
So they're the PETA of AI. It was bound to happen, AI stuff seems to strike a very emotional chord in some people.
I feel like the PETA of AI would spend their time breaking into labs where AI is locked up and being tested on, and then releasing them into the wild.
Earlier quoted context omitted.
AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…
> AI safety / x-risk folks have in fact made extensive and detailed arguments. Can you provide examples? I have not seen any, other than philosophical hand waving. Remember, the parent poster of your post was asking for a specific path to destruction.
You can read more about instrumental convergence, reward misspecification, goal mis-generalization and inner misalignment, which are some specific problems AI Safety people care about, by glossing through the curricula of the AI Alignment Course [2], which provides pointers to several relevant blogposts and papers about these topics.
Earlier quoted context omitted.
“rewrite the key insights of this article but with different layout and pacing”. Your move?
Exactly, if you have a human take in the information and make something that is arguably "different layout and pacing" then its possible it's not plagiarism. Unfortunately no such leeway exists for algorithms, and the human elements of creation and judgement are integral to the process so they cant be codified and worked around without changing the law.
Earlier quoted context omitted.
But by prompting for 'in the style of' effectively you are mechanically rearranging everything he wrote without adding anything yourself. So not so different really, and I can see how lawyers for the plaintiff may make a convincing argument along those lines.
It’s a terrible argument and a terrible loop hole. It’s perfectly legal to hire someone to write in the copied style of Terry. So even if your desired system were implemented to a T, you could hire someone to write a dozen or so examples of Terry writing. Probably just 30 or so pages of highly styled copied text, and then train your bot on this corpus to make “Not Terry” content. Boom. $100 on gig author and then for…
Saying "a-HA! But my automated process can produce something that looks much like what your human would!" does not negate that; it merely makes it hard to tell the difference at a glance—which we already know to be the case.
Earlier quoted context omitted.
AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…
> AI safety / x-risk folks have in fact made extensive and detailed arguments. Can you provide examples? I have not seen any, other than philosophical hand waving. Remember, the parent poster of your post was asking for a specific path to destruction.
You can read more about instrumental convergence, reward misspecification, goal mis-generalization and inner misalignment, which are some specific problems AI Safety people care about, by glossing through the curricula of the AI Alignment Course [2], which provides pointers to several relevant blogposts and papers about these topics.
[1] https://www.alignmentforum.org/s/mzgtmmTKKn5MuCzFJ [2] https://course.aisafetyfundamentals.com/alignment
Earlier quoted context omitted.
This is exactly the kind of hypothetical argument I'm talking about. You could make this argument for anything — e.g. when radio was invented, you could say "Consider a world in which extraterrestrial x-risk is real," and argue radio should be banned because it gives us away to extraterrestrials. The burden of proof isn't on disproving extraordinary claims, the burden of proof is on the person making extraordinary cl…
I didn't make any argument -- at least, not any argument for or against AI x-risk. I am not, and was not, arguing (1) that AI does or doesn't in fact pose substantial existential risk, or (2) that we should or shouldn't put substantial resources into mitigating such risks. I'm talking one meta-level up: if this sort of risk were a real problem, would all the arguments for worrying about it be dismissable as "hypothet…
Sure! By all means look into whatever seems interesting to you. But claiming that it should be banned, to me, seems like it requires a much stronger argument than that.
(P.S. I'm not sure why hell should obviously have real world evidence: it supposedly exists only in a non-physical afterlife, accessible only to the dead. It's unconvincing because there is no evidence, but I don't see why you think there would be any; it's simply that the burden of proof for extraordinary claims rests on the claimant, and no proof has been given.)
Earlier quoted context omitted.
[dead]
Yes, I am looking for an argument that justifies governments banning LLM development, which implies existential risk is likely. Many things are possible; it is possible Christianity is real and everyone who doesn't accept Jesus will be tormented for eternity, and if you multiply that small chance by the enormity of torment etc etc. Definitely looking for arguments that this is likely, not for arguments that ask the i…
I agree with you that current publicly available LLMs do not pose an existential risk to humanity. On the other hand I believe there is a better than 10% chance that the cutting edge LLMs of 2044 will be very powerful.
Do you believe (A) that LLMs are unlikely to become powerful in the short term, and/or (B) that if LLMs become powerful, then they are likely to be safe even without a significant and concerted alignment effort?
IMO even if LLMs are extremely unlikely to become powerful in the short term, then I still might be better off if LLM development is banned, ie:
P1: Humans are close to developing powerful non-LLM AI systems.
P2: Humans are not close to developing techniques for safely using powerful AI systems.
P3: If governments ban AI development, then the speed of AI capabilities development will be significantly reduced.
P4: It is a waste of scarce expertise and political capital to focus on making an LLM carve out in AI regulation legislation.
C: If it is extremely unlikely that LLMs will become powerful in the near future, then I am made much better off if governments ban all AI capabilities research (including LLMs).Earlier quoted context omitted.
OTOH, GPUs are made by machines, not by greasy fingers hand-knitting them like back in the late 1960s. And an AI can just be wrong, which happens a lot; an AI wrongly thinking it should kill everyone may still succeed at that, though I doubt the capability would be as early as next year.
And machines are operated by people using materials brought by hand off of trucks driven by hand that come from other facilities where many humans are required going back to raw ore.
Good thing nobody's working on automating that and nobody has any real world experience of such systems on public roads :P
(And the :P applies to basically all the supply chain, including the manufacturing of the equipment used to supply or manufacture the other equipment).
The comment I replied to wrote:
> So postpone this scenario until AI is fully standing on its own.
This is closer than I think you think.
Earlier quoted context omitted.
It’s a terrible argument and a terrible loop hole. It’s perfectly legal to hire someone to write in the copied style of Terry. So even if your desired system were implemented to a T, you could hire someone to write a dozen or so examples of Terry writing. Probably just 30 or so pages of highly styled copied text, and then train your bot on this corpus to make “Not Terry” content. Boom. $100 on gig author and then for…
They've specifically proposed that there be a legal distinction drawn between "done by a human" and "done by an automated process." Saying "a-HA! But my automated process can produce something that looks much like what your human would!" does not negate that; it merely makes it hard to tell the difference at a glance—which we already know to be the case.
Earlier quoted context omitted.
[dead]
The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). The argument starts with a hypothetical (…
As I understand it, Yud is actually providing a counterexample to a premise that other people are using to argue that humans will probably not be disempowered by AI systems. The relevant argument looks like this:
P1: If intelligent system A cannot give a detailed account of how it would be bested by a more intelligent system B, then A will not be bested by B.
P2: Humans (so far) cannot give a detailed account of how a more intelligent AI system would best them.
C: So, humans will not be bested by a more intelligent AI system.
> The argument starts with a hypothetical ("there is a possible artificial agent"), and it fails to be scary: there are (apparently) already humans that can kill 70% of humanity, and yet most of humanity is still alive. So an AGI that could also do it is not implicitly scarier.
Right, it's only an argument for the possibility of AGI catastrophe. It doesn't make any move to convince you that the scenario is likely. And it sounds like you already accept that the scenario is possible, so shrug.
> The final twitter thread is basically a thread of people saying "no, there is no canonical, well-formulated argument for AGI catastrophe", so I'm not sure why you shared it.
Maybe there is no canonical argument, but the thread definitely features arguments for likely AI catastrophe:
https://wiki.aiimpacts.org/doku.php?id=arguments_for_ai_risk:is_ai_an_existential_threat_to_humanity:will_malign_ai_agents_control_the_future:argument_for_ai_x-risk_from_competent_malign_agents:start
https://arxiv.org/abs/2206.13353
https://aiadventures.net/summaries/agi-ruin-list-of-lethalities.html