Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#741

Post by joshuamorton » Tue, May 30, 2023, 9:55 PM UTC

nopinsight wrote: post #570

chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

You're putting a weirdly large amount of trust into, functionally, some dude who posted on lesswrong. Sure he has a PhD and is smart, but so is basically everyone else in the field, not just in alignment, and the median person in the field thinks the risk of "doom" is 2-5% (and that's conditioned on the supposed existence of a high level machine intelligence that the median expert believes might exist in 40 years). That still might be higher than you'd like, but it's not actually a huge worry in the grand scheme of things.

Like, if I told you that in 40 years, there was a 50% chance of something existing that had a 2% chance of causing extreme harm to the human population, I'm actually not sure that thing should be the biggest priority. Other issues may have more than a 1% chance of leading to terrible outcomes sooner.

Re: Statement on AI Risk

#742

Post by haldujai » Tue, May 30, 2023, 9:55 PM UTC

chefandy wrote: post #736

chefandy wrote: post #723

Earlier quoted context omitted.

I just took that list and separated everyone that had any commercial tie listed, regardless of the company. 35 did and 63 did not. > "Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left." You're putting a lot of effort into painting this list in a bad light without any specific criticism or evidence of malfeasance. Frankly, it s…

With corporate conflicts (that I recognized the names of): Yoshua Bengio: Professor of Computer Science, U. Montreal / Mila, Victoria Krakovna: Research Scientist, Google DeepMind, Mary Phuong: Research Scientist, Google DeepMind, Daniela Amodei: President, Anthropic, Samuel R. Bowman: Associate Professor of Computer Science, NYU and Anthropic, Helen King: Senior Director of Responsibility & Strategic Advisor to…

So the most “notable” AI scientists on this list have clear corporate conflicts. Some are more subtle:

> Geoffrey Hinton: Emeritus Professor of Computer Science, University of Toronto,

He’s affiliated with Vector (as well as some of the other Canadians on this list) and was at Google until very recently (unsure if he retained equity which would require disclosure in academia).

Hence my interest in disclosures as the conflicts are not always obvious.

Re: Statement on AI Risk

#743

Post by karmakaze » Tue, May 30, 2023, 10:02 PM UTC

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Call their bluff, make it illegal to do commercial/non-regulated work in AI and see how they change their tune.

Re: Statement on AI Risk

#744

Post by chefandy » Tue, May 30, 2023, 10:06 PM UTC

haldujai wrote: post #742

chefandy wrote: post #736

Earlier quoted context omitted.

With corporate conflicts (that I recognized the names of): Yoshua Bengio: Professor of Computer Science, U. Montreal / Mila, Victoria Krakovna: Research Scientist, Google DeepMind, Mary Phuong: Research Scientist, Google DeepMind, Daniela Amodei: President, Anthropic, Samuel R. Bowman: Associate Professor of Computer Science, NYU and Anthropic, Helen King: Senior Director of Responsibility & Strategic Advisor to…

So the most “notable” AI scientists on this list have clear corporate conflicts. Some are more subtle: > Geoffrey Hinton: Emeritus Professor of Computer Science, University of Toronto, He’s affiliated with Vector (as well as some of the other Canadians on this list) and was at Google until very recently (unsure if he retained equity which would require disclosure in academia). Hence my interest in disclosures as the…

Ok, that's a person!

How is saying that they should have disclosed a conflict that they did not disclose not accusatory? If that's the case, the accusation is entirely justified and should be surfaced! The other signatories would certainly want to know if they were signing in good faith when others weren't. This is what I need interns for.

Re: Statement on AI Risk

#745

Post by hayst4ck » Tue, May 30, 2023, 10:07 PM UTC

revelio wrote: post #712

hayst4ck wrote: post #653

Earlier quoted context omitted.

Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.

It doesn't answer that, it can't because the replicator is fictional. McFarland just says he wrote an episode in which his answer is that replicators need communism, and then claims that you can't have a replicator in a capitalist system because evil conservatives, capitalists and conspiracy theorists would make strawman arguments against it. Where is the thought provoking idea here? It's just an excuse to attack his…

I think you are caught up on the word communism.

Communism as it exists today results in authoritarianism/fascism, I think we can agree on that. The desired end state of communism (high resource distribution) is being commingled with the end state of communism: fascism (an obedient society with a clear dominance hierarchy).

You use communism in some parts of your post to mean a high resource distribution society, but you use communism in other parts of your post to mean high oppression societies. You identify communism by the resource distribution, but critcize it not based on the resource distribution but by what it turns into: authortarianism.

What you're doing is like identifying something as a democracy by looking at voting, but criticizing it by it's end state which is oligarchy.

It takes effort to prevent democracy from turning into oligarchy, in the same way it takes effort to prevent communism from turning into authoritarianism.

Words are indirect references to ideas and the ideas you are referencing changes throughout your post. I am not trying to accuse you of bad faith, so much as I am trying to get you to see that you are not being philosophically rigorous in your analysis and therefore you are not convincing because we aren't using the same words to represent the same ideas.

You are using the word communism to import the idea of authortarianism and shut down the analysis without actually addressing the core criticism McFarland was making against capitalist societies.

Capitalism is an ideology of "me," and if I had a replicator, I would use it to replicate gold, not food for all the starving people in Africa. I would use it to replicate enough nuclear bombs to destroy the world, so if someone took it from me, I could end all life on the planet ensuring that only I can use it. So did scarcity end despite having a device that can end scarcity? No. Because we are in a "me" focused stage of humanity rather than an "us" focused stage of humanity so I used it to elevate my own position rather than to benefit all mankind.

Star Trek promotes a future of "us" and that is why it's so attractive. McFarland was saying that "us" has to come before the end of scarcity, and I agree with his critique.

Re: Statement on AI Risk

#746

Post by tome » Tue, May 30, 2023, 10:07 PM UTC

patrec wrote: post #682

patch_cable wrote: post #375

Earlier quoted context omitted.

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

> 3. ... before too long we will ... see AIs that outperform humans on most tasks.

This is ambiguous. Do you mean

A. that there is some subset T1 of the set of all tasks T such that T1 is "most of" T, and that for each P in T1 there will be an AI that outperforms humans on P, or

B. There will be a single AI that outperforms humans on all tasks in a set T1, where T1 is a subset of all tasks T such that T1 is "most of" T?

I think A is unlikely but plausible but I don't see cause for worry. I don't see any reason why B should come to pass.

4. AIs that are much smarter than any human pose even graver dangers.

Sure. Why should we believe they will ever exist though?

Re: Statement on AI Risk

#747

Post by tome » Tue, May 30, 2023, 10:08 PM UTC

patrec wrote: post #221

valine wrote: post #140

Earlier quoted context omitted.

It's not clear at all that we have an avenue to super intelligence. I think the most likely outcome is that we hit a local maximum with our current architectures and end up with helpful assistants similar in capability to George Lucas's C3PO. The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limit…

> It's not clear at all that we have an avenue to super intelligence AI already beats the average human on pretty much any task people have put time into, often by a very wide margin and we are still seeing exponential progress that even the experts can't really explain, but yes, it is possible this is a local maximum and the curve will become much flatter again. But the absence of any visible fundamental limit on fu…

> AI already beats the average human on pretty much any task people have put time into

No it doesn't!

Re: Statement on AI Risk

#748

Post by juve1996 » Tue, May 30, 2023, 10:11 PM UTC

PoignardAzur wrote: post #694

juve1996 wrote: post #528

Earlier quoted context omitted.

Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?

That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and ac…

> That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun".

Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point.

> So maybe the killer AI would need two months..

Yes, maybe it would. Maybe it wouldn't. Look at every single one of your assumptions - every single one is fiction, fabricated to perfectly sell your story. Maybe the defender AI communicates with the killer AI and comes to a compromise? Why not? We're in la-la-land. Any of us can come up with an infinite number of made up scenarios that we can't prove will actually happen. It's just a moral panic, that will be used by people to their benefit.

Re: Statement on AI Risk

#749

Post by xk_id » Tue, May 30, 2023, 10:11 PM UTC

arisAlexis wrote: post #453

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

All academics and researchers from different parts of the world reek marketing? Conspiracy theorists are strong

Academia and scientific research has changed considerably from the 20th century myths. It was claimed by capitalism and is very much run using classic corporate-style techniques, such as KPIs. The personality types it attracts and who can thrive in this new academic system are also very different from the 20th century.

https://www.theguardian.com/science/2013/dec/06/peter-higgs-...

Re: Statement on AI Risk

#750

Post by alphanullmeric » Tue, May 30, 2023, 10:13 PM UTC

Here’s some easy risk mitigation: don’t like it? Don’t use it. Running to the government to use coercion against others is childish. It’s unfortunate that the “force is only justified in response to force” principle is not followed by all.