Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

751–760 of 964 posts

Re: Statement on AI Risk

#751

Post by tome »

HDThoreaun wrote: post #530
valine wrote: post #140

Earlier quoted context omitted.

It's not clear at all that we have an avenue to super intelligence. I think the most likely outcome is that we hit a local maximum with our current architectures and end up with helpful assistants similar in capability to George Lucas's C3PO. The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limit…

A super intelligent AI is not necessary for AI to be an threat. Dumb AIs that are given access to the internet plus a credit card and told to maximize profit could easily cause massive damage. We are not far from such an AI being accessible to the masses. You can try to frame this like the gun debate "it's not the AI it's the people using it" but the AI would be acting autonomously here. I have no faith that people w…

> Dumb AIs that are given access to the internet plus a credit card and told to maximize profit could easily cause massive damage

North Korea and Iran are (essentially) already trying to do that, so I think that particular risk is well understood.

Re: Statement on AI Risk

#752

Post by jhanschoo »

HDThoreaun wrote: post #534
shafyy wrote: post #147

Earlier quoted context omitted.

This longtermist and Effective Altruism way of thinking is very dangerous. Because using this chain of argumentation, it's "trivial" to say what you're just saying: "So what if there's racism today, it doesn't matter if everybody dies tomorrow. We can't just say that we weigh humanity's extinction with a big number, and then multiply it by all humans that might be born in the future, and use that to say today's REAL…

Today's real issues are not that important compared to human extinction.

It seems to me that the most viable routes to human extinction are through superscaled versions of contemporary disasters: war, disease, and famine.

Re: Statement on AI Risk

#753

Post by rockemsockem »

sanderjd wrote: post #223
rockemsockem wrote: post #186

Earlier quoted context omitted.

Cite where it is being said that these companies are arguing "to make their models legal, but any competitive upstarts illegal". As far as I know nothing of the sort has been proposed. You may think this is obvious, but it is far from it.

That's what the "AI pause" proposed. But the main thing is: they could shut down their current technology themselves, so what they are arguing for must be regulation of future technology. I think this has been pretty clear in the congressional hearings for instance.

Right. Altman didn't sign the AI pause though.

It is clear in the congressional hearings, but people didn't watch them, they seem to have skimmed article titles and made up their own narrative.

EDIT:

Which, to my point, means that "these companies" are not calling for "competitive upstarts" to be regulated. They are calling for future very large models, which they themselves are currently the most likely to train due to the enormous computational cost, to be regulated. Which is completely contradictory to what you were saying.

Re: Statement on AI Risk

#754

Post by PostOnce »

Alternative title: "Sam Altman & Friends want More Money".

They want all the opportunity for themselves, and none for us. Control. Control of business, and control of your ability to engage in it.

Another AI company that wants money is Anthropic.

Other Anthropic backers include James McClave, Facebook and Asana co-founder Dustin Moskovitz, former Google CEO Eric Schmidt and founding Skype engineer Jaan Tallinn.

The signatories on this list are Anthropic investors.

First Altman robs us all of a charity that was supposed to benefit us, and now he wants to rob us all of opportunity as well? It's wrong and should be fought against.

Re: Statement on AI Risk

#755

Post by haldujai »

chefandy wrote: post #744
haldujai wrote: post #742

Earlier quoted context omitted.

So the most “notable” AI scientists on this list have clear corporate conflicts. Some are more subtle: > Geoffrey Hinton: Emeritus Professor of Computer Science, University of Toronto, He’s affiliated with Vector (as well as some of the other Canadians on this list) and was at Google until very recently (unsure if he retained equity which would require disclosure in academia). Hence my interest in disclosures as the…

Ok, that's a person! How is saying that they should have disclosed a conflict that they did not disclose not accusatory? If that's the case, the accusation is entirely justified and should be surfaced! The other signatories would certainly want to know if they were signing in good faith when others weren't. This is what I need interns for.

I think you’re misunderstanding my point.

I never said “they should have disclosed a conflict they did not disclose.”

Disclosures are absent from this initiative, some signatories have self-identified their affiliation by their own volition and even for those it is not in the context of a conflict disclosure.

There is no “signatories have no relevant disclosures” statement for those who did not for the omission to be malfeasance and pointing out the absence of a disclosure statement is not accusatory of the individuals, rather that the initiative is not transparent about potential conflicts.

Once again, it is standard practice in academia to make a disclosure statement if lecturing or publishing. While it is not mandatory for initiatives calling for regulation it would be nice to have.

Re: Statement on AI Risk

#756

Post by ben_w »

haldujai wrote: post #684
ben_w wrote: post #670

Earlier quoted context omitted.

> > We’re talking about nuclear war level risks here. > Are we? This seems a bit dramatic for LLMs. The signed statement isn't about just LLMs in much the same way that "animal" doesn't just mean "homo sapiens"

I used LLM because the people shouting the loudest come from a LLM company which claimed their newest language model can be used to create bioweapons in their whitepaper. Semantics aside the recent interest in AI risk was clearly stimulated by LLMs and the camp that believes this is the path to AGI which may or may not be true depending who you ask.

I can only imagine Eleizer Yudkowsky and Rob Miles looking on this conversation with a depressed scream and a facepalm respectively.

They've both been loudly concerned about optimisers doing over-optimisation, and society having a Nash equilibrium where everyone's using them as hard as possible regardless of errors, since before it was cool.

Re: Statement on AI Risk

#757

Post by hayst4ck »

yadaeno wrote: post #674
hayst4ck wrote: post #653

Earlier quoted context omitted.

Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.

Ive never seen Star Trek, but lets say you had an infinite food machine. The machine would have limited throughput, and it would require resources to distribute the food. These are both problems that capitalism solves in a fair and efficient way. I really don’t see how the “capitalism bad” is a satisfying conclusion to draw. The fact that we would use capitalism to distribute the resources is not an indictment of our…

If you are any kind of nerd I recommend watching it. It shows an optimistic view of the future. In many ways it's the anti-cyberpunk. Steve Jobs famously said "give me star trek" when telling his engineers what he wanted from iPhones. Star Trek has had a deep influence on many engineers and on science fiction.

When people talk about Star Trek, they are referring mainly to "Star Trek: The Next Generation."

"The Inner Light" is a highly regarded episode. "The Measure of a Man" is a high quality philosophical episode.

Given you haven't seen it, your criticism of McFarlane doesn't make any sense. You are trying to impart a practical analysis of a philosophical question and in the context of Star Trek, I think it denies what Star Trek asks you to imagine.

Re: Statement on AI Risk

#758

Post by pydry »

nico wrote: post #644
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

Maybe these people have good intentions and are just being naive They might not be getting paid, but that doesn’t mean they are not being influenced AI at this point is pretty much completely open, all the papers, math and science behind it are public Soon, people will have advanced AI running locally on their phones and watches So unless they scrub the Internet, start censoring this stuff, and pretty much ban comput…

>Maybe these people have good intentions and are just being naive

Ive noticed a lot of good people take awful political positions this way.

Usually they trust the wrong person - e.g. by falling victim to the just world fallacy ("X is a big deal in our world and X wouldn't be where they are if they werent a decent person. X must have a point.")

Re: Statement on AI Risk

#759

Post by haldujai »

ben_w wrote: post #756
haldujai wrote: post #684

Earlier quoted context omitted.

I used LLM because the people shouting the loudest come from a LLM company which claimed their newest language model can be used to create bioweapons in their whitepaper. Semantics aside the recent interest in AI risk was clearly stimulated by LLMs and the camp that believes this is the path to AGI which may or may not be true depending who you ask.

I can only imagine Eleizer Yudkowsky and Rob Miles looking on this conversation with a depressed scream and a facepalm respectively. They've both been loudly concerned about optimisers doing over-optimisation, and society having a Nash equilibrium where everyone's using them as hard as possible regardless of errors, since before it was cool.

While true the ones doing media tours and speaking the most vocally in May 2023 are the LLM crowd.

I don’t think it’s a mischaracterization to say OpenAI has sparked public debate on this topic.

Re: Statement on AI Risk

#760

Post by cwkoss »

patch_cable wrote: post #721
patrec wrote: post #682

Earlier quoted context omitted.

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

I think between point 3 and 4 there is a leap to talking about “danger”. Perhaps the disagreement is about what one calls “danger”. I had perhaps mistakenly assumed we were talking about an extinction risk. I’ll grant you concerns about scaling up things like surveillance but there is a leap to being an existential risk that I’m still not following.

AI will not have the instinctual drives for domination or hunger that humans do.

It seems likely that the majority of AI projects will be reasonably well aligned by default, so I think 1000 AIs monitoring what the others are doing is a lot safer than a single global consortium megaproject that humans can likely only inadequately control.

The only reasonable defense against rogue AI is prosocial AI.

Post reply on HN