Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

791–800 of 964 posts

Re: Statement on AI Risk

#791

Post by sanderjd »

rockemsockem wrote: post #753
sanderjd wrote: post #223

Earlier quoted context omitted.

That's what the "AI pause" proposed. But the main thing is: they could shut down their current technology themselves, so what they are arguing for must be regulation of future technology. I think this has been pretty clear in the congressional hearings for instance.

Right. Altman didn't sign the AI pause though. It is clear in the congressional hearings, but people didn't watch them, they seem to have skimmed article titles and made up their own narrative. EDIT: Which, to my point, means that "these companies" are not calling for "competitive upstarts" to be regulated. They are calling for future very large models, which they themselves are currently the most likely to train due…

I'll start by saying that I think ours is an honest difference of opinion (or actually even weaker: a difference of speculation) in a very uncertain space. With that out of the way:

When I watched that congressional hearing, what I saw was identical to every other oh-so-concerned executive of an incumbent dominant businesses in any other industry. I truly see no reason to give them any more benefit of the doubt than the CEO of Exxon expressing concern about climate change or of Philip Morris expressing concern about lung cancer. And just like in those cases, it's not that the concern isn't legitimate, it's that they aren't credible people to have leading the charge against it. I suspect that is obvious to you in the other two cases, so I ask you: why do you draw a different conclusion here? Do you believe tech executives are just fundamentally more ethical than the executives in other industries? I simply don't think that's accurate.

I also think this is credulous:

> They are calling for future very large models, which they themselves are currently the most likely to train due to the enormous computational cost, to be regulated.

I've highlighted the part I think is wrong. What is actually happening is that lots of people are finding lots of different ways to undermine that thesis and, having already sunk huge costs in the currently productized models, and also in the next generation, these companies are worried about getting undercut by "good enough" cheaper and mostly open models.

Have you read "we have no moat and neither does openai" that came out of Google? This is what that's talking about.

Trying to regulate those models out of existence before they can undercut them is just rational business, I don't begrudge them doing it, it's their job. But it's up to the rest of us to not just go along to get along.

Re: Statement on AI Risk

#792

Post by zzzeek »

chefandy wrote: post #487
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

> even if they were walking around handing out checks for 25k I doubt they'd have gotten a good chunk of these folks.

if the corporations in question get world governments to line up the way they want, the checks for everyone in these "letters" will be *way* bigger than 25k and they won't have "payment for signing our letter" in the memo either.

Re: Statement on AI Risk

#793

Post by randomdata »

sebzim4500 wrote: post #321
randomdata wrote: post #155

Earlier quoted context omitted.

What would be in it for a more intelligent agent to get rid of us? We are likely useful tools and, at worst, a curious zoo oddity. We have never been content when we have caused extinction. A more intelligent agent will have greater wherewithal to avoid doing the same. 'Able to play chess'-level AI is the greater concern, allowing humans to create more unavoidable tools of war. But we've been doing that for decades,…

>We have never been content when we have caused extinction. err what? Apparently there are 1 million species under threat of human caused extinction [1]. [1] https://www.nbcnews.com/mach/science/1-million-species-under...

I don’t get it. We’re not as intelligent, per the original premise and you’re just echoing that premise.

Re: Statement on AI Risk

#794

Post by cwkoss »

ericb wrote: post #273
SanderNL wrote: post #114

I'm not sure they can keep claiming this without becoming concrete about it. Nuclear weapons are not nebulous, vague threats of diffuse nature. They literally burn the living flesh right off the face of the earth and they do it dramatically. There is very little to argue about except "how" are we going to contain it, not "why". In this case I truly don't know "why". What fundamental risks are there? Dramatic, loud, l…

Agreed that it should be spelled out, but... If a superintelligence can be set on any specific task, it could be any task. - Make covid-ebola - Cause world war 3 You may have noticed that chatgpt is sort of goal-less until a human gives it a goal. Assuming nothing other than it can become superintelligent (no one seems to be arguing against that--I argue that it already is ) which is really an upgrade of capability,…

Humans could decide to do either of those things. Should we regulate the bounds of human thought to prevent this?

Re: Statement on AI Risk

#795

Post by [deleted] »

mahogany wrote: post #13
ggm wrote: post #2

Nothing about this risk or the statement implies AGI is real, because the risk exists in wide scale use of existing technology. It's the risk of belief in algorithmically derived information, and deployment of autonomous, unsupervised systems. It's great they signed the statement. It's important.

> It's the risk of belief in algorithmically derived information, and deployment of autonomous, unsupervised systems. And Sam Altman, head of one of the largest entities posing this exact risk, is one of the signatories. We can't take it too seriously, can we?

[deleted]

Re: Statement on AI Risk

#797

Post by moneywoes »

jiggawatts wrote: post #727
b3nji wrote: post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI. This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders. But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit. My predictio…

> Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI

Sorry I don’t understand what would be the impact? Aren’t the results not deterministic

What are the weights here

Re: Statement on AI Risk

#798

Post by NovaDudely »

cwkoss wrote: post #771

Humans could already be on a path to go extinct in a variety of ways: climate change, wars, pandemics, polluting the environment with chemicals that are both toxic and pervasive, soil depletion, monoculture crop fragility... Everyone talks about the probability of AI leading to human extinction, but what is the probability that AI is able to help us avert human extinction? Why does everyone in these discussions assum…

I think it is because anything that can be used for good can also be used for bad. Advances in gene editing can provide a miracle medicine or a new means for biological warfare.

AI is the same, we can use it to do some great things but it can also be leveraged by bad actors and very easily. The broad scale of what it can be implemented on means there is a lot of attach surface for change in either direction.

The AI systems don't even need to be that advanced to cause real issues simply because of the sheer scale of society as it stands. It can be used in a form of Akido in which it uses the weight of the system to bring itself down.

Failure is an emergent property of complexity.

Re: Statement on AI Risk

#799

Post by cwkoss »

duvenaud wrote: post #151

I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence.

Why do you assume AI will be as greedy as humans?
Post reply on HN