Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta disbanded its Responsible AI team

theverge.com

Post reply on HN
Print view View original

371–380 of 409 posts

Re: Meta disbanded its Responsible AI team

#371

Post by staticman2 »

0xDEAFBEAD wrote: post #362
riversflow wrote: post #349

Earlier quoted context omitted.

> If he's a crank, it should be easy to explain specifically why he's wrong. Sure. The premise that a super intelligent AI can create runaway intelligence on its own is completely insane. How can it iterate? How does it test? Humans run off consensus. We make predictions and test them against physical reality, then have others test them. Information has to be gathered and verified, it's the only rational way to build…

It sounds like you disagree with Eliezer about how AI technology is likely to develop. That's fine, but that doesn't show that he's a crank. I was hoping for something like a really basic factual error. People throughout history have made bold predictions. Sometimes they come true, sometimes they don't. Usually we forget how bold the prediction was at the time -- due to hindsight bias it doesn't seem so bold anymore.…

There used to be a subreddit called Sneerclub where people would make fun of Eliezer and some of his buddies. Here's a discussion of a basic factual error he made on how training AI works, even though this topic is supposedly his life's work:

https://www.reddit.com/r/SneerClub/comments/131rfg0/ey_gets_...

I enjoyed the comment that his understanding of how AI training works is like "thinking that you need to be extremely careful when solving the equations for designing a nuclear bomb, because if you solve them too quickly then they'll literally explode."

Re: Meta disbanded its Responsible AI team

#372

Post by ben_w »

tticvs wrote: post #359
ben_w wrote: post #348

Earlier quoted context omitted.

I don't think I could've been more explicit, as I just said that secrecy is simultaneously necessary for, and an existential threat to, the world we currently have.

Hence training an LLM to keep secrets (your apparent practical definition of safety) is as much of an existential risk as not doing it at all? Your position makes no sense, and your arguments are all based on basically false data.

That you’re phrasing it as “training an LLM to keep secrets” suggests a misapprehension: A downloadable LLM that knows secrets fundamentally cannot be expected to keep them. The fixable condition is much weaker: capabilities, if an LLM has the capability to do dangerous things.

The problem for secrets in general (which is a separate issue to LLMs, I interpreted you asking me about the EU Chat debate as an attempted gotcha not as a directly connected item) that no matter what you do, we’re unstable: not having secrets breaks all crypto which breaks all finance and approximately all of the internet, while having it creates a safe (cyber)space for conspiracies to develop without detection until too late. And also no room for conspiracies means no room to overthrow dictatorships, so if you get one you’re stuck. But surveillance can always beat cryptography so even having the benefits of crypto is an unstable state.

See also: Gordian Knot.

Find someone called 𐀀𐀩𐀏𐀭𐀅𐀨 to solve the paradox, I hear they’re great.

Re: Meta disbanded its Responsible AI team

#373

Post by nradov »

0xDEAFBEAD wrote: post #324
MaKey wrote: post #317

Earlier quoted context omitted.

> But the downside risk from e.g. bioweapons is spread across all of society. It gets old hearing about these "risks" in the context of AI. It's just an excuse used by companies to keep as much power as possible to themselves. The real risk is AI being applied in decision making where it affects humans.

Concern with AI risk actually originated decades ago, way before current AI hype. Here are some of Eliezer Yudkowsky's papers from 2006/2008: https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p... https://intelligence.org/files/AIPosNegFactor.pdf I am concerned with AI companies keeping all the power to themselves. The recent announcement from the OpenAI board was encouraging on that front, because it makes…

But Eliezer Yudkowsky just makes shit up. There's no valid scientific reason to take his concerns seriously.

Re: Meta disbanded its Responsible AI team

#374

Post by nradov »

0xDEAFBEAD wrote: post #345
spooky_action wrote: post #334

Earlier quoted context omitted.

Right there with you. A very influential crackpot who's read a bit too much sci-fi.

I see how some of his tweets could come across as crank-ish if you don't have a background in AI alignment. AI alignment is sort of like computer security in the sense that you're trying to guard against the unknown. If there was a way to push a button which told you the biggest security flaw in the software you're writing, then the task of writing secure software would be far easier. But instead we have to assume th…

Having a "background in AI alignment" is like having a background in defense against alien invasions. It's just mental masturbation about hypotheticals, a complete waste of time.

Re: Meta disbanded its Responsible AI team

#375

Post by 0xDEAFBEAD »

riversflow wrote: post #370
0xDEAFBEAD wrote: post #345

Earlier quoted context omitted.

I see how some of his tweets could come across as crank-ish if you don't have a background in AI alignment. AI alignment is sort of like computer security in the sense that you're trying to guard against the unknown. If there was a way to push a button which told you the biggest security flaw in the software you're writing, then the task of writing secure software would be far easier. But instead we have to assume th…

AI alignment is sort of like computer security except computer security is a real pressing problem and AI alignment is based on science fiction.

Suppose you were working on an early mission-critical computer system. Your coworker is thinking about a potential security issue. You say: "Yeah I read about that in a science fiction story. It's not something we need to worry about." Would that be a valid argument for you to make?

It seems to me that you should engage with the substance of your coworker's argument. Reading about something in science fiction doesn't prevent it from happening.

Re: Meta disbanded its Responsible AI team

#376

Post by nradov »

0xDEAFBEAD wrote: post #344
riversflow wrote: post #329

Earlier quoted context omitted.

Am I the only person who has read a bunch of Elizer writings and decided he's a crank?

If he's a crank, it should be easy to explain specifically why he's wrong. I don't agree with Eliezer on everything, and I often find him obnoxious personally, but being obnoxious isn't the same as being wrong. In general I think it's worth listening to people you disagree with and picking out the good parts of what they have to say. In any case, the broader point is that there are a lot of people concerned with AI r…

That's not how this works. There is no need to explain why he's wrong. Claims made by cranks without evidence can be dismissed without evidence.

Re: Meta disbanded its Responsible AI team

#377

Post by nradov »

DebtDeflation wrote: post #350
MaKey wrote: post #317

Earlier quoted context omitted.

> But the downside risk from e.g. bioweapons is spread across all of society. It gets old hearing about these "risks" in the context of AI. It's just an excuse used by companies to keep as much power as possible to themselves. The real risk is AI being applied in decision making where it affects humans.

I never understood it personally. "What if ChatGPT told someone how to build a bomb?" That information has been out there forever. Anyone can Google it. It's trivial. AI not required. "What if ChatGPT told someone how to build a nuke?" That information is only known to a handful of people in a handful of countries and is closely guarded. It's not in the text ChatGPT was trained on. An LLM is not going to just figure…

The information necessary to build a nuclear weapon has been largely available in open sources since the 1960s. It's really not a big secret. The Nth Country Experiment in 1964 showed that a few inexperienced physicists could come up with a working weapons design. The hard part is doing uranium enrichment at scale without getting caught.

https://en.wikipedia.org/wiki/Nth_Country_Experiment

Re: Meta disbanded its Responsible AI team

#378

Post by 0xDEAFBEAD »

staticman2 wrote: post #371
0xDEAFBEAD wrote: post #362

Earlier quoted context omitted.

It sounds like you disagree with Eliezer about how AI technology is likely to develop. That's fine, but that doesn't show that he's a crank. I was hoping for something like a really basic factual error. People throughout history have made bold predictions. Sometimes they come true, sometimes they don't. Usually we forget how bold the prediction was at the time -- due to hindsight bias it doesn't seem so bold anymore.…

There used to be a subreddit called Sneerclub where people would make fun of Eliezer and some of his buddies. Here's a discussion of a basic factual error he made on how training AI works, even though this topic is supposedly his life's work: https://www.reddit.com/r/SneerClub/comments/131rfg0/ey_gets_... I enjoyed the comment that his understanding of how AI training works is like "thinking that you need to be extre…

Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.

Re: Meta disbanded its Responsible AI team

#379

Post by 0xDEAFBEAD »

riversflow wrote: post #369
0xDEAFBEAD wrote: post #362

Earlier quoted context omitted.

It sounds like you disagree with Eliezer about how AI technology is likely to develop. That's fine, but that doesn't show that he's a crank. I was hoping for something like a really basic factual error. People throughout history have made bold predictions. Sometimes they come true, sometimes they don't. Usually we forget how bold the prediction was at the time -- due to hindsight bias it doesn't seem so bold anymore.…

He is engaging in magical thinking. I showed a factual error, that AI has neither information gathering and verifying capability or a network of peers to substantiate their hypothesis, and you refuse to engage it.

Opinions about what's necessary for AGI are a dime a dozen. You shared your opinion as though it was fact, and you claim that it's incompatible with Eliezer's opinion. I don't find your opinion particularly clear or compelling. But even if your forecast about what's needed for AGI is essentially accurate, I don't think it has much to do with Eliezer's claims. It can simultaneously be the case that AGI will make use of information gathering, verifying capability, and something like a "network of peers", AND that Eliezer's core claims are also correct. Even if we take your opinion as fact, I don't see how it represents a disagreement with Eliezer, except maybe in an incredibly vague "intelligence is hard, bro" sort of way.

Re: Meta disbanded its Responsible AI team

#380

Post by 0xDEAFBEAD »

nradov wrote: post #376
0xDEAFBEAD wrote: post #344

Earlier quoted context omitted.

If he's a crank, it should be easy to explain specifically why he's wrong. I don't agree with Eliezer on everything, and I often find him obnoxious personally, but being obnoxious isn't the same as being wrong. In general I think it's worth listening to people you disagree with and picking out the good parts of what they have to say. In any case, the broader point is that there are a lot of people concerned with AI r…

That's not how this works. There is no need to explain why he's wrong. Claims made by cranks without evidence can be dismissed without evidence.

Evidence isn't the only valid form of argument. Consider a mathematical proof or a thought experiment, for example.
Post reply on HN