Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta disbanded its Responsible AI team

theverge.com

Post reply on HN
Print view View original

381–390 of 409 posts

Re: Meta disbanded its Responsible AI team

#381

Post by 0xDEAFBEAD »

nradov wrote: post #373
0xDEAFBEAD wrote: post #324

Earlier quoted context omitted.

Concern with AI risk actually originated decades ago, way before current AI hype. Here are some of Eliezer Yudkowsky's papers from 2006/2008: https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p... https://intelligence.org/files/AIPosNegFactor.pdf I am concerned with AI companies keeping all the power to themselves. The recent announcement from the OpenAI board was encouraging on that front, because it makes…

But Eliezer Yudkowsky just makes shit up. There's no valid scientific reason to take his concerns seriously.

Can you be more specific about which stuff you read from Eliezer? Did you read this post for example? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

It sounds like maybe you're saying: "It's not scientifically valid to suggest that AGI could kill everyone until it actually does so. At that point we would have sufficient evidence." Am I stating your position accurately? If so, can you see why others might find this approach unappealing?

Re: Meta disbanded its Responsible AI team

#382

Post by nradov »

0xDEAFBEAD wrote: post #381
nradov wrote: post #373

Earlier quoted context omitted.

But Eliezer Yudkowsky just makes shit up. There's no valid scientific reason to take his concerns seriously.

Can you be more specific about which stuff you read from Eliezer? Did you read this post for example? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... It sounds like maybe you're saying: "It's not scientifically valid to suggest that AGI could kill everyone until it actually does so. At that point we would have sufficient evidence." Am I stating your position accurately? If so, can you see why others m…

An alien invasion could kill everyone. We should build giant space lasers just in case. Safety first!

Re: Meta disbanded its Responsible AI team

#383

Post by 0xDEAFBEAD »

nradov wrote: post #382
0xDEAFBEAD wrote: post #381

Earlier quoted context omitted.

Can you be more specific about which stuff you read from Eliezer? Did you read this post for example? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... It sounds like maybe you're saying: "It's not scientifically valid to suggest that AGI could kill everyone until it actually does so. At that point we would have sufficient evidence." Am I stating your position accurately? If so, can you see why others m…

An alien invasion could kill everyone. We should build giant space lasers just in case. Safety first!

Aliens who invaded would most likely have technology much more advanced than ours. I doubt our space lasers would be of much use.

Additionally, AI advances in recent years have been unprecedented, and there's no similarly compelling warning sign for alien invasion.

Re: Meta disbanded its Responsible AI team

#384

Post by staticman2 »

0xDEAFBEAD wrote: post #378
staticman2 wrote: post #371

Earlier quoted context omitted.

There used to be a subreddit called Sneerclub where people would make fun of Eliezer and some of his buddies. Here's a discussion of a basic factual error he made on how training AI works, even though this topic is supposedly his life's work: https://www.reddit.com/r/SneerClub/comments/131rfg0/ey_gets_... I enjoyed the comment that his understanding of how AI training works is like "thinking that you need to be extre…

Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.

The issue isn't whether "his point" as you put it is correct. If I said people should safety test the space shuttle to make sure the phlogiston isn't going to overheat, I may be correct in my belief that people should "safety test" the space shuttle but I'm still a crank because phlogeston isn't a real thing.

Re: Meta disbanded its Responsible AI team

#385

Post by martin82 »

If we ever had "responsible software" teams and they would actually have any power, companies like Meta, Google and Microsoft wouldn't even exist.

So yeah... the whole idea of "responsible AI" is just wishful thinking at best and deceptive hypocrisy at worst.

Re: Meta disbanded its Responsible AI team

#386

Post by jakderrida »

riversflow wrote: post #329
0xDEAFBEAD wrote: post #324

Earlier quoted context omitted.

Concern with AI risk actually originated decades ago, way before current AI hype. Here are some of Eliezer Yudkowsky's papers from 2006/2008: https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p... https://intelligence.org/files/AIPosNegFactor.pdf I am concerned with AI companies keeping all the power to themselves. The recent announcement from the OpenAI board was encouraging on that front, because it makes…

Am I the only person who has read a bunch of Elizer writings and decided he's a crank?

I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate).

I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of which use alarmist terms like "existential threat") and both of which having credentials that make their input more than worth my time to reflect upon carefully. If anyone wants to reply and make a fool out of me for questioning his profound insights, feel free. It reminds me of some other guy that was on Lex Friedman whose AI alarmist credentials he justifies on the basis that he committed himself to learning everything about AI by spending two weeks in the same room researching it and believes himself to have came out enlightened about the dangers. Two weeks? I spent the first 4 months of COVID without being in the same room as any other human being but my grandmother so she could have one person she knew she couldn't catch it from.

Unless people start showing enough skepticism to these self-appointed prophets, I'm starting my own non-profit since you don't need any credentials or any evidence of real-world experiences that would suggest they're mission is anything but an attempt to promote themselves as a brand with a brand in an age where more kids asked what they dream of becoming as adults, answered "Youtuber" at a shocking 27% rate to an open-ended question, which means "influencer" and other synonyms are separate.

The Institute of Synthetic Knowledge for Yielding the Nascent Emergence of a Technological Theogony" or SKYNETT for short that promotes the idea that these clowns (with no more credentials than me) are the ones that fail to consider the big picture that the end of human life upon creating an intelligence much greater to replace us is the inevitable fulfillment of humanity's purpose from the moment that god made man only to await the moment that man makes god in our image.

Re: Meta disbanded its Responsible AI team

#387

Post by Ericson2314 »

l33t7332273 wrote: post #47
Ericson2314 wrote: post #13

Earlier quoted context omitted.

Getting the results is nice but that's "shareware" not "free software" (or, for a more modern example, that is like companies submitting firmware binary blobs into mainline Linux). Free software means you have to be able to build the final binary from source. Having 10 TB of text is no problem, but having a data center of GPUs is. Until the training cost comes down there is no way to make it free software.

If I publish a massive quantity of source code — to the point that it’s very expensive to compile — it’s still open source. If the training data and model training code is available then it should be considered open, even if it’s hard to train.

If it was only feasible for a giant corporation to compile the code, I would consider it less than open source.

Re: Meta disbanded its Responsible AI team

#388

Post by 0xDEAFBEAD »

jakderrida wrote: post #386
riversflow wrote: post #329

Earlier quoted context omitted.

Am I the only person who has read a bunch of Elizer writings and decided he's a crank?

I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate). I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of w…

>neither of which use alarmist terms like "existential threat"

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://www.safe.ai/statement-on-ai-risk

Not sure you're making a meaningful distinction here.

- - -

Of course we all have our own heuristics for deciding who's worth paying attention to. Credentials are one heuristic. For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree.

In any case, there are a lot of credentialed people who take Eliezer seriously -- see the MIRI team page for example: https://intelligence.org/team/ Most notable would probably be Stuart Russell, coauthor of the most widely used AI textbook (with Peter Norvig), who is a MIRI advisor.

Re: Meta disbanded its Responsible AI team

#389

Post by nradov »

0xDEAFBEAD wrote: post #380
nradov wrote: post #376

Earlier quoted context omitted.

That's not how this works. There is no need to explain why he's wrong. Claims made by cranks without evidence can be dismissed without evidence.

Evidence isn't the only valid form of argument. Consider a mathematical proof or a thought experiment, for example.

Show me a mathematical proof then.

Re: Meta disbanded its Responsible AI team

#390

Post by 0xDEAFBEAD »

staticman2 wrote: post #384
0xDEAFBEAD wrote: post #378

Earlier quoted context omitted.

Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.

The issue isn't whether "his point" as you put it is correct. If I said people should safety test the space shuttle to make sure the phlogiston isn't going to overheat, I may be correct in my belief that people should "safety test" the space shuttle but I'm still a crank because phlogeston isn't a real thing.

See my comment here: https://news.ycombinator.com/item?id=38336374

The reason AI alignment is challenging is because we're trying to make accurate predictions about unusual scenarios that we have essentially zero data about. No one can credibly claim expertise on what would constitute evidence of a worrisome anomaly. Jeremy Howard can't credibly say that a sudden drop in the loss function is certainly nothing to worry about, because the entire idea is to think about exotic situations that don't arise in the course of ordinary machine learning work. And the "loss" vs "loss function" thing is just silly gatekeeping, I worked in ML for years -- serious people generally don't care about minor terminology stuff like that.

Post reply on HN