Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

221–230 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#221

Post by fabrice_d »

MVissers wrote: post #191
fabrice_d wrote: post #57

Earlier quoted context omitted.

And still, Yann LeCun (head of AI at Meta, renowned AI/ML researcher) is convinced that we are far from reaching AGI. He makes convincing arguments, especially around the fact that we are not able to expose models to the amount of redundant information that even a young kid is exposed to. I guess we'll see some shifting goals around "what is even AGI". You say the we'll have soon "models that most of us consider AGI…

I mean, this new openai model is trained on video and voice as well. So already, his argument is outdated. He had made a few of these claims over the last year that have been quickly disproven by a new model.

They are trained on a ridiculously small amount of content compared to what the brain of a 3 years child is subject to. Current models still have a very narrow application field compared to what a human can achieve.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#222

Post by [deleted] »

N0b8ez wrote: post #201
tbrownaw wrote: post #185

Earlier quoted context omitted.

It's essentially Pascal's wager.

What probability for extinction do you consider to be low enough that it effectively becomes a Pascalian wager? Pascal was also writing about a single person's fate (AFAICT), whereas this is about the fate of everyone.

[deleted]

Re: Sam and Greg's response to OpenAI Safety researcher claims

#223

Post by Jedd »

Dalewyn wrote: post #175
hn_go_brrrrr wrote: post #136

Earlier quoted context omitted.

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dang…

I don't believe asserting two parallel and comparably false equivalences meaningfully progresses the discussion around this technology and risk.

If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it.

If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#224

Post by tracerbulletx »

Terretta wrote: post #128
tummler wrote: post #85

Earlier quoted context omitted.

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

Text continuation poses "the greatest of dangers"?

Well as you surely must know transformers are general sequence continuation machines and have been proven to work well with almost any type of input and output sequence. You also must be aware that people are trying to plug these general sequence continuation machines in to all kinds of different sequences and giving them all kinds of power to take action. These sequence continuation machines are also getting more complex and multi layered every day and also being wired together in novel ways. This is a dangerous thing to do depending on what actions you allow them to take because of the impact on society of having agents taking actions without oversight.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#225

Post by tbrownaw »

m3kw9 wrote: post #95
nicce wrote: post #81

Earlier quoted context omitted.

We need protection from people who control AI. Who will provide that…

A lot of hand waving from those words. Right now LLM isn’t more dangerous than harmful google results. The future where AI sends a terminator to destroy us is the part I’m having issues with

Perhaps gangs will start sending assassin-bots at eachother, but the more realistic threat for most people is getting banned from society after you post a wrongthink somewhere and get your social credit score nuked.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#226

Post by elicksaur »

drdeca wrote: post #121
Aloisius wrote: post #48

Earlier quoted context omitted.

LLMs don't reason. Without reasoning, you don't have AGI. I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.

It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods).…

Think about the difference between what an LLM is doing and a human is doing in response to being told they are wrong.

If you tell a human, “You are wrong on this, and here is why.” They may reject it, but their mental model is updating with the new information that you, the speaker, think X for YZQ reasons. Their response is based on a judgment of how trustworthy you are and the credibility of the evidence.

For an LLM, the response is not based on these logical connections, but simply the additional prompt context of YZQ tokens being close to each other.

This is not “logic” in any traditional sense or in the sense of how a human incorporates and responds to this new information.

The LLM’s method of responding is also inherent to the architecture of the model. It’s predicting tokens based on input. It’s not reasoning.

Critically, this flaw is inherent in all LLM output. Giving an LLM’s output the power to affect real world activities means trusting that the decision can be made by sophisticated word association rather than more complex reasoning.

There may be lots of decisions where word association is all you need, but I doubt that is the case for all decisions humans make.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#227

Post by brandall10 »

Dalewyn wrote: post #194
comp_throw7 wrote: post #184

Earlier quoted context omitted.

> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."

An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living…

There absolutely is a difference. Nuclear war is considered a likely extinction level event for humanity. A war between the US/NATO and the Soviet Union could take out major cities around the globe within hours, and the resultant nuclear winter would blanket much of the planet within a month making living above ground impossible.

GP's comparison is apt here.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#228

Post by _heimdall »

TaylorAlexander wrote: post #193
_heimdall wrote: post #119

Earlier quoted context omitted.

I didn't actually mean to refer to the DEI-flavored AI ethics. That was totally unintentional on my part, I should have caught that. I raised both because I see them going down very similar paths. DEI conceptually is a really important idea and could lead to really important change or course correction. As implemented today though, it seems to most often be implemented as a much more surface level program that sure f…

> DEI conceptually is a really important idea and could lead to really important change or course correction. As implemented today though, it seems to most often be implemented as a much more surface level program that sure feels more like PR or legal protection than anything else. I suspect that some DEI efforts are helpful and effective, and some DEI efforts are hollow or foolhardy. We probably can’t speak of “DEI…

I do agree that my referring to DEI today may be too broad, that's a great point.

> Instances where it works well and gets more people hired and engaged are less interesting to a predominantly white society, so maybe aren’t discussed as much outside of non-white communities.

This got me curious, have you sewn any examples of DEI programs helping to get more people hired rather than different people hired? Either can be useful, but that distinction would be a big one as the former means DEI is somehow growing the job market rather than refocusing hiring practices.

Nothing wrong with speculation as far as I'm concerned! Reliable and accurate data is hard to come by, I'd argue that most of what is presented as fact is little more than speculation backed by fuzzy data full of assumptions.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#229

Post by _heimdall »

Silphendio wrote: post #182
francasso wrote: post #123

Earlier quoted context omitted.

They are going to say the usual things, child pornography, risk of terrorist use etc... and conclude that we need our overlords to protect us. Same reasons that have been used in the past to take away freedoms. Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disre…

Child porn is already heavily "regulated". Producing the stuff is illegal and AI service providers must take steps to prevent it.

This is a blind spot for me, do existing legal protections generally cover fake/generated content?

To be clear, the context of this thread is extremely important. I don't know how far our existing laws might go to make sure ML generated child porn is illegal and I'd personally feel much more comfortable to know that ML generation isn't some kind of legal loophole.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#230

Post by nopromisessir »

corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

Yeah I stopped reading about halfway through.

For crying out loud hn... Out outputs is ultimately a series of 1 and/or 0....

That doesn't mean we have to carry that through to every analysis and comment.

Things are not so binary y'all. Regular life operates in the quantum or non discrete rules. Pls think on it.

Post reply on HN