Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

121–130 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#121

Post by drdeca »

Aloisius wrote: post #48
MVissers wrote: post #45

Earlier quoted context omitted.

If you look at compute scaling and model improvement on that compute, we’re going to get there pretty fast. Both compute, architecture and cost matter, and those have been improving like crazy. I don’t know about you, but I didn’t expect GPT-4o to come out at half the price one year later, with real-time voice, image and text. There is zero sign of slowing, Nvidia keeps building beefier GPUs specialized in LLMs, the…

LLMs don't reason. Without reasoning, you don't have AGI. I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.

It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods).

Do you have some insights you could share on this question?

I mean, “they can’t reason” might be true, but I’m not exactly sure what it is that it is saying that they can’t do.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#122

Post by photonthug »

corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

This is pretty much true for any touchy subject where the mission (say infosec) isn’t actually aligned with org motives (say cutting costs at the same time as increasing profits).

The result is basically theater where doublethink and doublespeak becomes the norm, and to stick around you have to care more about the theater and the least onerous interpretations of the letter of any applicable law more than the spirit it was intended.

See also [1] for a really pragmatic way of thinking about the realities of a sustainable safety culture

[1] https://github.com/lorin/resilience-engineering/raw/master/b...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#123

Post by francasso »

aaomidi wrote: post #104
_heimdall wrote: post #93

Paraphrasing a bit, and pulling in other recent OpenAI mews. "We know AGI is risky, and we've stated that while also shipping new products every 6 months. We hope you are excited by the recent announcement that OpenAI is looking into how to allow our customers to responsibly generate erotica, and want you to know that we are doing this with full concern of the risks and potential outcomes. We look forward to offering…

Tf is the risk of generating erotica?

They are going to say the usual things, child pornography, risk of terrorist use etc... and conclude that we need our overlords to protect us. Same reasons that have been used in the past to take away freedoms.

Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disregard the upsides of technology that is not heavily regulated.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#124

Post by 0xDEAFBEAD »

zer00eyz wrote: post #87
consumer451 wrote: post #76

Earlier quoted context omitted.

There seem to be only three options here: 1) AGI is not around the corner, so no worries 2) He no longer cares about the possible negative effects, a departure from his past statements 3) I missed something, please help me learn

> 1) AGI is not around the corner, so no worries I agree with this. I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. > He no longer cares ... I missed something Sam was gone, none of these people spoke up, and now they are leaving. I think the story is plain as day but we're pretending it isnt. Sam isnt a leade…

>I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984.

It's interesting that's how it looks like to you from the outside.

If you look a little closer at the "AI Safety" community, it consists of two competing factions: the faction responsible for the Bard debacle, and the faction that's focused on trying to prevent human extinction -- "AI notkilleveryoneism", as in https://x.com/aisafetymemes

They both insist that the other faction is "distracting everyone from the real issues".

Re: Sam and Greg's response to OpenAI Safety researcher claims

#125

Post by next_xibalba »

What harms, in the real world, have GPTs produced? If few or none, does that not indicate that OpenAI has done a good job at safety? If so, should that not buy the company some credibility looking forward?

It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI. Seems to be mostly the doomer crowd and it’s Blake Lemoine types (boy do his claims look ridiculous only 2 years on). To my thinking the open source crew poses a greater danger.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#126

Post by gwern »

plaidfuji wrote: post #63
sashank_1509 wrote: post #54

Earlier quoted context omitted.

AGI seems like a stretch when none of the LLM’s right now can solve the prompt: “ Write a sentence that has an odd number of words, with the third word being "keyboard", the last word being "anyway", the next-to-last word being a number, and the first word being a palindrome “ Do we have to add these examples to the dataset now, so that they can solve this? Then maybe we find another prompt and add that too, as a hum…

Interesting, gpt-4o took seven tries to do this, and I had to give it very specific feedback about what it was doing wrong. Here was the answer: Level says keyboard broke down today, needing repair within five anyway.

GPT-4o still uses BPEs.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#127

Post by tbrownaw »

aaomidi wrote: post #104
_heimdall wrote: post #93

Paraphrasing a bit, and pulling in other recent OpenAI mews. "We know AGI is risky, and we've stated that while also shipping new products every 6 months. We hope you are excited by the recent announcement that OpenAI is looking into how to allow our customers to responsibly generate erotica, and want you to know that we are doing this with full concern of the risks and potential outcomes. We look forward to offering…

Tf is the risk of generating erotica?

Negative reputational or legal consequences directed at the company providing the AI.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#128

Post by Terretta »

tummler wrote: post #85
corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

Text continuation poses "the greatest of dangers"?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#129

Post by _heimdall »

francasso wrote: post #123
aaomidi wrote: post #104

Earlier quoted context omitted.

Tf is the risk of generating erotica?

They are going to say the usual things, child pornography, risk of terrorist use etc... and conclude that we need our overlords to protect us. Same reasons that have been used in the past to take away freedoms. Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disre…

Oh that's so interesting, I fall very much on the side of avoiding regulations as the solution at all costs.

Regulations won't fix a damn thing and regulators have no chance of keeping up even if they wanted to. We need these tools to not be developed or offered at all by choice. Unfortunately that really means either the company, investors, or customers need to make a moral decision at the cost of economic gain or convenience/novelty.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#130

Post by pdonis »

next_xibalba wrote: post #125

What harms, in the real world, have GPTs produced? If few or none, does that not indicate that OpenAI has done a good job at safety? If so, should that not buy the company some credibility looking forward? It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI. Seems to be mostly the doomer crowd and it’s Blake Lemoine types (boy do his claims look ridiculous only 2 years on). To my thinking the…

> If few or none, does that not indicate that OpenAI has done a good job at safety?

No. It only indicates that OpenAI is less technically capable than it claims. To judge OpenAI's competence at safety, you have to look at the ethics of their behavior, since building the right ethics into an AI before it gets powerful enough to do serious harm to humans is the essence of AI safety. And the ethics of OpenAI's behavior, to put it as gently as possible, does not look good.

Post reply on HN