Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

231–240 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#231

Post by menacingly »

ren_engineer wrote: post #61
behnamoh wrote: post #8

Any one noticed how Sam sometimes intentionally tries to attack his own work? For example, he has said GPT-2 was very bad, GPT-3 was bad, and GPT-4 is okay. He also fed the AI doomers by talking about AGI dangers and how the world needs to watch out for a future AI catastrophe. He just gives off the vibe of someone trying to act humble but I don't know how much of it is just an act and how much is real. I've never se…

it's an attempt to keep the hype train going by implying they have internal models that make GPT-4 look bad. The question is if that's actually true, we'll find out soon because plenty of other companies are now approaching GPT-4 level performance and OpenAI will have to show their hand

Completely agree. I am shocked that this isn't apparent to everyone.

I am frustrated at the pure-hype tone this week took on. There are otherwise intelligent people with strong beliefs that OpenAI "has AGI". It's _ludicrous_ and embarrassing. There are people generating entire philosophies predicated on OpenAI just quietly sitting on a profound technological leap.

Even their missteps are explained as actually good. We've seen some regressions on gpt-4o performance. But that's OK, because it isn't supposed to be smarter, because of its very aggressive claims about future features, but they can distract with fearhype, never release or nerf them, and cite unverifiable safety concerns.

Google released some impressive, useful things this week. I really want to believe that oai's striking claims of simultaneous generation-leaping features, reduced cost, and wide scale will come to pass. But if that tech demo was the equivalent of a concept car that will never really see production, I'm looking for a new AI hero.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#232

Post by wrsh07 »

voiceblue wrote: post #80
wrsh07 wrote: post #55

Earlier quoted context omitted.

Greg has some agency in this He also stuck by Altman in November. I'm surprised by how much credit and how much hate goes to the CEO when there are clearly other people making their own decisions, too. Say whatever you want, they've done something impressive. That credit goes to more than just Sam. Ilya & Jan aren't the only ones making decisions. Also, if you've ever been at a startup growing at the speed OpenAI is…

> The reason why everybody thinks OpenAI looks particularly bad is because everybody is watching them A reason, sure. The reason? Come on. There's a clue in their name about what part of their history is particularly controversial.

Sorry, in the context of this tweet and my message, I was referring to "senior leadership leaving"

This sort of high level turnover isn't that uncommon. OpenAI is growing quickly and going from initial concept to hundreds of millions in revenue - this is a time of transition for a lot of startups.

You can absolutely criticize them for hypocrisy or whatever other issue, but I don't think most people should read too much into Ilya/Jan leaving.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#233

Post by MVissers »

drdeca wrote: post #121
Aloisius wrote: post #48

Earlier quoted context omitted.

LLMs don't reason. Without reasoning, you don't have AGI. I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.

It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods).…

Not OP, and not a computer scientist. But amateur neuroscientist.

From what I understand, current models seem to have issue with planning and reasoning because they “blurt” everything out zero shot. They can’t recurrently process information.

Part of it is how we train these models, which is rapidly being experimented with and being improved.

Another part is that our brain can both consciously and unconsciously process information recurrently until we solve a problem. Especially helpful when we haven’t solved a problem before.

With LLMs, we can do that rudimentary with recurrent prompting. In a way, it can see the steps it has tried, re-evaluate and come up with a new try.

But it’s not innate yet, where models can “think” about a complex problem until it solved it.

Also- According to some theories around consciousness, this is when consciousness will really emerge (integrated information theory).

I’m pretty sure we’ll be able to solve this recurrent processing issue in the next few years.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#234

Post by pdonis »

next_xibalba wrote: post #163
pdonis wrote: post #155

Earlier quoted context omitted.

Um, closing access to their model code when "Open" is right there in their name and was the original promise they made? Plenty of other posts in this discussion give other examples. That's without even getting into all that happened with Altman leaving and then coming back, which has already been discussed to death in other HN threads.

I can see how the switch to closed source could be annoying to some, but it definitely doesn’t rise to the level of “unethical” IMO. In fact, the explanation of the rationale for the decision seems perfectly reasonable. WRT to Altman’s ousting and return, I still haven’t seen any evidence of compromised ethics on his part. It looked a lot more like clumsy doomers botching a coup. A supermajority of OpenAIs employees,…

> I can see how the switch to closed source could be annoying to some, but it definitely doesn’t rise to the level of “unethical” IMO.

Sure it does. Particularly for a company that claims to be a responsible steward of a technology that could pose existential risk to humans. If you can't even do a simple thing like keep your promises, how can I possibly trust you as a steward of a potential existential risk?

> WRT to Altman’s ousting and return, I still haven’t seen any evidence of compromised ethics on his part.

I commented on this in a number of previous HN threads and don't want to rehash it all here. (And btw, I also commented that I didn't think the people who ousted Altman showed very good ethics or judgment either. I didn't think anybody came out of that whole brouhaha looking good.) But just the fact that it happened at all should be a red flag. Again, these are people who claim to be stewards of a technology that they say can pose an existential risk. You'd expect them to at least be able to cooperate with each other like responsible adults.

> A supermajority of OpenAIs employees, people who I’d venture to guess have a far better handle on Altmans ethics than you or I

They might well know Altman's ethics better than we do, and be perfectly fine with them, because their own ethics are just as bad. They're getting paid well and the fact that core promises of the company were broken is simply Not Their Problem.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#235

Post by optomas »

ta988 wrote: post #167
tummler wrote: post #164

Earlier quoted context omitted.

Hoo boy... if you think chatbots are all this is... wait til you find out what's next.

What's next?

Are you capable of being manipulated through text? How about an AI trained to generate text that will extract exactly the desired response from you? An advertiser's wet dream.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#236

Post by PaulDavisThe1st »

tbrownaw wrote: post #188
ants_everywhere wrote: post #114

Earlier quoted context omitted.

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

> Could AI running amok destroy the human race? Yes. Could a human running amok destroy the human race? If yes, why is an AI doing it worse? If no, what capabilities is this AI assumed to have that a human can never have?

It isn't necessary that "this AI [have capabilities] that a human can never have".

The issue is how those capabilities (possibly shared, possibly not shared between some humans and AI) are deployed and to what ends.

There's also a speed question. Maybe AI will never be able to do things human cannot, but it seems likely that for the sorts of things most AI-excited folk are excited about, it will be faster than a human most of the time.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#237

Post by nathanasmith »

llamaimperative wrote: post #91
bambax wrote: post #49

Earlier quoted context omitted.

Yes! It's debatable current models demonstrate any kind of "intelligence", but AGI is so far off it's ridiculous to even mention it. But not everyone's delusional. Here's what Yann Le Cun is currently saying: > It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat. and > don't confuse…

Okay, so describe to me when and how we’ll detect and mitigate problems as we approach AGI or superintelligence. The promise of upside grows exactly in lockstep with the danger of downside. It is absolutely rational to say: “if our AI safety research lab is being hijacked by commercial interests before it’s even commercially viable as a business, then how will we possibly pull the plug on the final descent?” P.S. nei…

We're about as close to AGI as we are to developing superluminal space flight and working time machines.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#238

Post by nvy »

next_xibalba wrote: post #158
nvy wrote: post #138

Earlier quoted context omitted.

>It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI Perhaps because he's a slimy used-car salesman who got rich doing things that are illegal for you and me (ripping off copyrighted works en masse) and he pulled a bait and switch as far as for-/non-profit is concerned. He's everything wrong with Silicon Valley/Venture Capital/faux meritocratic culture rolled into one.

Altman was rich before OpenAI, so that claim doesn’t hold much water. It is a legitimate and open question as to what fair use constitutes when it comes to training models on copyrighted content. It will definitely be settled in the courts, if not by legislatures in the coming years. Apart from ad hominem attacks and matters about which there are legitimate differences of opinion, I return to my original question: wh…

The fact that you cannot perceive the harms, or that they are not yet apparent, does not mean they do not exist.

Look at Facebook and other social media: at first we all thought it was great and then we realized it's a giant depression machine, and all it's really good for is giving young girls eating disorders.

But since we're on the topic, the immediate casualties seem to be the livelihoods of authors and artists, on whose content the LLMs were trained and who received no compensation.

And now every publisher is inundated with infinite ChatGPT-created books. We won't get into the climate impact of wasting all that compute just to create a better phishing-email generator.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#240

Post by 0xDEAFBEAD »

Dalewyn wrote: post #175
hn_go_brrrrr wrote: post #136

Earlier quoted context omitted.

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dang…

>* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.

Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.

Post reply on HN