Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

341–350 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#341

Post by TeMPOraL »

MVissers wrote: post #233
drdeca wrote: post #121

Earlier quoted context omitted.

It seems unclear to me what it is that LLMs can’t/don’t do. I agree that there are substantial limitations of current ones, but I have little idea of which of these limitations are fundamental to the architecture (as well as which are a result of limits of the scale, and which are a result of limits to the training data (using the current training strategies), and which are a result of the overall training methods).…

Not OP, and not a computer scientist. But amateur neuroscientist. From what I understand, current models seem to have issue with planning and reasoning because they “blurt” everything out zero shot. They can’t recurrently process information. Part of it is how we train these models, which is rapidly being experimented with and being improved. Another part is that our brain can both consciously and unconsciously proce…

> From what I understand, current models seem to have issue with planning and reasoning because they “blurt” everything out zero shot. They can’t recurrently process information.

IMO, this is not necessarily a model issue, but an interface issue - recurrently processing information is enabled by just feeding the model's output back to it, perhaps many times, before surfacing it to the user. Right now, with the interfaces we're exposed to - direct evaluation, perhaps wrapped in a chatbot UI - the user must supply the recursion themselves.

> But it’s not innate yet, where models can “think” about a complex problem until it solved it.

That's indeed the part I see us struggling with - we can make the model recursive, but we don't know how to do it unsupervised, so we can box it behind an abstraction layer, letting it work stuff out on its own and break recursion at the right moment.

I think the mental model I developed a good year ago still stands: LLM isn't best compared to a human mind, but to a human's inner voice. The bit that "blurts everything out zero-shot" into your consciousness. It has the same issues of overconfidence, hallucinations, and needing to be recursively fed back to itself.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#343

Post by shinycode »

wk_end wrote: post #334
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

For the #1, when I saw the demo of GPT-4o talk to an other AI in a somewhat convincing scenario, I felt like it was the first time I could saw AI as a real threat to thousands of customer service companies. A majority of those jobs is scripted and the employee doesn’t have a deep enough knowledge to be helpful in any other way. I worked at places where the customer service support wrote all those processes and the teams were trained to follow the scripts. With AI having the ability to analyze text and talk with expressions and feelings… they’re a perfect and cheap replacement that can be scaled easily. If companies are not training their employees to gain deeper insights and level up their support I feel like they are in great danger

Re: Sam and Greg's response to OpenAI Safety researcher claims

#344

Post by edanm »

robbie-c wrote: post #328
edanm wrote: post #301

Earlier quoted context omitted.

That can't actually be the way it works. You can't say "sign this or we *take away something you're already legally entitled to". It has to have been something people agreed to beforehand, to at least some extent. In any case, Sama's tweet says he is cancelling this practice and anyone affected by it can contact him to get their equity back.

Here's the link if you want to read the contract terms: https://twitter.com/KelseyTuoc/status/1791539443016536265 They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring. Here's…

That tweet thread confirms what I said, at least in a roundabout way. Per the next tweets:

> Equity is part of negotiated compensation; this is shares (worth a lot of $$) that the employees already earned over their tenure at OpenAI.

> Employees are not informed of this when they're offered compensation packages that are heavy on equity.

That makes it sound like it's something that they agreed to when accepting their compensation, perhaps unknowingly.

And Kelsey's article links to this LessWrong post about one of the employees that refused this deal and gave up massive amounts of equity, and he writes this:

> To clarify: I did sign something when I joined the company, so I'm still not completely free to speak (still under confidentiality obligations). But I didn't take on any additional obligations when I left.

So again, there are some things that they signed on joining OpenAI, perhaps non-standard things. Now they're asked to sign something more.

I'm not sure of the structure of what they signed before and this article doesn't get into it, but I maintain that there is no legal mechanism to tell someone "sign this or I take away something you already received". There has to be more to the story than the way I've seen it presented so far.

And for the record, I'm very much in the worried-about-AI-safety camp, I have a lot of issues with OpenAI in that and other regards, and I am absolutely willing to believe that the "more to the story" that I gesture at above is still OpenAI being really bad here - it might be that the conditions negotiated with employees made this "we take away your equity" a loophole they weren't aware of, or that OpenAI has some mechanism to make equity effectively zero out somehow, or something.

I'm not saying OpenAI is in the right, at all. I'm saying that the story as told so far doesn't fit with how reality works, and I'm interested to know more details. That's all.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#345

Post by protomolecule »

m3kw9 wrote: post #100
troupo wrote: post #10

> called for international governance of AGI before such calls were popular And immediately threatened to pull out of EU if such governance were ever put in place > we have been putting in place the foundations needed for safe deployment of increasingly capable systems. Figuring out how to make a new technology safe for the first time isn't easy. Especially when "OpenAI" is anything but open. > This includes thoughtf…

Translation: we put in these show “safe guards” to satisfy the law and people peoples nightmare after watching [your AI extinction movie here], but we already had enough safe guards to begin with. They also do not wish to die if you think they have no incentive to keep everyone safe.

> They also do not wish to die if you think they have no incentive to keep everyone safe.

I wouldn't be so sure even about that.[0] That's Page but I wouldn't bet that Altman is any more sane.

"Mr. Page, hampered for more than a decade by an unusual ailment in his vocal cords, described his vision of a digital utopia in a whisper. Humans would eventually merge with artificially intelligent machines, he said. One day there would be many kinds of intelligence competing for resources, and the best would win.

If that happens, Mr. Musk said, we’re doomed. The machines will destroy humanity.

With a rasp of frustration, Mr. Page insisted his utopia should be pursued. Finally he called Mr. Musk a “specieist,” a person who favors humans over the digital life-forms of the future."

[0] https://www.nytimes.com/2023/12/03/technology/ai-openai-musk...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#346

Post by edanm »

d1sxeyes wrote: post #336
edanm wrote: post #299

Earlier quoted context omitted.

> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. That's not true, because you don't address the degree of intelligence or the scale that computers can achieve. Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.

One point to recall on this as well is that if we do develop an intelligent AI, it may have taken us millions of years, but we did it. An intelligent AI would potentially be able to iterate and self develop much faster than we have been able to. It probably won’t take long to go from ‘human equivalent’ to being beyond that.

That's true, but I long ago got convinced that this particular worry isn't the most important one (the recursive-self-improvement one).

I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#347

Post by edanm »

RandomLensman wrote: post #307
edanm wrote: post #298

Earlier quoted context omitted.

> Or, rather, its been actively marketed by Altman as doing so, as part of his effort to buy influence with government to restrict competition. I mean, maybe sama is also doing so. But you're ignoring the many people who have been warning of these dangers for years, some before OpenAI even existed, and many with no part of OpenAI and no monetary incentive. To describe everything as "Altman is marketing this as danger…

The world is full of dangers. AI danger has gotten some attention now - whether it turns out to be warranted beyond certain levels is still unclear. While there has been some regulatory action it also feels there is some "moving on" from the more apocalyptic risk views towards more mundane risks and their management (i.e., more vanilla product and technology risk management).

> While there has been some regulatory action it also feels there is some "moving on" from the more apocalyptic risk views towards more mundane risks and their management

I don't think this is correct. It's more accurate to say that multiple different camps with different fears have risen up over the last few years, and ones worried about more "mundane" AI risks have gotten their view heard more.

I think both groups - "mundane" AI-safety and AI-existential-risk worriers - have both gotten more audience for their views as AI has proven more capable.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#348

Post by shinycode »

verisimi wrote: post #340
wk_end wrote: post #334

Earlier quoted context omitted.

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.

Bullshit jobs yes but still they are paid. What happens in a capitalist society when thousands of people loose their job and can’t produce value and contribute to society ? It’s tearing apart the fabric of capitalism is some way. What happen when all those people without jobs won’t be able to train for better jobs (because they would have done it already and won’t have a bullshit job in the first place) and either fall poor or move to less paid jobs ?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#349

Post by Dalewyn »

BuyMyBitcoins wrote: post #327
Dalewyn wrote: post #320

Earlier quoted context omitted.

This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options r…

>”I don't see nuclear weapons as any worse than conventional weapons because ultimately people die.” If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.

I think of threats that are unknown and/or can't be adequately defended against:

* Asteroid impacts like Chicxulub.

* The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse.

* Pandemics such as The Black Death and, indeed, covid for a recent example.

* Social, political, or commercial intrigue.

* Cancer.

"AI" is a known threat that can be adequately defended against, so I agree with the sentiment that it's stupid to call it a "gravest of dangers".

Re: Sam and Greg's response to OpenAI Safety researcher claims

#350

Post by hnlmorg »

verisimi wrote: post #340
wk_end wrote: post #334

Earlier quoted context omitted.

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.

The “computers are replacing our jobs” line is very emotive and I feel both sides of the argument are looking at the problem too binary.

Yes some jobs are menial but they are still some people’s literal livelihoods. And yes, industries need to evolve and job preservation shouldn’t be the #1 criteria for holding back change.

The real question we should be asking ourselves isn’t either of those two former points but actually this:

Is the pace of change too quick, or is there any support to allow those economically affected by change to be able to retrain?

Removing jobs doesn’t help society because it just leads to unemployment and thus more people dependent on the state. But retraining people is absolutely a benefit. For example, if call centres replaced the first line support with AI, that could lead to shorter call waiting times. But the existing call centre staff could become 2nd line support, account managers for business accounts, install engineers, or even AI supervisors that review a subset of AI interactions to ensure customer satisfaction is maintained.

Post reply on HN