Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

391–400 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#391

Post by emrehan »

Some facts:

* This is a response to Jan’s claims: https://x.com/janleike/status/1791498174659715494

* There’s nothing concrete on AI alignment in the response.

* We don’t know when AI alignment could be developed.

* AI ethics / AI bias are necessary, but different concerns than AI alignment.

* We don’t know when existential-risk posing AI could be developed.

* There’s some risk of human extinction due to development of x-risk posing AI before AI alignment.

* OpenAI is at the frontier of AI development, which risks human extinction, without allocating sufficient resources to AI alignment.

I am uneasy with subscribing to ChatGPT…

Re: Sam and Greg's response to OpenAI Safety researcher claims

#392

Post by 0xDEAFBEAD »

blksv wrote: post #326
0xDEAFBEAD wrote: post #240

Earlier quoted context omitted.

>* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.

But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. I agree, that AI…

>But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way.

One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem"

Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#393

Post by mjburgess »

nl wrote: post #356
mjburgess wrote: post #325

Earlier quoted context omitted.

all statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture. eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words. But these words ar…

> The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes. This sounds so logical and authoritative. And yet: me> What event would cause a change in what all people said about Ukraine/Russia pre/post a few hours of 2022 GPT4O> A significant event that caused a drastic change in global discussions about Ukraine and Russia in 202…

Right... because it's been trained on those news stories.

The point is a model whose training stopped in 2021 would not produce a history of ukraine (etc.) that a person writing in 2023 would.

The later GPTs are trained on the user-provided prompts/answers of previous GPTs, so this process (which isnt the LLM, but it's the activity of research staff at OpenAI) is what's inducing approximate tracking of some changes in meaning.

Whilst this works for any changes over-represented in the new training data, (1) the LLM isnt doing that, its the researchers; and (2) this process is vastly expensive and time-intensive; and (3) only tracks changes with a high word frequency in new data.

If you could run the months-long, 1GWh, 10s-million-USD training process each minutes of the day, you would resolve the inability of the model to track major news stores... but would not resolve its ability to track, say, the user changing their clothes.

The sensitivity to the model of stuff in the world arises because of humans preparing the training data to bring about apparent sensitivity. Absent the activity of these humans, the whole thing drifts gradually into irrelvance.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#394

Post by fullspectrumdev »

thayne wrote: post #244
primer42 wrote: post #52

Safe for whom or what? Who are we protecting from AI again? Are they aiming to protect the interests of the many, or the interest of a few?

My reading of this is that what OpenAI wants is safety from competition.

Yep. Monopoly through regulatory capture is the end goal of most of the AI x-risk nerds.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#395

Post by shinycode »

hnlmorg wrote: post #350
verisimi wrote: post #340

Earlier quoted context omitted.

Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.

The “computers are replacing our jobs” line is very emotive and I feel both sides of the argument are looking at the problem too binary. Yes some jobs are menial but they are still some people’s literal livelihoods. And yes, industries need to evolve and job preservation shouldn’t be the #1 criteria for holding back change. The real question we should be asking ourselves isn’t either of those two former points but ac…

I completely agree with your conclusion and that is ideal. Will there be companies that make the choice to invest money into retraining a major part of their workforce ? I really hope so because it’s an elegant solution.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#396

Post by mofeien »

fullspectrumdev wrote: post #389
tummler wrote: post #85

Earlier quoted context omitted.

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

> The technology they're working on poses the gravest of dangers Extraordinary claims require extraordinary evidence.

ChatGPT-4 won't kill us, and probably neither will the next generation of models. But if we keep building more and more powerful AI systems, eventually we will reach a point where one will become an existential threat.

Present dangers of AI include Fake News, impersonation through deepfakes, biases and discrimination in the models, job loss and economic instability, loss of human purpose through individuals becoming redundant, individuals becoming addicted to the AI systems, use by authoritarian governments, autonomous weapons.

Near future dangers include bioweapon engineering capabilities, computer hacks, human disempowerment, existential risk, or a state where the values of the world are locked in by an uncontrolled AI, and possibly filled with suffering if locked in in a bad way, e. g. as we lock in animals.

It might take months until AGI, it might take decades, nobody knows for sure. However, we do know that the pace of AI progress is often grossly underestimated. Just three years ago we thought we’d have SAT-passing AI systems in 2055. We got there in April 2023. So it makes sense to act as if we have very little time.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#397

Post by bjackman »

emrehan wrote: post #391

Some facts: * This is a response to Jan’s claims: https://x.com/janleike/status/1791498174659715494 * There’s nothing concrete on AI alignment in the response. * We don’t know when AI alignment could be developed. * AI ethics / AI bias are necessary, but different concerns than AI alignment. * We don’t know when existential-risk posing AI could be developed. * There’s some risk of human extinction due to development…

Can anyone summarise the claims? Twitter doesn't show threads for web users (at least if not logged in).

Re: Sam and Greg's response to OpenAI Safety researcher claims

#398

Post by fullspectrumdev »

0xDEAFBEAD wrote: post #146
m3kw9 wrote: post #92

He is basically saying there was a mis-balance in the perception of risk between this likely doomer and most tech staff that is there. There is no Terminator to align and LLM going “rogue” and killing everyone is way overblown to pause development

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/work/statement-on-ai-risk#open-letter What do you know that Geoff Hinton, Yoshua Bengio, Demis Hassabis, Bill Gates, Stuart Russell, etc. don't?

Personally I think they are delusional. But that’s fine. Autocomplete isn’t gonna nuke us.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#399

Post by emrehan »

bjackman wrote: post #397
emrehan wrote: post #391

Some facts: * This is a response to Jan’s claims: https://x.com/janleike/status/1791498174659715494 * There’s nothing concrete on AI alignment in the response. * We don’t know when AI alignment could be developed. * AI ethics / AI bias are necessary, but different concerns than AI alignment. * We don’t know when existential-risk posing AI could be developed. * There’s some risk of human extinction due to development…

Can anyone summarise the claims? Twitter doesn't show threads for web users (at least if not logged in).

https://threadreaderapp.com/thread/1791498174659715494.html
Post reply on HN