Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

351–360 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#351

Post by nl »

conception wrote: post #208
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the…

> map tool lets users “nominate targets of interest” for “the target nomination process”. She meant it helps people choose which places get bombed.

So... people are choosing who to kill?

I'm reasonably sure there are historical precedents for this.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#352

Post by nl »

wk_end wrote: post #334
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

Agree with this.

And generally "AI Safety" researchers aren't concerned with these risks and instead are looking at existential risk.

The boring but real risks you outline (to which I'd add reinforcement of unknown biases) generally fall into what people call "AI Ethics" or sometimes "Data Ethics".

Re: Sam and Greg's response to OpenAI Safety researcher claims

#353

Post by wordpad »

shinycode wrote: post #348
verisimi wrote: post #340

Earlier quoted context omitted.

Re 1 - aren't people mostly doing bullshit jobs already? Re 2 - fakery in videos and images has been a part since their inception. So, don't worry, I guess? The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.

Bullshit jobs yes but still they are paid. What happens in a capitalist society when thousands of people loose their job and can’t produce value and contribute to society ? It’s tearing apart the fabric of capitalism is some way. What happen when all those people without jobs won’t be able to train for better jobs (because they would have done it already and won’t have a bullshit job in the first place) and either fa…

Technology and automation tends to create more jobs. As operational costs decrease, your end service or product becomes cheaper, so all your b2b or b2c customers can afford more of it which leads to more opportunities, doing things which were cost prohibitive before.

Some of those things may too require manual labor.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#354

Post by nl »

shinycode wrote: post #343
wk_end wrote: post #334

Earlier quoted context omitted.

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

For the #1, when I saw the demo of GPT-4o talk to an other AI in a somewhat convincing scenario, I felt like it was the first time I could saw AI as a real threat to thousands of customer service companies. A majority of those jobs is scripted and the employee doesn’t have a deep enough knowledge to be helpful in any other way. I worked at places where the customer service support wrote all those processes and the te…

Klana reduced it's workforce by 700 (out of 3000) people after implementing an AI-powered chatbot:

> We made the announcement to say the consequence of us launching the technology is we need the equivalent of 700 fewer full-time agents than what we usually use on an average basis. On average, we need 3,000 agents, now we need a little more than 2,000.

> It's on par with humans in terms of satisfaction and it resulted in a 25% reduction in repeat inquiries from customers

> We've stopped hiring in the last six months. We're shrinking as a company, not by layoffs, but by natural attrition. Klarna tries to apply AI across all products and services and work we do. It's having implications on how many people we need as a company. This is one time that a single product improvement led to a massive reduction in need for customer service agents.

https://www.cbsnews.com/news/klarna-ceo-ai-chatbot-replacing...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#355

Post by mjburgess »

golol wrote: post #339
mjburgess wrote: post #331

Earlier quoted context omitted.

Turing was a WW2 era mathematician. He had no insight or understanding of intelligence, made no study of intelligent systems, and so on (he believed in ESP of all things). Turing's test is a restatement of a now pseudoscientific behaviourism common at the time; and also, egregiously, places a dumb ape as the system which measures intelligence. If an ape can be fooled, the system is intelligent: people worshiped the s…

The Turing test insight is that text is a sufficient medium to test for AGI. And this still holds true.

That has nothing to do with why turing proposed it; nor does it have anything to do with general intelligence. This is just pseudoscience.

There's no scientific account of the capacities of a system with intelligence, no account of how these combine, no account of how communicative practices arise, etc. None. Any such attempt would immediately expose the "test" as ridiculous.

General intelligence arises as skillful adaptive control over one's environment, through sensory-motor concept aquistion, and so on.

It has absolutely nothing to do with whether you can emit text tokens in the right order to fool a user about whether the machine is a man or a woman (turing's actual test). Nor does it have anything to do with whether you can fool a person at all.

No machine whose goal is to fool a user about the machine's intelligence has thereby any capacities. Kinda, obviously.

Turing's test not only displays a gross lack of concern to produce any capacities of intelligence in a system; as a research goal, it's actively hostile to the production of any such capacities. Since it is trivial to fool people; this requires no intelligence at all.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#356

Post by nl »

mjburgess wrote: post #325
lucubratory wrote: post #313

Earlier quoted context omitted.

Can you provide those examples?

all statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture. eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words. But these words ar…

> The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes.

This sounds so logical and authoritative. And yet:

me> What event would cause a change in what all people said about Ukraine/Russia pre/post a few hours of 2022

GPT4O> A significant event that caused a drastic change in global discussions about Ukraine and Russia in 2022 was the Russian invasion of Ukraine, which began on February 24, 2022. This military escalation led to widespread condemnation from the international community, significant geopolitical shifts, and a surge in media coverage. Before this invasion, discussions were likely more focused on diplomatic tensions, historical conflicts, and regional stability. After the invasion, the discourse shifted to topics such as warfare, humanitarian crises, sanctions against Russia, global security, and support for Ukraine.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#357

Post by elktown »

camillomiller wrote: post #13

Why do we give this people credit by giving for granted that AGI is somehow magically around the corner? It is not, and everyone’s delusional.

The only reason "AGI" is around the corner is because people with vested interests - emotional, profit etc - are going to redefine it to suit their purposes. We've already seen this with AI which is now just any bog standard algorithm/app with an attached marketing sticker.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#358

Post by robbie-c »

edanm wrote: post #344
robbie-c wrote: post #328

Earlier quoted context omitted.

Here's the link if you want to read the contract terms: https://twitter.com/KelseyTuoc/status/1791539443016536265 They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring. Here's…

That tweet thread confirms what I said, at least in a roundabout way. Per the next tweets: > Equity is part of negotiated compensation; this is shares (worth a lot of $$) that the employees already earned over their tenure at OpenAI. > Employees are not informed of this when they're offered compensation packages that are heavy on equity. That makes it sound like it's something that they agreed to when accepting their…

> I'm not sure of the structure of what they signed before and this article doesn't get into it, but I maintain that there is no legal mechanism to tell someone "sign this or I take away something you already received". There has to be more to the story than the way I've seen it presented so far.

It's in that whole collection of tweets, but essentially the contract which you sign on joining says that if you leave and you don't sign the exit paperwork then you lose your equity. You aren't told what is in the exit paperwork at this point.

When you leave, the exit paperwork contains the non-disparagement clause.

Whether or not this is legally enforceable, no idea, IANAL. It does sound like enough of a threat that most people would just sign though

Re: Sam and Greg's response to OpenAI Safety researcher claims

#359

Post by ayhoung »

carliton wrote: post #259

LLMs are useful to a certain extent, but from my usage they are not ready for anything harder than very basic tasks. I feel like this megaphone about AI safety and creating a sense of doom is a strategy to increase importance of OpenAI and exaggerate the capabilities of LLMs. This era of “AI” is all about pretending that machines can think and the people working in these machines are prophets.

These comments are always confusing to me. Do you not believe that LLMs are going to get better?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#360

Post by ToastWithShrimp »

AlbertCory wrote: post #267

some telltale word smells of CorpSpeak BS. Where's Orwell when we need him? "putting in place the foundations" "keep elevating our safety work" "As models continue to become much more capable" mindless throat-clearing. In fact, beginning a sentence or phrase with "as" is classic CorpSpeak "a very tight feedback loop" "We are also continuing to collaborate with governments and many stakeholders " anytime anyone says t…

What else did you expect?
Post reply on HN