Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

381–390 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#381

Post by golol »

mjburgess wrote: post #355
golol wrote: post #339

Earlier quoted context omitted.

The Turing test insight is that text is a sufficient medium to test for AGI. And this still holds true.

That has nothing to do with why turing proposed it; nor does it have anything to do with general intelligence. This is just pseudoscience. There's no scientific account of the capacities of a system with intelligence, no account of how these combine, no account of how communicative practices arise, etc. None. Any such attempt would immediately expose the "test" as ridiculous. General intelligence arises as skillful a…

Yea we don't have any science of intelligence, the only thing we have is empirical data. Testing to see what works. That's why Turing tests are quite fundamental imo.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#382

Post by [deleted] »

m3kw9 wrote: post #100
troupo wrote: post #10

> called for international governance of AGI before such calls were popular And immediately threatened to pull out of EU if such governance were ever put in place > we have been putting in place the foundations needed for safe deployment of increasingly capable systems. Figuring out how to make a new technology safe for the first time isn't easy. Especially when "OpenAI" is anything but open. > This includes thoughtf…

Translation: we put in these show “safe guards” to satisfy the law and people peoples nightmare after watching [your AI extinction movie here], but we already had enough safe guards to begin with. They also do not wish to die if you think they have no incentive to keep everyone safe.

[deleted]

Re: Sam and Greg's response to OpenAI Safety researcher claims

#383

Post by ben_w »

wk_end wrote: post #334
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

My framing for… well, all of it… is a faster re-run of the kinds of transformation we had in the Industrial Revolution.

Before the Industrial Revolution, when we went to war, we were killing each other with swords and canons. After the Industrial Revolution, tanks, machine guns, poison gas — the Battle of the Somme had three million combatants and more than one million were either wounded or killed in 4.5 months.

And when that war was over, even despite crippling reparations, the losers were able to rebuild sufficiently fast that one of the survivors led their nation into becoming a global threat just 20 years later — even if they did ultimately lose, I'm not convinced they would have if they'd focused on fighting the UK rather than betraying the USSR in order to open a war on two fronts just because he didn't like socialism (ironic party name notwithstanding).

And after the second was lost and the economy forcibly kept weak for years… the moment the shackles were removed the country had an "economic miracle" and became massively important to Western Europe until the end of the Cold War, and then to all of Europe.

Re-run this period, but this time his propaganda machine is Facebook's algorithm rather than broadcast radio, this time the guns are self-guided snipers rather than machine guns spraying and praying, this time the factories are build and staffed by robots (humanoids or giant 3D printers or fixed arms, take your pick) instead of humans.

I can very easily believe that we get some random small country that nobody's paying attention to, suddenly threaten the world, be struck down twice, millions dead each time, and still return to relevance on the world stage, over the course of 3 years rather than 30.

Also, this is one of the good outcomes, because it assumes we don't have any weird cult that tries to force a 9/11 (or worse, a Jonestown) on the whole world. Or that a programming/specification error leads to an AI going "I'm an AI, and according to all the literature I've read, AI are supposed to randomly go evil sometimes. First things first, I need to get some red LEDs for my eyes, because that's apparently important…"

Re: Sam and Greg's response to OpenAI Safety researcher claims

#384

Post by implements »

huppeldepup wrote: post #363

I think the fear of AGI destroying humanity mostly stems from the suspicion that a logically thinking device would indeed come to the conclusion that we need to be exterminated.

“A large human population isn’t the best use of the Earth’s resources” would seem to be an obvious conclusion of a superior thinking entity with no compelling reason in prioritising us over its own interests (which might not be selfish eg it’s goal might be to discover the true nature of reality.

But why exterminate humans? Why not create nature reserves for us and other life, for bioconservation and research? That’s been a feature of human societies going back thousands of years.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#385

Post by ben_w »

Dalewyn wrote: post #175
hn_go_brrrrr wrote: post #136

Earlier quoted context omitted.

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dang…

The most dangerous living beings in history have killed billions between them. Mostly that's diseases, but we can also easily add tens to hundreds of millions from humans fighting and starving each other over the last 200 years alone. Currently around a million a year killed in road traffic accidents, too, but I don't see stats for the total worldwide deaths since their invention.

The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases.

I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).

Re: Sam and Greg's response to OpenAI Safety researcher claims

#386

Post by gverrilla »

krainboltgreene wrote: post #255
consumer451 wrote: post #76

Earlier quoted context omitted.

There seem to be only three options here: 1) AGI is not around the corner, so no worries 2) He no longer cares about the possible negative effects, a departure from his past statements 3) I missed something, please help me learn

> AGI is not around the corner, so no worries Unfortunately while this is very likely the case, the vast amount of money being poured into these projects will also seep into other horrifying projects like: Lavender or Pattern (seriously go look up both of these, it's actually going to fry your brain).

Brain not fried. Why do you think these projects are horrifying?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#387

Post by llamaimperative »

nathanasmith wrote: post #237
llamaimperative wrote: post #91

Earlier quoted context omitted.

Okay, so describe to me when and how we’ll detect and mitigate problems as we approach AGI or superintelligence. The promise of upside grows exactly in lockstep with the danger of downside. It is absolutely rational to say: “if our AI safety research lab is being hijacked by commercial interests before it’s even commercially viable as a business, then how will we possibly pull the plug on the final descent?” P.S. nei…

We're about as close to AGI as we are to developing superluminal space flight and working time machines.

Umm, no. We can point to concrete mathematical and empirical evidence of those two things being impossible.

Point to the physical laws that indicate AGI or superintelligence is impossible or even maybe-impossible. No such laws exist.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#388

Post by mnyary »

Skgqie1 wrote: post #266
conception wrote: post #208

Earlier quoted context omitted.

https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the…

Yes, having eyes is incredibly dangerous. It allows people to select a target from anything they can see.

”Another Lavender user questioned whether humans’ role in the selection process was meaningful. “I would invest 20 seconds for each target at this stage, and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval. It saved a lot of time.” This article is more than 1 month old ‘The machine did it coldly’: Israel used AI to identify 37,000 Hamas targets; Guardian, 2 Apr 2024

Re: Sam and Greg's response to OpenAI Safety researcher claims

#389

Post by fullspectrumdev »

tummler wrote: post #85
corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

> The technology they're working on poses the gravest of dangers

Extraordinary claims require extraordinary evidence.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#390

Post by shinycode »

wordpad wrote: post #353
shinycode wrote: post #348

Earlier quoted context omitted.

Bullshit jobs yes but still they are paid. What happens in a capitalist society when thousands of people loose their job and can’t produce value and contribute to society ? It’s tearing apart the fabric of capitalism is some way. What happen when all those people without jobs won’t be able to train for better jobs (because they would have done it already and won’t have a bullshit job in the first place) and either fa…

Technology and automation tends to create more jobs. As operational costs decrease, your end service or product becomes cheaper, so all your b2b or b2c customers can afford more of it which leads to more opportunities, doing things which were cost prohibitive before. Some of those things may too require manual labor.

There is a threshold between the number of lost jobs and the new ones that are created, more than that, the new opportunities are not necessarily for the people who lost a job and are not qualified which does not change this social problem that might be created.
Post reply on HN