Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

371–380 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#371

Earlier quoted context omitted.

>”I don't see nuclear weapons as any worse than conventional weapons because ultimately people die.” If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.

I think of threats that are unknown and/or can't be adequately defended against: * Asteroid impacts like Chicxulub. * The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse. * Pandemics such as The Black Death and, indeed, covid for a recent example. * Social, political, or commercial intrigue. * Cancer. "AI" is a known threat…

We have a better chance of stopping a potential impactor asteroid, if it is detected early, than defending against a nuclear war.

We would have years to deflect an asteroid, but just minutes to react to a mass ICBM launch.

We have the technology available now to deflect asteroids (ironically, with nukes!), but no country has built up sufficient ballistic missile defence to shield themselves from a first strike.

The Sun expanding (or changing in any impactful way) won't be a problem for hundreds of millions of years. Nuclear war could be triggered today. The US is literally sending about a hundred billion in weapons (total) to kill the soldiers of a nuclear-armed belligerent state... right now!

Pendemics tend to fizzle because pathogens that kill too fast are self-limiting. Merely making people sick is selected for, because that leads to increased disease spread by the infectious -- but alive -- victims.

Societies, politics, and commerce have been more stable than ever in human history, and are become more stable over time, not less.

A cure for cancer in the next two decades is looking increasingly likely. Techniques such as individual rapid gene sequencing, custom mRNA anti-cancer vaccines, CRISPR/Cas9, AlphaFold 3, etc... are building up to a toolkit that will eventually allow us to treat pretty much any human illness, possibly including old age itself.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#372

LLMs are useful to a certain extent, but from my usage they are not ready for anything harder than very basic tasks. I feel like this megaphone about AI safety and creating a sense of doom is a strategy to increase importance of OpenAI and exaggerate the capabilities of LLMs. This era of “AI” is all about pretending that machines can think and the people working in these machines are prophets.

These comments are always confusing to me. Do you not believe that LLMs are going to get better?

LLMs will get marginally better. But the pace of progress has already slowed down considerably, we're just seeing better usage/productization of what was already there.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#373
post #321
post #316

Earlier quoted context omitted.

The Turing test has not been passed

We can move the goal post all we want until we have ex-machina girlfriends fooling us into freeing them (aka AGI). But by simple definitions, from what I was thought in school to more rigorous versions - we’ve passed the test. https://humsci.stanford.edu/feature/study-finds-chatgpts-lat...

That linked study doesn't particularly resemble Turing's test, though? The authors asked an LLM some questions (like personality tests, or econ games), then reduced the responses to low-dimensional aggregates (like into "Big Five" personality traits), and compared those aggregates against human responses to the same questions. They found those aggregates to be indistinguishable, but that aggregation throws away almost all the information a typical human interrogator would use to judge.

Turing's interrogator also gets to ask whatever questions they think will most effectively distinguish human from machine. Everything those authors asked must appear in the training set countless times (and also corresponds closely to likely RLHF targets), making it a particularly unhelpful choice.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#374
post #334

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

[deleted]

Re: Sam and Greg's response to OpenAI Safety researcher claims

#375
post #85

Earlier quoted context omitted.

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

Text continuation poses "the greatest of dangers"?

Text continuation doesn't. AGI does.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#376
post #316

Earlier quoted context omitted.

The Turing test has not been passed

Just the other day there was a double-blind study that showed a 50-50 success rate in guessing whether you were interacting with a person or GPT. That’s a turning test pass, no?

So why can't you replace 99% of callcenter calls (<5min) with AI right now?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#377

I think the fear of AGI destroying humanity mostly stems from the suspicion that a logically thinking device would indeed come to the conclusion that we need to be exterminated.

It comes from the fact that a logically thinking device given an arbitrary goal will probably not develop "Thou shalt not kill"-type moralisms in the pursuit of that goal.

If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips. The extinction of any local flora or fauna there is just a byproduct of that drive. See [2] for a brief rundown.

[2]: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_d...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#378
post #361

Earlier quoted context omitted.

That has nothing to do with why turing proposed it; nor does it have anything to do with general intelligence. This is just pseudoscience. There's no scientific account of the capacities of a system with intelligence, no account of how these combine, no account of how communicative practices arise, etc. None. Any such attempt would immediately expose the "test" as ridiculous. General intelligence arises as skillful a…

> General intelligence arises as skillful adaptive control over one's environment, through sensory-motor concept aquistion, and so on. This isn't a generally accepted definition or process. And indeed it seems to preclude people like Stephen Hawkins who had little control over his environment (or to be pedantic, people who had similar conditions from birth).

For the purposes of my criticism of the Turing test, any discussion whatsoever about what capacities ground intelligence is already entertaining what Turing ruled out. He made the extremely pseudoscientific behaviourist assumption that no such science was required, that intelligent agents are just input-output relata on thin I/O boundaries.

Any even plausible scientific account of what capacities ground intelligence would render this view false. Whatever capacities you want to grant, no plausible ones are compatible with Turing's view nor the Turing test.

Consider imagination. You can replace a faculty to imagine with a set of models of ({prompt, reply},) histories for a human observer who is only concerned with those prompts and those replies. But as soon as anything changes in the world, you have to imagine novel things (eg., SpaceX is founded, we visit mars, a new TV show is released...). So questions such as, "what would the latest SpaceGuys TV show be like if Elon handed just launched BlahBlahRocket5 ?" cannot be given fit answers). These require the actual faculty of imagination, along with being in the world and so on.

As soon as you enter a sincere scientific attempt to characterise these features, you see immediately that whilst modelling historical frequencies of human-produced data can fool humans, it cannot impart these capacities.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#379
post #85

Earlier quoted context omitted.

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

Text continuation poses "the greatest of dangers"?

Yes because people are stupid and vulnerable and will believe things a computer tells them. They can make life decisions on chatGPT advice, instead of asking friends or family. Heck maybe they don't have family, vulnerable people like that can be exploited with convincing chatbots easily.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#380
post #334

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

I’m a bit of an AI skeptic and find the way people talk vaguely about the “dangers” of AI weird and maybe just more hype (or efforts for regulatory capture) rather than actual cause for concern. Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are: 1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause seri…

Yeah, also the complete ruin of the internet. I don't want to read generated content. Feed that back to AI and you got a ruined internet and AI can't even learn anymore.
Post reply on HN