Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

261–270 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#261
post #201

Earlier quoted context omitted.

It's essentially Pascal's wager.

What probability for extinction do you consider to be low enough that it effectively becomes a Pascalian wager? Pascal was also writing about a single person's fate (AFAICT), whereas this is about the fate of everyone.

The point is that the probability doesn’t matter because the outcome, total human extinction, is given infinite weight.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#262
post #155

Earlier quoted context omitted.

Ok, what actions have they taken that are unethical?

Um, closing access to their model code when "Open" is right there in their name and was the original promise they made? Plenty of other posts in this discussion give other examples. That's without even getting into all that happened with Altman leaving and then coming back, which has already been discussed to death in other HN threads.

It's closed source but freely available to over 100 million people. To everyone but tech people, it's open.

What's more valuable for the normal person, Lllama, that I will never be able to use because I don't know how to get it to run, or ChatGPT that I can use for whatever I want?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#263

Earlier quoted context omitted.

> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

> Could AI running amok destroy the human race?

How do you envisage AI to run amok at the current level of the technlogical progress? Insofar, AI taking over the world and obliterating humanity is the subject of the sci-fi material in books and movies only.

Misuse of AI by wicked state agents or evil inclined parties with nefarious intentions is a major issue, but we are in the same situation as with, say, a fork that can be a utilitarian utensil or a killing weapon depending on who holds it and one's aims.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#264

Earlier quoted context omitted.

If you look at compute scaling and model improvement on that compute, we’re going to get there pretty fast. Both compute, architecture and cost matter, and those have been improving like crazy. I don’t know about you, but I didn’t expect GPT-4o to come out at half the price one year later, with real-time voice, image and text. There is zero sign of slowing, Nvidia keeps building beefier GPUs specialized in LLMs, the…

AGI seems like a stretch when none of the LLM’s right now can solve the prompt: “ Write a sentence that has an odd number of words, with the third word being "keyboard", the last word being "anyway", the next-to-last word being a number, and the first word being a palindrome “ Do we have to add these examples to the dataset now, so that they can solve this? Then maybe we find another prompt and add that too, as a hum…

Why is this a goalpost for just about anything? This seems completely uncorrelated, and I'm sure many fully grown adults would fail under similar timing conditions too.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#266

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the…

Yes, having eyes is incredibly dangerous. It allows people to select a target from anything they can see.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#267
some telltale word smells of CorpSpeak BS. Where's Orwell when we need him?

"putting in place the foundations"

"keep elevating our safety work"

"As models continue to become much more capable"

mindless throat-clearing. In fact, beginning a sentence or phrase with "as" is classic CorpSpeak

"a very tight feedback loop"

"We are also continuing to collaborate with governments and many stakeholders "

anytime anyone says they are "continuing to collaborate" it's usually BS

"carefully weigh feedback" "carefully" is a twin of "collaborate"

"proven playbook"

someone on Twitter comments, "This was definitely written by Sam. You can tell because it says absolutely nothing at all."

Re: Sam and Greg's response to OpenAI Safety researcher claims

#268

Earlier quoted context omitted.

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

> Could AI running amok destroy the human race? Yes. Could a human running amok destroy the human race? If yes, why is an AI doing it worse? If no, what capabilities is this AI assumed to have that a human can never have?

Humans are generally afraid of death, or have others they are fighting for; there are certainly humans that don't want power so much as they want to watch the world burn, but they seem to not intersect well with the set of humans who are adept enough to cause a lot of world-scale damage.

With an AI, it isn't clear that it will care about things the same way we care about things, or if it does care about things if those things are things we care about as well; the AI also is capable of knowing simultaneously all of the varied expertise of an army of humans.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#269

Earlier quoted context omitted.

They are going to say the usual things, child pornography, risk of terrorist use etc... and conclude that we need our overlords to protect us. Same reasons that have been used in the past to take away freedoms. Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disre…

Child porn is already heavily "regulated". Producing the stuff is illegal and AI service providers must take steps to prevent it.

Are you saying they aren't already preventing it? I would find that hard to believe...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#270

Earlier quoted context omitted.

> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. ...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."

An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living…

I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology.

You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.

Post reply on HN