Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

201–210 of 990 posts

Re: I resigned from Anthropic today

#201

Post by slicktux »

copperx wrote: post #153
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

Re: I resigned from Anthropic today

#202

Post by par1970 »

jeffrwells wrote: post #189
par1970 wrote: post #171

Earlier quoted context omitted.

AFAIK this is the document that talks about GPT-2 being dangerous: https://openai.com/index/better-language-models/ Here are some direct quotes: “We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate): * Generate misleading news articles * Impersonate others online * Automate the production of abusive or faked content to pos…

Nice try Dario. Alignment is a real and valuable discussion topic. The GP fake tweetstorm is not the correct approach, is my point

You said this: "Screams of propaganda. Sama saying GPT-2 is too dangerous to release…all over again."

So show me Sam's "too dangerous to release" propaganda for GPT-2.

Re: I resigned from Anthropic today

#204

Post by skybrian »

EA-3167 wrote: post #191
skybrian wrote: post #162

Earlier quoted context omitted.

Trends don't go on forever, but the market can stay irrational longer than you can stay solvent. There's no good rule of thumb for this, other than maybe the Lindy effect.

I won’t presume to time it, but at this point I think anyone can see what’s coming. It’s precisely because it can’t be timed that a sane person should stand well clear.

I certainly can't see what's coming. I believe that nobody really knows what's coming. Some people are overconfident.

Re: I resigned from Anthropic today

#206

Post by areoform »

nunez wrote: post #196
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

Re: I resigned from Anthropic today

#207

Post by chews »

nunez wrote: post #196
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

you should kick the tires on an unfiltered (abliterated model) it's the closest thing to having a real conversation with the devil. There is good reason for the concern's outlined above and undoubtedly Anthropic / OpenAI have internal unfiltered models with no safety... they got freaked out based on how they work and are virtue signaling alarm... all while selling out to defense contractors.

Re: I resigned from Anthropic today

#208

Post by reasonableklout »

skeledrew wrote: post #148
hegelstoleit wrote: post #119

Earlier quoted context omitted.

That's not necessarily true, and you can use that argument to justify doing any immoral job. Just because someone else might be willing to do it isn't a reason to continue doing it.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

But organizations are made of humans. Jacob leaving might've moved some of his coworkers and his counterparts at OpenAI. And the same can be said for those who would fill positions at the frontier. Then finally there is a political component; his post went viral, 100K+ users appear to agree with it, and it is further fuel to the fire for regulation, which we already know most Americans want.

Re: I resigned from Anthropic today

#209

Post by bpodgursky »

areoform wrote: post #206
nunez wrote: post #196

Earlier quoted context omitted.

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

Design a novel virus which is far more lethal than COVID-19 (Ebola, smallpox, take your pick) and can evade existing vaccines.

Re: I resigned from Anthropic today

#210

Post by beezlewax »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

> They do as they are told

This isn't strictly true.

It it also where part of the problem might lie.

Nefarious humans making bad decisions.

Post reply on HN