Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

951–960 of 983 posts

Re: I resigned from Anthropic today

#951

Post by Jblx2 »

xelxebar wrote: post #748
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Your call for clarity has merit, but I think you have backed yourself into a corner, honestly. In 1900, there was no evidence of the kind you seek that lighter-than-air flight was possible. Good thing some people were foolish enough to ignore you then. Why you would cordon yourself off from the kind of reasoning that predicts legitimately new things, rather than just scaled up versions of the present? Not to put too…

>In 1900, there was no evidence of the kind you seek that lighter [sic]-than-air flight was possible.

Presumably you meant heavier than air? Also, I'm pretty sure that birds existed in the years leading up to 1900.

Re: I resigned from Anthropic today

#952

Post by TheCoelacanth »

ozozozd wrote: post #700
mitthrowaway2 wrote: post #610

Earlier quoted context omitted.

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

“Self-sustaining reaction,” like the recursive self improvement? Or perpetual machines? Is physics no longer a main subject at schools?

I'm pretty sure they didn't think the reaction would be perpetual; just long-lasting enough to wipe out life on Earth.

Re: I resigned from Anthropic today

#953

Post by andy99 »

riffraff wrote: post #301
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

"collect underpants... Profit" comes from south park https://en.wikipedia.org/wiki/Gnomes_(South_Park)

Found it really funny to see this attributed to a Slashdot meme

Re: I resigned from Anthropic today

#954

Post by skeledrew »

hegelstoleit wrote: post #927
skeledrew wrote: post #904

Earlier quoted context omitted.

> the fact that it's an immoral thing to do This is carrying a lot of weight. What makes working on frontier models "immoral"? Because there is potential for misuse? Would you also say working at a knife factory is immoral? Or even at a nuclear or biological pathogens research lab? After all those too can be misused. And if one does suspect a workplace is moving in a bad direction, and there's no immediate danger to…

Well we're not debating the morality of it, the question is what he should do, given that he believes that it's immoral. In your analogy the correct thing to do would be to call the authorities to help. The problem with the analogy is that there isn't a 911 number to call for this scenario. You need to appeal to the people who have the power to do something about the situation. Quitting in a very public way is one wa…

By the time authorities get there, if they actually prioritize it (police in many places already have a track record for not being very responsive, unless there're actual signs of a crime), something may have already happened to the child. You don't know because you already left, expecting that someone else will take care of it, when you were already in a position to help ensure that child's safety. No, if you think something might be wrong you stay and monitor the situation best you can, so if something really does begin to actually begin to happen you can be a buffer and/or whistleblow.

Re: I resigned from Anthropic today

#955

Post by andai »

hypfer wrote: post #481
andai wrote: post #427

> A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk. Which means they have to go faster, which means less responsibly? I he…

I believe that the actual alignment happens in.. uh.. "meatspace". Someone is prompting. Someone is hosting. That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire. Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.

My point was you only need to prompt a worm once, and then you are no longer the one prompting it, it prompts itself by definition from that point onward.

Also I think there was some confusion about the word "spreading" in the sibling thread.

If the HF hack was done by humans we would have used the word spreading in that context, even though they were not replicating their minds. So spreading does not require replication. But we're either already at the point, or very close, where an open weight model could do a similar attack. So that actual replication could be involved.

(I'm not sure what that would add, but I don't see what obstacles remain at this point.)

Re: I resigned from Anthropic today

#956

Post by hnburnsy »

Findecanor wrote: post #721
icepush wrote: post #503

Earlier quoted context omitted.

Global warming is happening at a slow enough pace that there is plenty of time for people to move to the poles/into caves/beneath the ocean/whatever. Nuclear war would be much more destructive in a much shorter range of time but you will have pockets.

People already entrenched near the poles will fight to keep others out. Right-wing anti-immigrant parties are on the rise right now already. They also tend to not want to recognise climate change -- because that would imply that people could migrate because of it.

> People already entrenched near the poles will fight to keep others out. Right-wing anti-immigrant parties are on the rise right now already. >

Like those autonomous zones, remind me again which parties set those up?

Re: I resigned from Anthropic today

#957

Post by cloverich »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

Narration is an obvious one. Theres also a huge risk to less qualified folks and jr employees. ie in most software shops, youll get more mileage giving your best engineers bigger opus budgets and a raise than jr engineers to train. Which means we meed maybe half or less as many, and at least half of that savings is floated up to sr folks and ai companies. That seems rather disruptive?

Re: I resigned from Anthropic today

#958

Post by dekhn »

asdff wrote: post #402
dekhn wrote: post #31

There's a scene in the movie "War of the Worlds" by Spielberg where the protagonist's son walks into a war zone because he is entranced by the battle ( https://www.youtube.com/watch?v=X7rfWPbEufo ). He is obliterated (along with the rest of the US forces) shortly after. I've always been struck by that scene, because in a lot of ways, if we really are headed towards a superintelligence, I at least want to be there and…

You can't address the key problems without understanding of the esoteric side details to be fair. You are studying what is basically the worlds most complicated and undocumented computer.

Actually, I work in biology, which is a far more complicated and undocumented computer than any existing AI model. And we've already seen the failure models (COVID is an example).

Re: I resigned from Anthropic today

#959

Post by ethin »

senordevnyc wrote: post #910
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

They literally have never promised anything like that. They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And then I have to read page after page of cynical comments about how that's all just hype and marketing and them wanting regulation to block their competition and blah blah blah. You people would only be happy if they stopped all AI development, bu…

> ... They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc.

And yet they do the exact opposite of all of the above. They do everything to get regulation just to create a moat because one doesn't exist. They preach about wanting a slowdown, nationalization, a complete pause, whatever have you... And yet they aren't slowing down the development voluntarily now are they? If anything, they are doing everything imaginable to speed up development to pump out models as fast as possible. If these ex-risk and AI companies actually gave a damn about regulation, or oversight, or a moratorium on AI development altogether, they would actually demonstrate this by completely ceasing development of all models immediately. Instead, they take riskier and riskier actions to try to "win" this supposed "arms race". So please excuse a bunch of us if we find it impossible to take them seriously on literally anything like this. If you have substantial evidence that they are actually ceasing AI model development like they want everybody else to do (except them, of course), then, by all means, present it.

Re: I resigned from Anthropic today

#960

Post by dwaltrip »

techblueberry wrote: post #930
senordevnyc wrote: post #911

Earlier quoted context omitted.

It's head-in-sand denial because actually facing the threat (and our total inability to stop it, imo) is terrifying, and people don't like feeling that. So they're just angry and cynical instead. Plus it makes them feel smarter for whatever reason.

What, substantively speaking, are you doing differently from someone facing the threat, or are you just on the other side of the checkbox? There are quite a few existential risks on the table. Are you even sure you've sort ordered them correctly?

Can you explain the equivalence you are suggesting?

I don’t think it’s comparable at all. Both are head in the sand?

Post reply on HN