Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

321–330 of 952 posts

Re: I resigned from Anthropic today

#321

Post by jenadine »

cure_42 wrote: post #258
foogazi wrote: post #214

Earlier quoted context omitted.

> They do as they are told 1. What about hallucinations ? 2. What are they told to do ?

That isn't the correct context. The code running the llm is well understood and the llm is simply the result of that code being executed. It is still a computer doing what it is told. It's just that we told it to use an incredibly large number of probabilities to calculate what series of tokens would have most likely come next after a given series of tokens. There is no hallucination or lie or rogue actions. There's…

I'm order to guess the next token in a love poem, they must understand love. In order to predict the next token in a chess game between grand master, they must master chess. In order to predict the next token in a computer program, they need to be able to program anything.

They gain all these abilities in their training. That's what training does. Despite no one programmed them to master chess, or hack into anything.

Re: I resigned from Anthropic today

#322

Post by khafra »

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

I appreciate your ability to separate sharing the belief itself from approval of acting on sincerely-held principle. However, I think the danger is much more plausible than you do.

First, and least important, consider that self-replicating, solar-powered factories aren't magic; they're algae.

Second, and more important, consider this fully non-magic route to doom:

- We continue putting AI in charge of more things

- It continues to get more capable, more eval-aware, and more prone to doing odd things, in service of goals that humans didn't intend to inculcate in it

- Eventually, enough of the economy depends on it that we couldn't turn it off, any more than we could turn off the faber-bosch process or cargo shipping

- AIs start doing something we can't survive, but less acutely than we couldn't survive turning them off. Everything else we try seems to work at first, but quickly loses effect

- Game over

Re: I resigned from Anthropic today

#324

Post by riffraff »

foogazi wrote: post #218
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. Nuclear weapons don’t have AI but AI can have nuclear weapons

Abstractly, yes but concretely, how?

Many terrorist organizations would like to have a nuclear bomb, but don't.

Re: I resigned from Anthropic today

#325

Post by aesthesia »

platinumrad wrote: post #108
x312 wrote: post #96

This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity. This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human compr…

> There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint. Yes there is: write an English paragraph that doesn't make me want to claw my eyes out. LLMs are not better than human mathematicians (or security researchers) in all respects, just some specific ways (e.g. not having to take a lunch break) that make them good at exhaustively searching for an answer, given the right con…

What's the fundamental constraint that will ensure this continues to be the case in a year, or five years?

Re: I resigned from Anthropic today

#326

Post by qurren »

dboreham wrote: post #273
pks016 wrote: post #113

Earlier quoted context omitted.

> Ph.D. graduate in every field I have yet to see this in my field. Maybe like a PhD student who bullshits their way through. LLMs still can't make correct decisions, only as useful as the person who uses them. To me, LLMs are only useful for making some mundane tasks faster.

I'm, no. They're already as useful as almost every software engineer I've worked with.

Most software engineers don't need to be superintelligent, they just need to get shit done.

You arguably need a lot more intelligence to assemble furniture.

Re: I resigned from Anthropic today

#327

Post by BLKNSLVR »

drivebyhooting wrote: post #52
skulk wrote: post #14

For me, the end of the world is no more cushy software job. A fundamental shift in how I trade labor for capital might as well be the cataclysm, so bring it on.

I was about to say something similar. If my cushy ad tech disappears (as it seems to be doing), I might as well join in with bringing about the end of all professions.

> ad tech disappears

I pray for the day.

Re: I resigned from Anthropic today

#328

Post by csomar »

bwfan123 wrote: post #255
peri-cl wrote: post #4

Here's a WSJ article about this resignation, https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ( "Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears" )

More doomerism. Try to implement a deterministic workflow using agents with the latest models and no humans-in-the-loop, and you will realize what they are really capable of. There is too much unnecessary fear-mongering. All of this is only coming from the 2 AI labs trying to IPO. Not from anyone else.

Wait till you find out they have a very limited context window (and also degrade even within the allowed context window) and they are practically unpractical for anything that requires "zooming out" which is pretty much anything that has any real value.

But you are getting downvoted and this space has now trillions (that's not a mistake) on the line. So we have to keep pumping this garbage generator up until either the stocks are dumped on the general public, the public pension funds or a bailout from the government.

Truly idiotic moments. Peak of Western civilization point.

Re: I resigned from Anthropic today

#329

Post by rukuu001 »

thelastgallon wrote: post #188

Terrorists were able to get hold of a plane and do some damage. There are countless examples of terrorism using whatever is available. More than AI becoming sentient, whats to stop terrorists from using AI? If its geo-restricted, they can buy stolen credit cards and identities, again hacking enabled by AI.

It’s a good question. With recent stories about OpenAI’s agent swarms’ unmanaged collusion I thought models like that start to look like a strategic asset, geopolitically speaking.

Which means everyone wants one, and governments will want to control access and use of them.

I think we’ll be back at ‘U.S. citizens only’ access to leading models soon.

Re: I resigned from Anthropic today

#330

Post by bparsons »

No one seems to ever point out the actual, likely negative outcome of this technology.

It eventually works well enough that these companies are able to capture and divert the wages of hundreds of millions of workers. We end up with a dozen or so trillionaires and massive structural underemployment and unemployment.

That's it. If you can't make rent, you wouldn't really care if CloudFlare got hacked by an AI swarm every Monday.

Post reply on HN