Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

301–310 of 956 posts

Re: I resigned from Anthropic today

#301

Post by riffraff »

copperx wrote: post #153
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

"collect underpants... Profit" comes from south park

https://en.wikipedia.org/wiki/Gnomes_(South_Park)

Re: I resigned from Anthropic today

#303

Post by gorgoiler »

SaucyWrong wrote: post #132
achenatx wrote: post #90

Earlier quoted context omitted.

they dont need to be smarter than humans. They just need to be able to hack into vital infrastructure systems faster than we can repair them while also replicating wildly

> while replicating wildly Earnest question: by what mechanism that exists today would the achieve that in a way humans on top top of the situation could not curtail? All of this runs on top of compute in meatspace that humans can disconnect.

I am not an AI super mind hell bent on consolidating my power by leveraging chaos to take control of humanity’s resources, but if I were then sending one million deepfaked ransom emails to impressionable people would be the best tool for effecting change in meatspace.

We have your daughter / dog / Amazon delivery. If you ever want to see her / him / it again, plug this USB drive into the control panel at your station / let off the parking brake roll your car into this substation / change the meatpacking thermometers to read 8C lower than calibrated / ground your vessel on this sandbank / send an envelope of white powder to these addresses / set fire to the following hospitals / …

Re: I resigned from Anthropic today

#304

Post by epihelix »

platinumrad wrote: post #97
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Anthropic is a company full of basilisk believers.

Yes, but the really weird thing is that they seem to:

a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it

I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)?

That seems to be why this individual resigned, but I'm surprised it's not all of them. The cakeism is strong in that company.

Re: I resigned from Anthropic today

#305

Post by caconym_ »

csomar wrote: post #222
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Are the models improving? Because I am not seeing it. I have been trying Astra for a few quantifiable tasks in my codebase and performance wise, it's pretty similar to sol 5.6. Now when it comes to expressing the problem/solution, holy Christ, what a mess the writing has become. It is on the level of Opus 5. Now when it comes to burning money, Astra is just insane. With a $100/month subscription, you can easily burn…

Some people claim Astra is significantly better than anything else and significantly more token-efficient, and others (like you) say it's meh and way more expensive to boot. I really don't know what to think.

Kind of a tangent, but one thing I am curious about is to what degree the Navier-Stokes result announced today was primarily a brute-forced result based on the 'program' previously established by researchers to find counterexamples (blowups), or whether the model actually added significant/novel intellectual value beyond its ability to run at arbitrary parallelism. With 10K agents and a staggering $15M in compute (IIRC), I am feeling like a lot of the former may have been involved, but I don't really understand either the problem or the approach (or, indeed, the solution).

Obviously the potential for parallelism and coordination between so many agents is quite scary by itself, but I think brute force by 10K mediocre AI mathematicians is much less scary than ~one AI mathematician reasoning its way through the problem where all human attempts have failed. It seems fairly obvious that massive parallelism lends itself to brute-force counterexample-finding, and I suspect it isn't a coincidence that most of the touted AI math results have been counterexamples.

It's all still quite scary, but coming full circle: I really don't know what to think.

Re: I resigned from Anthropic today

#307

Post by choppsv1 »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

Like a magic Monkeys Paw, perhaps.

Re: I resigned from Anthropic today

#308

Post by cure_42 »

cameldrv wrote: post #293
cure_42 wrote: post #258

Earlier quoted context omitted.

That isn't the correct context. The code running the llm is well understood and the llm is simply the result of that code being executed. It is still a computer doing what it is told. It's just that we told it to use an incredibly large number of probabilities to calculate what series of tokens would have most likely come next after a given series of tokens. There is no hallucination or lie or rogue actions. There's…

You’re just a bunch of molecules following the laws of physics. It’s all just physics and chemistry, and those are well understood. Now explain the causes of World War I using chemistry and physics. Simple, right?

It isn't hard to program a gpt. You can do it in a weekend with a few hundred lines of python. The code is pretty simple. The math is not particularly high level.

The complexity and scale with LLMs come from the amount of training data used, not some kind of black magic in the programming.

Re: I resigned from Anthropic today

#309

Post by aogaili »

foogazi wrote: post #226
aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

> There are thousands willing to do his role 1. Does that matter ? There are thousands willing to do my role - what impact does that have on me doing it or not? 2. Why weren’t these thousands doing it already? Willing to and able to are different things

I think you are looking at it from individuals perspective.

I see a fast moving train with no brakes. Just like biological evolution, we are locked in an a global technological arm race, that is beyond any individual. It is as if the universe decided to wake up and run, who are you to say no?

One would argue that the best solution for this is to own the most sophisticated AI that is aligned with what we perceive as good values. Because given the situation we are in, if those tools are going to be gods anytime soon, then we better have some gods working on our side.

Re: I resigned from Anthropic today

#310

Post by jiggawatts »

frrt wrote: post #276
jiggawatts wrote: post #266

Earlier quoted context omitted.

> ... current evidence ... Is a load bearing term! (pardon the pun). AIs are now tackling Millennium Prize Problems, which our best and brightest have failed to solve, despite trying very hard for decades to claim the $1 million reward money, not to mention the fame! You have no way to judge from the AIs of "today" what the AIs of... literally tomorrow (not even next year) will be able to do in terms of replacing hum…

[dead]

[flagged]
Post reply on HN