Live data from Hacker News

I resigned from Anthropic today

twitter.com

991–1000 of 1001 posts

Re: I resigned from Anthropic today

#991
post #324

Earlier quoted context omitted.

I'm, no. They're already as useful as almost every software engineer I've worked with.

Most software engineers don't need to be superintelligent, they just need to get shit done. You arguably need a lot more intelligence to assemble furniture.

As someone who has a lot of experience with both… no. You don’t need to be smart to assembly furniture, you just need to follow directions very closely and get shit done.

Re: I resigned from Anthropic today

#992

Earlier quoted context omitted.

> After the events of the summer What events are you talking about?

Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild

You do understand that these are PR stunts, right?

Re: I resigned from Anthropic today

#993
a lot of the discourse and mania around this topic reminds of of the movie don't look up.

while not my cup of tea and not going to rewatch it, i see a repeat of a lot of the ways people react to "world-changing" topics.

or in other words, a lot of hand-waving since past few years is only going to be crying wolf when maybe we ever get there. perhaps before nuclear fusion or gta 6!

Re: I resigned from Anthropic today

#994

Earlier quoted context omitted.

> I’m not advocating for this. You're not advocating for it because murdering people is highly immoral and illegal. That answers your question for the most part. You also have to remember that the magnitude and speed of this stuff is very new to pretty much everybody . It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially…

This is a non sequitur. Yes, if someone doesn't actually believe in imminent AI doomsday they are not going full Sarah Connor on Anthropic. Yes, HN will not like if someone pipe bombs the office. What does it have to do with @ericmay's point? He directly says "They are racing straight to self-improving superintelligence and gambling with our lives", that's a claim of imminent threat to humanity. If he truly believes…

> Why just make some tepid tweets if you think humanity is at risk?

Who says that's the only thing he is doing?

> Why doesn't he leak documents and messages with the unfiltered opinions of leadership?

Again, illegal. You're asking him to sacrifice his life (somewhat similar to Snowden). You can call not doing that cowardice or wisdom, but not proof that he doesn't believe what he is saying.

> Why isn't screaming "we are going to die" in CNBC?

Because that is up to CNBC to broadcast. Do you really think you can just call them up and get a timeslot for your soap boxing? Also, maybe he believes the tweets are a start to achieve that. He sure managed to get a lot of attention through them already..

Now I don't know the guy, nor do I care much about his tweets or life. I just think some people here try to confirm their own worldview by fallaciously framing his behavior as suspect.

Re: I resigned from Anthropic today

#995
post #712

Earlier quoted context omitted.

Does it check out? India and Pakistan both have nukes and keep semi-regularly fighting each other. MAD "on paper" prevents either side from going far enough to provoke the other into using nukes, but even then it's fundamentally flawed because it works on the assumption that both sides are both rational and believes the other side to be rational , as well as that both sides understands the others red lines well enoug…

Minor skirmishes are not war. China and India have border skirmishes with literal wooden sticks, neither side will use live ammo because they both have nukes.

Conveniently shifting to a different conflict, and ignoring that India and Pakistan have carried out missile strikes on each other, killed civilians, occupied territory.

Re: I resigned from Anthropic today

#996
post #336
post #96

This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity. This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human compr…

I really hate how people who have always thought AI research to be an existential risk for humanity, now are apparently bundled to be on the same side of the Sam Altmans and Dario Amodeis that are using the existential risk as a sneaky form of marketing for their products. You cannot discuss existential risks of AI without being seen as a booster, and that is very unhealthy for the discourse around this tech. I hate…

I have been wary of the risks surrounding AI since watching Robert Miles on Computerphile more than a decade ago. Since then I have paid at least some attention to developments coming out of AI safety research. Even back then the discussion was fraught with doubt which, given the nature of this topic, was to be expected. It is unfortunate that it is worse now, when the stakes are much higher.

Re: I resigned from Anthropic today

#997

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Exactly; what's stopping them, --- or someone --- from writing a firmware worm that essentially bricks all devices and turns them into pricey paperweights? It is not too hard to imagine that capable AI can probably exploit any back-doors that were created by three-letter agencies.

Reframing the issue: The way to get working nuclear non-proliferation agreements is if people stop pursuing the creation of nuclear weapons... the same goes for the scientists clamoring to not create or research mirror life, and for computer scientists warning against AI. There are theoretical dangers that are just too big to ignore.

Re: I resigned from Anthropic today

#998
post #995

Earlier quoted context omitted.

Minor skirmishes are not war. China and India have border skirmishes with literal wooden sticks, neither side will use live ammo because they both have nukes.

Conveniently shifting to a different conflict, and ignoring that India and Pakistan have carried out missile strikes on each other, killed civilians, occupied territory.

Like I said, minor skirmishes. No war between India and Pakistan since they both got nukes in the 70s. Not bad for two states that have been hostile their entire existence.

Is it all nukes? Probably not. International trade and globalisation is part of it. But it wouldn't exist without nukes.

Re: I resigned from Anthropic today

#999
post #509

Earlier quoted context omitted.

> Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Does a virus need to be a God? Replication and annihilation do not require very much intelligence. They're going to have all the intelligence they need though, to replicate and annihilate as much as they like. We should probably start building some kind of immune system.

What virus? People are seriously confused what is being created here. The claim here we will have misaligned AGI, not a mindless LLM. If you don't know and you don't agree what you are even creating, how can you defend against it? If this LLMs are statistical parrots, then the risk is someone using it as a weapon, to hack etc, but that is different risk category that why this person quit his job with different diffen…

>what virus

https://www.youtube.com/watch?v=5v5z5rjgOQY&t=71

Post reply on HN