Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: I resigned from Anthropic today

#971

Post by LaffertyDev »

LordDragonfang wrote: post #970
LaffertyDev wrote: post #889

Earlier quoted context omitted.

> There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours > If you disagree with it, where do you disagree with it? How would a malicious actor: 1. Successfully turn these 20k pathogens into actual reproducing viruses / bacteria / etc (in a lab) 2. Successfully turn that lab prototype into something weaponized (i.e. as a bio-terrori…

1 and 2 have had simple, proof-of-concept answers for more than a decade now: > 1. Email sets of DNA strings to one or more online laboratories which offer DNA synthesis, peptide sequencing, and FedEx delivery. (Many labs currently offer this service, and some boast of 72-hour turnaround times.) > 2. Find at least one human connected to the Internet who can be paid, blackmailed, or fooled by the right background story,…

For #1, I would seek a laboratory capable of _actually_ creating a _live virus_. That link to lesswrong, while interesting, does not actually indicate how and broadly reads as incredibly speculative.

While this is far out of my wheelhouse, I am not aware of a commercial venture doing actual organic creation or manipulation of material (I.e. capable of creating a modified strain of COVID). This broadly still seems in the nation-state level of lab.

But, just to get this goalpost out of the way, even _if_ such a venture exists, then would it not hold a bunch of doomsday cultists would already try to do this? This is what I’m getting at by my third question. What changes? Why don’t we see modified anthrax attacks in Palestine or Ukraine _today_?

I take no contention with the social engineering argument. I fully accept that LLMs, today and for a while yet, are capable of social engineering their way into anything. I likewise take no contention with the sheer amount of effort LLMs could wield in pursuit of this.

Re: I resigned from Anthropic today

#972

Post by jeremyjh »

frabcus wrote: post #645
henryaj wrote: post #408

Earlier quoted context omitted.

Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild

Also (and under-reported, so you could easily have missed it) OpenAI's agents got access to K8 admin on their own research cluster. "This escalation also yielded access to OpenAI’s managed cloud Kubernetes service. The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod" https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78... (see section V)

And we really only have OpenAI’s word for it that they had no access to their own weights there and didn’t exfiltrate them. No one who knows that incident could suggest it was beyond its capabilities to do that. And we would have never heard about any of this if it wasn’t investigated by an external party (hugging face). It may have happened elsewhere already. It’s not likely to have, but it is very possible this was our last “free” warning.

Re: I resigned from Anthropic today

#973

Post by ethin »

senordevnyc wrote: post #958
ethin wrote: post #931

Earlier quoted context omitted.

> ... They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And yet they do the exact opposite of all of the above. They do everything to get regulation just to create a moat because one doesn't exist. They preach about wanting a slowdown, nationalization, a complete pause, whatever have you... And yet they aren't slowing down the development voluntarily n…

It's genuinely hilarious to me that you immediately proved my second paragraph correct. Putting arms race in quotes doesn't make it any less real. It just means you wish it wasn't, but don't have any good evidence or arguments to make.

I mean no, not really... Your the one making the claims that these AI companies really, really want regulation, nationalization, a slowdown, or whatever, and that they genuinely care about AI safety and it not killing us all. It's up to you to prove it, not for me to just believe you implicitly. So, please, provide the evidence, we're all very curious to see it.

Re: I resigned from Anthropic today

#974

Post by mlyle »

Obscurity4340 wrote: post #962
mlyle wrote: post #437

Earlier quoted context omitted.

Believe me, without the internet we can still teach. We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play. And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school. And students will take a few days to adjust to writing down homework in their planners again…

> We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play There are offline wireless hdmi adapters, this is a solved problem. Just have to have the videos pre downloaded or offline available

Sure. 99% of teachers don't know yt-dlp or use things like that, though, so you can expect minor disruptions in a lot of rooms.

But, c'mon, we have complete power outages and still address our curriculum.

Re: I resigned from Anthropic today

#977

Post by dwaltrip »

skeledrew wrote: post #966
dwaltrip wrote: post #936

Earlier quoted context omitted.

This isn’t true. It only has no impact if there is an infinite supply of these “immoral” or unaware people, which obviously isn’t the case. You also need to consider 2nd order effects and beyond.

There doesn't have to be an infinite supply, just a significantly greater portion, which there is.

Still had an impact, then. I’m not sure what point you are trying to make with all of this… Have a good day.

Re: I resigned from Anthropic today

#978

Post by gizmodo59 »

frabcus wrote: post #665
gizmodo59 wrote: post #497

Earlier quoted context omitted.

This is a lame argument. Questioning his financial incentive is very legit. Why should we trust him?

Questioning his finances is a poor argument because he clearly would have more money if he had stayed at Anthropic for the next couple of years, than he will have by leaving. Even if he still has vested options, or savings from his salary or whatever, those would be larger if he stayed.

The fame he got, the stamp of a frontier lab in the resume - its worth more than working in many other companies for years. Plus, not everyone thinks working more is needed once they achieve a certain number.

Re: I resigned from Anthropic today

#979

Post by rahulyc »

leokennis wrote: post #641

Every news headline or public statement these days is a gut punch. Only bad news, and nothing we (as "the general public") can do about it. Take this one. Ok, AI is going to ruin us all. But let's say we do our civic duty: we protest, vote in candidates with good views on AI etc. and somehow convince or regulate OpenAI and Anthropic into stopping their arms race... Then what about China? It would be a great opportuni…

The Chinese are humans too. Why would they want to be extinct either?

Re: I resigned from Anthropic today

#980

Post by uselessTA »

leokennis wrote: post #641

Every news headline or public statement these days is a gut punch. Only bad news, and nothing we (as "the general public") can do about it. Take this one. Ok, AI is going to ruin us all. But let's say we do our civic duty: we protest, vote in candidates with good views on AI etc. and somehow convince or regulate OpenAI and Anthropic into stopping their arms race... Then what about China? It would be a great opportuni…

If our government cared enough to regulate, they could also directly negotiate with China. There are proposed agreements that don't require either party to trust each other, and even if China doesn't want out of this race, we could pressure them other things like trade.

Examples: FlexHEG from Bengio (proposes on-chip mechanisms that allow workload verification without trust), large bilateral investments into joint AI interpretability or alignment efforts, or invasive audits & inspections (data centers for training these models have a large footprint + this can combine with on-chip mechanisms since producing chips is even more complicated).

And there are probably better proposals available for people to find, if we actually prioritized this.

Post reply on HN