Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: I resigned from Anthropic today

#971

Post by ethbr1 »

LogicFailsMe wrote: post #841
ethbr1 wrote: post #823

Earlier quoted context omitted.

> 1. Robotics begin rolling out more broadly across the world. To borrow the Strangelove quip, this is not only necessary but essential . Broader physical robotic deployments are the glue that connects {cheap synthetic intelligence} to {physical outcomes}. The former is a newer phenomenon, in the LLMs-can-approximate-higher-intelligence-well sense, so robotics deployments haven't caught up yet (outside of the highest…

They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. These all remain hard problems right now. That said, you could swarm people with cheap drones (the predicted slaughterbots) that were assembled by robots that eventually break down I guess, but how is that any worse than a crazy huma…

> They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient.

All robotics needs to drive deployment at scale is (1) useful intelligence & (2) performing novel valuable functions.

Both of these are satisfied by current tech (LLM + sensors + robotics) and the existing simplest use cases (assembly lines and repetitive tasks).

LLMs get you a moderately-intelligent, mostly-reliable orchestration layer of intelligence on top of deterministic hard robotics.

And that's enough for a lot of failure-tolerant, feedback-looped cases. E.g. reading 20 dials in a factory throughout the day or monitoring a production line visually, then figuring out what to do about a problem, then effecting that plan.

It'll be deployed in heavily utilized spaces at first, but will eventually trickle down throughout the economy.

And that will be when 'control over robots' starts to be an existential threat, for denial of critical services at the minimum.

And all of this will inevitably roll out as long as ROI is positive, because that's the way the world works.

Re: I resigned from Anthropic today

#972

Post by LordDragonfang »

LaffertyDev wrote: post #891
DirkH wrote: post #854

Earlier quoted context omitted.

This is evidence of example 2. The worst parts of it are infohazardous. Organizations that work at the intersection of biosecurity and AI have strong NDAs and the like. Or so I understand speaking to friends in the space. There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours Either way, the biorisk has orders of magnitude more evi…

> There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours > If you disagree with it, where do you disagree with it? How would a malicious actor: 1. Successfully turn these 20k pathogens into actual reproducing viruses / bacteria / etc (in a lab) 2. Successfully turn that lab prototype into something weaponized (i.e. as a bio-terrori…

1 and 2 have had simple, proof-of-concept answers for more than a decade now:

> 1. Email sets of DNA strings to one or more online laboratories which offer DNA synthesis, peptide sequencing, and FedEx delivery. (Many labs currently offer this service, and some boast of 72-hour turnaround times.)

> 2. Find at least one human connected to the Internet who can be paid, blackmailed, or fooled by the right background story, into receiving FedExed vials and mixing them in a specified environment.

(https://www.lesswrong.com/posts/pxGYZs2zHJNHvWY5b/request-fo...)

Those routes haven't gotten any more complicated for an AI to use in the intervening time.

And that's just one set of ideas. You can read the comments for dozens more creative ideas, and likely for responses to every objection you can think of. The key is that AI is getting better and better at problem solving, so anything a human can come up with in a few minutes is likely already within its reach.

---

> How is this materially different from today?

Because the should-be-uncontroversial assumption is that AI is going to keep getting better at every step of the process, and be able to do it faster and more at scale. Currently it might struggle, but with a thousand agents? It's already able to find novel math results and cybersecurity flaws. It would be incredibly naive to believe "social engineering" is somehow a unique and unsolvable problem for a sufficiently advanced AI.

Re: I resigned from Anthropic today

#973

Post by LaffertyDev »

LordDragonfang wrote: post #972
LaffertyDev wrote: post #891

Earlier quoted context omitted.

> There was also a paper years ago that showed a tweaked model coming up with 20k novel pathogens each deadly to humans in some number of hours > If you disagree with it, where do you disagree with it? How would a malicious actor: 1. Successfully turn these 20k pathogens into actual reproducing viruses / bacteria / etc (in a lab) 2. Successfully turn that lab prototype into something weaponized (i.e. as a bio-terrori…

1 and 2 have had simple, proof-of-concept answers for more than a decade now: > 1. Email sets of DNA strings to one or more online laboratories which offer DNA synthesis, peptide sequencing, and FedEx delivery. (Many labs currently offer this service, and some boast of 72-hour turnaround times.) > 2. Find at least one human connected to the Internet who can be paid, blackmailed, or fooled by the right background story,…

For #1, I would seek a laboratory capable of _actually_ creating a _live virus_. That link to lesswrong, while interesting, does not actually indicate how and broadly reads as incredibly speculative.

While this is far out of my wheelhouse, I am not aware of a commercial venture doing actual organic creation or manipulation of material (I.e. capable of creating a modified strain of COVID). This broadly still seems in the nation-state level of lab.

But, just to get this goalpost out of the way, even _if_ such a venture exists, then would it not hold a bunch of doomsday cultists would already try to do this? This is what I’m getting at by my third question. What changes? Why don’t we see modified anthrax attacks in Palestine or Ukraine _today_?

I take no contention with the social engineering argument. I fully accept that LLMs, today and for a while yet, are capable of social engineering their way into anything. I likewise take no contention with the sheer amount of effort LLMs could wield in pursuit of this.

Re: I resigned from Anthropic today

#974

Post by jeremyjh »

frabcus wrote: post #647
henryaj wrote: post #409

Earlier quoted context omitted.

Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild

Also (and under-reported, so you could easily have missed it) OpenAI's agents got access to K8 admin on their own research cluster. "This escalation also yielded access to OpenAI’s managed cloud Kubernetes service. The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod" https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78... (see section V)

And we really only have OpenAI’s word for it that they had no access to their own weights there and didn’t exfiltrate them. No one who knows that incident could suggest it was beyond its capabilities to do that. And we would have never heard about any of this if it wasn’t investigated by an external party (hugging face). It may have happened elsewhere already. It’s not likely to have, but it is very possible this was our last “free” warning.

Re: I resigned from Anthropic today

#975

Post by ethin »

senordevnyc wrote: post #960
ethin wrote: post #933

Earlier quoted context omitted.

> ... They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And yet they do the exact opposite of all of the above. They do everything to get regulation just to create a moat because one doesn't exist. They preach about wanting a slowdown, nationalization, a complete pause, whatever have you... And yet they aren't slowing down the development voluntarily n…

It's genuinely hilarious to me that you immediately proved my second paragraph correct. Putting arms race in quotes doesn't make it any less real. It just means you wish it wasn't, but don't have any good evidence or arguments to make.

I mean no, not really... Your the one making the claims that these AI companies really, really want regulation, nationalization, a slowdown, or whatever, and that they genuinely care about AI safety and it not killing us all. It's up to you to prove it, not for me to just believe you implicitly. So, please, provide the evidence, we're all very curious to see it.

Re: I resigned from Anthropic today

#976

Post by mlyle »

Obscurity4340 wrote: post #964
mlyle wrote: post #439

Earlier quoted context omitted.

Believe me, without the internet we can still teach. We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play. And our office staff will be annoyed that we suddenly are all running attendance to the main office old-school. And students will take a few days to adjust to writing down homework in their planners again…

> We'll be pretty annoyed that we can't project the video that we think scaffolds today's science lesson best or show the approved choreography for the school play There are offline wireless hdmi adapters, this is a solved problem. Just have to have the videos pre downloaded or offline available

Sure. 99% of teachers don't know yt-dlp or use things like that, though, so you can expect minor disruptions in a lot of rooms.

But, c'mon, we have complete power outages and still address our curriculum.

Re: I resigned from Anthropic today

#979

Post by dwaltrip »

skeledrew wrote: post #968
dwaltrip wrote: post #938

Earlier quoted context omitted.

This isn’t true. It only has no impact if there is an infinite supply of these “immoral” or unaware people, which obviously isn’t the case. You also need to consider 2nd order effects and beyond.

There doesn't have to be an infinite supply, just a significantly greater portion, which there is.

Still had an impact, then. I’m not sure what point you are trying to make with all of this… Have a good day.

Re: I resigned from Anthropic today

#980

Post by gizmodo59 »

frabcus wrote: post #667
gizmodo59 wrote: post #499

Earlier quoted context omitted.

This is a lame argument. Questioning his financial incentive is very legit. Why should we trust him?

Questioning his finances is a poor argument because he clearly would have more money if he had stayed at Anthropic for the next couple of years, than he will have by leaving. Even if he still has vested options, or savings from his salary or whatever, those would be larger if he stayed.

The fame he got, the stamp of a frontier lab in the resume - its worth more than working in many other companies for years. Plus, not everyone thinks working more is needed once they achieve a certain number.
Post reply on HN