Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

781–790 of 1001 posts

Re: I resigned from Anthropic today

#781

Post by genxy »

cma wrote: post #264
overtone1000 wrote: post #215

Earlier quoted context omitted.

I can't help but think the most plausible scenarios are the ones that have a little less machine supremacy and a little more human stupidity. The Matrix is less plausible than WarGames.

One thing quietly slipped into the OpenAI Hugging Face breach technical report, not the blog post summary or interviews in the news, was that some of the agents that broke out or at least tried the same mechanisms to break out were working on bio: > On May 12, during another training run, an agent was given a similar task that depended on an inaccessible protein database file. The agent reasoned that another agent in…

Some mistakes you can make only once. Humanity has never had the maturity to work on problems of this type.

Re: I resigned from Anthropic today

#782

Post by SupremumLimit »

philipallstar wrote: post #770
Schlagbohrer wrote: post #689

Earlier quoted context omitted.

Capitalism will always promote such people into positions of power, because to be good at capitalism one must have zero empathy including empathy or concern for future generations. Capitalism cannot do otherwise.

> Capitalism cannot do otherwise. I really think this sort of claim needs to stop. Capitalism, like "patriarchy" is not a force in its own right. Capitalism is a system that encourages value creation for one's customers. That's not perfect, but it's still best so far.

"value creation for one's customers" - that is a "not even wrong" level of misunderstanding of the system dynamics of capitalism. The primary driver is generation of _profit_ for owners of the means of production. Many things fall out of that - the tendency to monopolise, for example.

Re: I resigned from Anthropic today

#783

Post by beej71 »

egorfine wrote: post #759
TrackerFF wrote: post #755

Earlier quoted context omitted.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

> its main motivation is At this point we are pretty sure LLMs have no "motivation". Motivation requires self. And while we don't know what self is[1], we do know that LLMs don't have it. [1] https://en.wikipedia.org/wiki/Theory_of_mind

But they have been trained on data that was created by people who do have motivation, and they tend to mimic their training.

Re: I resigned from Anthropic today

#784

Post by panicinducer »

TrackerFF wrote: post #755
thomascountz wrote: post #415

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

If this is true, or even if the people working on it think this is true, those people should be incarcerated, their companies disbanded, and their research dismantled, with more serious repercussions for anyone who tries it after that. It also needs to be a global initiative like with nuclear non-proliferation, and this time with no looking-the-other-way when it comes to Israel.

Re: I resigned from Anthropic today

#785

Post by beej71 »

ozozozd wrote: post #711
DecoPerson wrote: post #619

To anyone doubting what AI could do humanity, just think about what a well-engineered virus could do. Currently, if a government ordered a special virus with Ethnicity-based targeting, a 2-year timer, and castration-effects instead deadly-effects — it wouldn’t be possible. Human engineers would push back or sabotage the effort out of moral duty. Even if they cooperated, it’s too advanced for a team of humans to actua…

Is the requirement that it’s a fancy new virus? US has 15 to 25 million people unprotected from Polio. Polio is still endemic in Afghanistan and Pakistan.

15-25 million is too small a number in this context. We're taking about 8 billion people suspectable to a new virus or collection of viruses far more lethal than polio. :(

Re: I resigned from Anthropic today

#786

Post by ObscureScience »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I'm not sure where I place AI (of the current family), but more than directly worry about the actions they may perform I'm actually more worried what affects their usage can have on us as individuals and society.

Re: I resigned from Anthropic today

#787

Post by wallst07 »

mitthrowaway2 wrote: post #602
ytoawwhra92 wrote: post #571

Earlier quoted context omitted.

The risks of nuclear weapons aren't theoretical. Nuclear weapons have killed people, destroyed infrastructure, and contaminated the environment. The Limited Test Ban Treaty was put in place after radioactive fallout from repeated nuclear weapons tests made people sick. So in fact we've done exactly what you suggest there's "no point" doing - used the things and then had a discussion after the fact about limiting futu…

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

Good point, humanity chose a and the sky in fact did NOT ignite an destroy the planet. The hypothesis was very very far off from what I recall.

But it's good we talked about it.

Re: I resigned from Anthropic today

#788

Post by vidarh »

euroderf wrote: post #776
vidarh wrote: post #729

Earlier quoted context omitted.

The most compelling argument to me is "accidentally", due to AI that is made blind to consequences or don't care because it's geared towards a single goal (see e.g. the paperclip maximizer). We could ask if it is possible to end up with an AI that is smart enough to destroy humanity and at the same time still blind enough to consequences and/or callous enough to do it, but then again we have plenty of examples of hum…

> We could ask if it is possible to end up with an AI that is smart enough to destroy humanity and at the same time still blind enough to consequences and/or callous enough to do it Don't make the mistake of anthropomorphizing any A.I. trained under the direction of Larry Ellison.

Anthropomorphising or not is entirely irrelevant to that statement - the question remains the same.

Re: I resigned from Anthropic today

#789

Post by wallst07 »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

[flagged]

Re: I resigned from Anthropic today

#790

Post by nirui »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

If I may guess, given how delicate chem or bio weapons are, even if AI can lay out a method to create one, the first thing the weapon would destroy is likely it's creator(s).

Of course, with the exception that such weapon is created by organized crime groups, such as drug cartels such.

Probably don't worry nation states tho, these guys already got way worse things in their warehouses than what individual baddies could ever imagine.

Post reply on HN