Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

411–420 of 931 posts

Re: I resigned from Anthropic today

#411

Post by matsemann »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Non-example 3 feels like a straw man. This is a force behind possibly a huge change to society, and you dismiss it offhandedly with "don't think it will be widespread starvation".

For instance have you seen what this has done to the school system? We're not equipped or ready to handle the changes. Consequences are unknown.

Re: I resigned from Anthropic today

#412

Post by Scrapemist »

asdff wrote: post #397
Scrapemist wrote: post #291

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of the other way around. That this automatically leads to extinction and apocalypse I don’t understand.

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

It might consider us a very efficient processing unit. Useful to hand over simple tasks with correct alignment and some oversight.

Re: I resigned from Anthropic today

#413

Post by henryaj »

esperent wrote: post #383
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

> After the events of the summer What events are you talking about?

Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild

Re: I resigned from Anthropic today

#414

Post by cameldrv »

doix wrote: post #313
cameldrv wrote: post #293

Earlier quoted context omitted.

You’re just a bunch of molecules following the laws of physics. It’s all just physics and chemistry, and those are well understood. Now explain the causes of World War I using chemistry and physics. Simple, right?

> It’s all just physics and chemistry, and those are well understood. Not really. We cannot model physics and chemistry to a level which allows us to accurately predict a humans action (even a tiny time-step into the future) This is vastly different to an LLM, where the model is the model (for a lack of better phrasing).

We can model physics and chemistry pretty well, just not beyond small scales, because the computational effort blows up.

You could just as easily say that if you can write a python interpreter that you can understand every program written in python. Ok, now what if the program is two terabytes?

A frontier LLM is nothing but a 2 terabyte program written in a weird programming language. Just because you can understand the interpreter does not mean you understand the program in a meaningful way.

Re: I resigned from Anthropic today

#415

Post by keithnz »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Its speculation on whether it is truly dangerous. I think approaching it with "what's the most dangerous thing that's happened?" while possibly interesting in terms of pending danger, it says nothing about potential cliff edge danger. I don't think we can quantify the danger, it's not out of the question there is cliff like danger in creating self improving super intelligence. Some peoples danger senses are going to be based on concrete observed threats, others are going to worried about potential hypotheticals that seem plausible. I'm mostly skeptical of the danger but I do think the impact of AI is going to change things a lot. But much like climate change, economics is going to guide what we actually do.

Re: I resigned from Anthropic today

#416

Post by vlyan »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

>After the events of the summer

After the blatant marketing campaigns of the summer, you mean. do you need a reminder that those very same people had touted GPT-2 as a dangerous model?

worrying about sci-fi doomsday scenarios with the current AI tech is absurd. LLMs predict the next token, that's literally all they do. they aren't going to escape into the cyberspace, self-replicate, self-improve, jump over air gaps and launch the nukes at John Connor's grandma. they can't. people pretend to believe the dumbest shit.

Re: I resigned from Anthropic today

#417

Post by thomascountz » 

   No other human activity poses this level of danger.
I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc.

Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Re: I resigned from Anthropic today

#418

Post by underdeserver »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Climate doomers: Climate change will kill us all by the end of the century!

AI doomers: End of the century? Hold my beer.

Re: I resigned from Anthropic today

#419

Post by krautsauer »

csomar wrote: post #395
altcognito wrote: post #374

Earlier quoted context omitted.

> Non-example 2: There are worries about AI-facilitated biological weapons. I haven’t seen any evidence that’s happening. I think this is a good example of poor risk management reasoning. there is evidence bioengineering is already happening. No, nobody is going to announce when somebody has decided to use these tools (even if isn’t an LLM) to bioengineer a weapon. Are the tools power enough to do so? Not sure. But I…

> there is evidence bioengineering is already happening Mind sharing this evidence with us?

Does https://news.ycombinator.com/item?id=30698803 count?

Re: I resigned from Anthropic today

#420

Post by Negitivefrags »

skew-aberration wrote: post #256
huitzitziltzin wrote: post #177

Earlier quoted context omitted.

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

Past changes to economic relationships haven't replaced labor either, yet they have led to conflict and starvation. You are setting an incredibly high bar here, essentially a strawman. If people feel disenfranchised due to their diminishing political and economic power, there will be enormous potential for conflict. This is a pattern across history and central to all the economics I've ever read. As an economist, do…

I suspect that the conditions the textile workers lived in when western countries were creating textiles is not so different in absolute terms from the condition that they live in now.

It's just that the western world moved on.

Almost all economies that have developed have started with textiles. This is the starting point on the ladder. Eventually, we will run out of poor countries that haven't had a textile industry yet, and at that point, either it will be entirely automated, or we will no longer have cheap textiles!

But I'm betting on the automation.

Post reply on HN