Live data from Hacker News

Did Anthropic ask for this?

verysane.ai

131–140 of 176 posts

Re: Did Anthropic ask for this?

#131

Earlier quoted context omitted.

I was going to say. "Doomsday Device Company" is a wildly loaded description coming from an allegedly "neutral" party.

It is, but… you’ve seen the marketing material being put out? “AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.

Sure, but the same is true of nuclear reaction research.

One camp, using the same mechanism, is trying to make devices that end the world. One camp, using the same mechanism, is trying to save the world.

That nuclear/AI COULD end the world or COULD save humanity is exactly the reason that a truly neutral party would not describe either tech as a doomsday device but would instead describe either as technologies bearing massive potential.

No one building an LLM has ever publically stated that they aim to build a doomsday device and in fact have only ever specifically said that they are trying to avoid doing so. We've no reason to beleive that their interest is any different from any other corporation with agressive shareholders: increasing profits. Dooming humanity leads to lower profits. Lower profits is outside shareholder interest. They will do anything they can to maximize profit, just as any other profit driven org does.

No objective, neutral, observer would apply such loaded terms.

It's okay to have an opinion, but having an opinion makes you...opinionated. Not neutral. The opposite of neutral.

Re: Did Anthropic ask for this?

#132

Earlier quoted context omitted.

Ok, but that only benefits them. If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others. But of course they don't actually believe that. It is just marketing hype. You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices".…

@gork what is a coordination problem and can you just like "shut down your participation" and solve them?

Yes! As long as everyone else does too.

Re: Did Anthropic ask for this?

#133
post #66

Earlier quoted context omitted.

OpenAI is happy to sell their device to the gov’t to blow things up with, Anthropic tried to tell the gov’t to pound sand, no blowing things up for you.

To be clear, Anthropic was completely okay with their tech being used in war, including current conflicts. They just wanted a human making some decisions. The military didn’t want a vendor giving them restrictions on how to operate a tool, and since Anthropic’s restrictions could be a problem anywhere they appear among other military vendors as well, the government used the supply chain risk designation to say “this…

After a contract had been negotiated with the exact terms that Anthropic pushed to keep in place.

Re: Did Anthropic ask for this?

#134

Anybody who isn't at least treating this situation as possibly just an authoritarian government picking winners and losers is not paying attention to the political environment. Companies/countries/people are paying off the government in all sorts of various ways (crypto, gifts, bogus settlements, planes, inaugurations, ballrooms). The companies that pay off the government get big fat contracts and merger agreements,…

Ah but you have forgotten that HN is full of people who are so aloof and "above the fray" (compliments to their raw intellect, of course) that it would be beneath them to consider the realities of the political situation.

I just think that it's dodging the question in this case. It's true that the US government is run by crooks who'd all be serving prison sentences in a just world, and that they'd certainly do whatever Anthropic wants in exchange for a sufficiently large bribe. But the US government also serves a number of important roles in American society that we can't simply turn off and come back to in 2029.

Re: Did Anthropic ask for this?

#135

I posted this comment on the other thread, but it deserves mention here too, because Anthropic also asked for this ~10 days ago, separately from the post linked in the article. https://www.anthropic.com/institute/recursive-self-improveme ... > We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep u…

the link appears to be broken. Only shows a 404 poem :(

[deleted]

Re: Did Anthropic ask for this?

#136

Half of me wonders if it was all a live simulation/drill, to practice what happens if a much more serious event occurs, and a model needed to be quickly shut down. Under such conditions we would be looking at Amazon's actions through a much more benevolent lens. Not saying it has been, but it certainly crossed my mind as something worth doing regardless.

Even if it's not intended to be a drill, this dynamic is why I can't take Anthropic's side here. If you believe what they claim to believe about the future of AI, which I at least do, there are going to be future cases where an emergency block is required for much more real threats than this one. Why in the world would Amodei think that the model provider, with massive financial incentives to keep the money flowing, should be allowed to make the final shutdown call?

If a team of researchers at Netflix discovers a few days after release that GPT 6.0 has a safety-compromising jailbreak, I want Sam Altman initiating a shutdown the second he gets that call, and I had thought until Friday that Anthropic agreed with me.

Re: Did Anthropic ask for this?

#137

Earlier quoted context omitted.

I was going to say. "Doomsday Device Company" is a wildly loaded description coming from an allegedly "neutral" party.

It is, but… you’ve seen the marketing material being put out? “AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.

[deleted]

Re: Did Anthropic ask for this?

#138
post #122

Earlier quoted context omitted.

It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO? I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emoti…

"Are AIs conscious?" is not currently answerable because "Are humans conscious?" is not currently answerable. I know I'm conscious because I perceive (it) directly. Everyone else's consciousness basically depends on an assumption; on taking them at their word when they say they're conscious. Now I ask an AI if it's conscious and it says it doesn't know, but it sorta thinks it might be. Okay, it's probably not conscio…

I agree with this, but I don't think Anthropic would, they appear to be much more convinced than what I think would be reasonable.

Re: Did Anthropic ask for this?

#139

Earlier quoted context omitted.

It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO? I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emoti…

Suggesting that the probability is non-zero is not the same thing as suggesting it as fact.

You mean for AI consciousness or existential risk? I think Anthropic downplay the existential risk (they might talk about it now and then, but they still build frontier models) but are overly confident about AI being conscious, and I think these two things are pretty strongly related to each other.

Re: Did Anthropic ask for this?

#140
post #4

Earlier quoted context omitted.

they've been offering refunds for people specifically because of the Fable situation

Isn’t that just because people started doing chargebacks, and refunds are cheaper than chargebacks?

good intentions or not, they are letting people have their money back - not that this would put a dent in current enterprise spend anyways
Post reply on HN