Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Anthropic ask for this?

verysane.ai

Post reply on HN
Print view View original

141–150 of 176 posts

Re: Did Anthropic ask for this?

#141

Post by saurik »

goodmythical wrote: post #131
danielheath wrote: post #106

Earlier quoted context omitted.

It is, but… you’ve seen the marketing material being put out? “AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.

Sure, but the same is true of nuclear reaction research. One camp, using the same mechanism, is trying to make devices that end the world. One camp, using the same mechanism, is trying to save the world. That nuclear/AI COULD end the world or COULD save humanity is exactly the reason that a truly neutral party would not describe either tech as a doomsday device but would instead describe either as technologies bearin…

Your example kind of proves your position wrong, no? Nuclear power--another device you are yourself admitting is widely understood to be a doomsday device in the wrong hands--is also highly regulated and I cannot just create a company that sells a nuclear power device and I extra cannot do so and sell it to foreign nationals. If you make a device that you claim is capable of ending the world, you can and should be regulated.

Re: Did Anthropic ask for this?

#142

Post by pdimitar »

SpicyLemonZest wrote: post #134
estearum wrote: post #125

Earlier quoted context omitted.

Ah but you have forgotten that HN is full of people who are so aloof and "above the fray" (compliments to their raw intellect, of course) that it would be beneath them to consider the realities of the political situation.

I just think that it's dodging the question in this case. It's true that the US government is run by crooks who'd all be serving prison sentences in a just world, and that they'd certainly do whatever Anthropic wants in exchange for a sufficiently large bribe. But the US government also serves a number of important roles in American society that we can't simply turn off and come back to in 2029.

Your last sentence sounds reasonable. Can you give examples?

Re: Did Anthropic ask for this?

#143

Post by airstrike »

ianm218 wrote: post #112
airstrike wrote: post #58

Earlier quoted context omitted.

We do not know for a fact what they genuinely believe, and many of us have seen companies act in opposition to their stated goals so the burden of proof is on them. It's healthy to suspect ulterior motives from them.

Can you be more specific on what you mean - how would you prove what anyone believes about something short of reading minds and what would proof look like to you? Here is Dario writing about AI safety in 2016 [1]. Dario and others in the Anthropic circle have long been associated [2] with the effective altruism movement, which whatever you think about them, they are very concerned about AI existential risks. Ronan Fa…

It's not about proving people are honest. It's about knowing that people either die a hero or live long enough to become the villain. Not because they want to, but because it's a natural journey given this competitive game of corporations.

OpenAI and Sam Altman being dishonest (or not) really has bearing on Anthropic. It's wholly unnecessary to argue about those other parties when discussing Anthropic or companies in general. Put simply, Alice being dishonest doesn't make Bob honest. But interestingly enough, OpenAI at first was also regarded as uncompromising stewards of "AI for everyone". It's in their name!

Speaking specifically about Anthropic, being concerned with AI existential risks is incredibly self-serving because guess who owns the very powerful AI that needs to be treated carefully and not allowed into the hands of other countries or competitors? All under the guise of being for the people. Even if a hundred people within the organization are doing this out of the goodness of their hearts, the company is not actually bound to that behavior and can renege on it at any point in time—just like OpenAI did.

Besides, Anthropic cofounders left OpenAI effectively forever ago. Just after GPT-2, which was pretty useless still. It was a whole different company. And the young are often naive but that naiveté eventually fades. Investors make demands, business realities sink in, and founders compromise.

This has happened time and time again. Or did you forget how Google Chrome started? Or how the shading and labeling of ads on Google has evolved over time? Or that "Do no evil" has been dropped?

The thing about my argument is that it doesn't require a purposeful nefarious motive. Corporations exist to maximize profits. When they act perfectly according top their stated goals, they make some decisions that are not in the interest of everyone else. It's important to remember that.

Re: Did Anthropic ask for this?

#144

Post by adahn »

ivraatiems wrote: post #11

Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would. Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday dev…

When I read this comment I don't get "Neutral Party" from it. I'm finding it hard to decide whether to focus on the loaded characterisations or (my best guess at) the underlying substance.

To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them.

>As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1]

But with respect to your specific post:

>high-minded philosophical nonsense

Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter.

You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states.

>Suppose a company calls themselves The Doomsday Device Company

Anthropic hasn't. What's the explanatory purpose of adding this to the analogy?

>They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.

Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology.

>The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!"

They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do.

1: https://www.anthropic.com/news/fable-mythos-access

Re: Did Anthropic ask for this?

#146

Post by ianm218 »

airstrike wrote: post #143
ianm218 wrote: post #112

Earlier quoted context omitted.

Can you be more specific on what you mean - how would you prove what anyone believes about something short of reading minds and what would proof look like to you? Here is Dario writing about AI safety in 2016 [1]. Dario and others in the Anthropic circle have long been associated [2] with the effective altruism movement, which whatever you think about them, they are very concerned about AI existential risks. Ronan Fa…

It's not about proving people are honest. It's about knowing that people either die a hero or live long enough to become the villain. Not because they want to, but because it's a natural journey given this competitive game of corporations. OpenAI and Sam Altman being dishonest (or not) really has bearing on Anthropic. It's wholly unnecessary to argue about those other parties when discussing Anthropic or companies in…

> so the burden of proof is on them

I was mainly responding to this I know what you mean.

The reason I bring up OpenAI is because a lot of the discourse has boiled down to "both sides are the same" which I don't think is fair.

Re: Did Anthropic ask for this?

#147

Post by SilverElfin »

ajmurmann wrote: post #133
SilverElfin wrote: post #100

Earlier quoted context omitted.

To be clear, Anthropic was completely okay with their tech being used in war, including current conflicts. They just wanted a human making some decisions. The military didn’t want a vendor giving them restrictions on how to operate a tool, and since Anthropic’s restrictions could be a problem anywhere they appear among other military vendors as well, the government used the supply chain risk designation to say “this…

After a contract had been negotiated with the exact terms that Anthropic pushed to keep in place.

We don't have the actual contracts publicly. So I bet it has terms that let the government do what it wants ultimately by naming exceptions. That way OpenAI can claim it has some controls in place while the military has the freedom it needs practically.

Re: Did Anthropic ask for this?

#148

Post by threatofrain »

gerdesj wrote: post #92
sillysaurusx wrote: post #37

Earlier quoted context omitted.

My eighty year old father brought up OpenAI unprompted a few months ago. At this point it’s hard to find anyone who hasn’t heard of OpenAI.

I think my point still stands 8) I know plenty of people who will put the letters A and I together and get rather confused about what on earth is going on but very few of those would mention any co apart from Microsoft, Google, Facebook or errm Twitter. No one has a bloody clue about all this stuff apart from us lot and we have no real idea about it either! Oooh tulips!

Is your point that the name people actually remember is ChatGPT? Okay, let's say the actually popular name that threatens to become the Kleenex of LLMs. ChatGPT.

Re: Did Anthropic ask for this?

#149

Post by NickNaraghi »

ianm218 wrote: post #15

What I feel like is missing in the common discourse here is that Anthropic genuinely believes that AI poses an existential risk for humanity either in terms of literal survival or extreme mass surveillance, human disempowerment etc. So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do? I.e. OpenAI just went full evil corpo mode and went all in on the…

Super interesting. How does regulation affect IPO prospects of OAI and ANTH? What’s the timing of each?

Re: Did Anthropic ask for this?

#150

Post by bbippin »

matheusmoreira wrote: post #95
bbippin wrote: post #79

Earlier quoted context omitted.

I’m surprised people are reacting so strongly over this. Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks? I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do. It’s very capable and equally aggressive in accomplishing its goals.

> would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks? Yes, because then we get to use SOTA models to defend against the exact same attacks. Fable detected issues in my projects but got downgraded back to Opus before it could tell me about them or fix them. In what world could that possibly be reasonable?

I agree and don’t think they should have been forced to pull it, but what I’m surprised about is how many people are accusing Anthropic of being liars or trying to overhype the danger. It IS dangerous, which is why it can identify vulnerabilities.
Post reply on HN