Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

641–650 of 716 posts

Re: Anthropic drops flagship safety pledge

#641

Post by ACCount37 »

kilpikaarna wrote: post #560
ACCount37 wrote: post #533

Earlier quoted context omitted.

At sufficient levels of intelligence, one can increasingly substitute it for the other things. Intelligence can be the difference between having to build 20 prototypes and building one that works first try, or having to run a series of 50 experiments and nailing it down with 5. The upper limit of human intelligence doesn't go high enough for something like "a man has designed an entire 5th gen fighter jet in his mind…

Exceedingly elaborate, internally-consistent mind constructs, untested against the real world, sounds like a good definition of schizophrenia. May or may not correlate with high intelligence.

We only call it "schizophrenia" when those constructs are utterly useless.

They don't have to be. When they aren't, sometimes we call it "mathematics".

You only have to "test against the real world" if you don't already know the outcome in advance. And you often don't. But you could have. You could have, with the right knowledge and methods, tested the entire thing internally and learned the real world outcome in advance, to an acceptable degree of precision.

We have the knowledge to build CFD models already. The same knowledge could be used to construct a CFD model in your own mind. We have a lot of scattered knowledge that could be used to make extremely elaborate and accurate internal world models to develop things in - if only, you know, your mind was capable of supporting such a thing. And it isn't! Skill issue?

Re: Anthropic drops flagship safety pledge

#642

Post by freejazz »

pera wrote: post #119
baq wrote: post #68

Earlier quoted context omitted.

Foundational model provider manifesto: ‘While there’s value in safety, we value the Pentagon’s dollars more’

It turns out the biggest threat to AI safety is capitalism, who would have thought

Anthropic is a public benefit corp

Re: Anthropic drops flagship safety pledge

#643

Post by ben_w »

slibhb wrote: post #639
ben_w wrote: post #620

Earlier quoted context omitted.

> has the potential to supercharge technology and productivity, which would massively benefit humanity The opportunities you chose to list are the greedy ones. > Yes there are risks, which have to be managed. How? As a reminder, we've known about the effect of burning coal on the climate for well over a century, we knew that said climate change would be socially and economically disasterous for half a century, yet th…

> The opportunities you chose to list are the greedy ones. Technology covers healthcare. I don't see how it's "greedy" to want to cure cancer. But on some level I guess "wanting life to be better" is greedy. Your attitude is very European, and it's basically why your continent is being left behind. I'm not totally against Europe becoming the world's retirement home, as long as there are places in the world where peop…

> Technology covers healthcare.

If you'd chosen to list that in the first place, I wouldn't have said what I did; "supercharge technology and productivity" is looking at everything through the lens of money and profit, not the lens of improving the human condition.

> Your attitude is very European, and it's basically why your continent is being left behind

And yours is very American. You talk about managing the risks, but the moment you see anyone doing so, you're against it.

And of course, Europe does have AI, both because keeping up is so much easier and cheaper than being bleeding edge on everything all the time, and of course, how DeepMind may be owned by Google but is a British thing.

Plus: https://mistral.ai

Also, to be blunt, China's almost certain to win any economic or literal arms race you think you're part of; they make too much critical hardware now.

> as long as there are places in the world where people are allowed to innovate.

I would like there to be a world.

When people worry about the end of the world, they usually don't mean to imply its physical disassembly. Sometimes people even respond as if speakers did mean that, saying things like "nukes or climate change wouldn't actually destroy the planet, it will still be here, spinning", as if this was the point.

AI is one of the few things that could, actually, literally, end up with the planet being physically disassembled. "All it needs" is solving the extremely hard challenges of a von Neumann replicator, and, well, solving hard problems is kinda the point of making AI in the first place.

Re: Anthropic drops flagship safety pledge

#644

Post by helloplanets »

IAmGraydon wrote: post #577
nazgulsenpai wrote: post #551

More and more I have just come to accept that the majority of people, at least those I am exposed to in the US, don't fundamentally believe in anything. Everyt conviction has a buyout price.

You have to understand that people only believe in things and have "morals" because it either helps them get what they want or makes them feel better about themselves. Of course such a thing has a buyout price. That's human nature. Capitalism just allows it to be on display in the worst way.

> get what they want or makes them feel better about themselves

So... all acts are selfish because if it looks unselfish, that just means it was selfish in a hidden way?

Re: Anthropic drops flagship safety pledge

#645

Post by noumenon1111 »

jMyles wrote: post #460

I pray that we can all get to the following simple standard: * AI and states cannot peacefully coexist, and AI is not going to be stopped. Therefore, we must begin to deprecate states. I think it's very unlikely that this is unrelated to the pressure from the US administration, as the anonymous-but-obvious-anthropic-spokesperson asserts. We're at a point now where the nation states are all totally separate creatures…

That's hilarious, and very sweet.

Humans are, by nature, forgetful and argumentative. Fourteen hundred years ago, the Qur'an said this unequivocally (20:115, 18:54, 22:8, 18:73). Not to moralize here, I'm just saying if camel-herders could build a medieval superpower out of nothing, they knew something we don't.

Any state or system that insists good humans are always nice, smart, cogent, and/or aware is doomed to fail. A Washington or a Cincinnatus that can get out of his own way (and that of society) is rare indeed, a one-in-a-billion soul. We shouldn't sit around and wait for that, while your run-of-the-mill dictator in a funny hat (or a funny toupée for that one orange fellow) has his way with us.

Re: Anthropic drops flagship safety pledge

#646

Post by ilumanty »

overgard wrote: post #605
keeda wrote: post #588

Earlier quoted context omitted.

I think he's right and we should be thinking about this a lot more. Even the IMF is worried about 40 - 60% of global employment : https://www.imf.org/en/blogs/articles/2024/01/14/ai-will-tra... Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he act…

> Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he actually said, he didn't really say AI will replace 90% of all coders, as people widely report, he said it will be able to write 90% of all code. Ugh, people here seem to think that all software…

Claude keeps getting SQLite's weird GROUP BY with MIN/MAX behavior completely wrong. Generally, complex SQL is not its strong side.

Re: Anthropic drops flagship safety pledge

#647

Post by nextaccountic »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

> The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter.

This sounds like a lie. But if they are telling the truth, that's a terrible timing nonetheless.

Re: Anthropic drops flagship safety pledge

#648

Post by nextaccountic »

freejazz wrote: post #642
pera wrote: post #119

Earlier quoted context omitted.

It turns out the biggest threat to AI safety is capitalism, who would have thought

Anthropic is a public benefit corp

And OpenAI was founded as a non profit, back in the time it was open

Re: Anthropic drops flagship safety pledge

#649

Post by freejazz »

nextaccountic wrote: post #648
freejazz wrote: post #642

Earlier quoted context omitted.

Anthropic is a public benefit corp

And OpenAI was founded as a non profit, back in the time it was open

Exactly. Neither firm would have been (successfully) sued by their shareholders for failing to make significant profits, so let's not blame on capitalism what is instead the individual greed driving these decisions. In fact, OpenAI is now going to trial because it gave up its non-profit status, reneging on the commitments it made to its shareholders (fraud, by another word).

Re: Anthropic drops flagship safety pledge

#650

Post by hollerith »

observationist wrote: post #512
LeifCarrotson wrote: post #491

Earlier quoted context omitted.

"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet. Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular pe…

I'm not saying it's technically impossible, I'm saying that in the real world, it's not going to stop. Nobody is going to stop it. A significant number of people don't want it to stop. A minority of people are in the "stop AI" camp, and the ones with the money and power are on the other side. It's an arms race replete with tribalism and the quest for power and taps into everything primal at the root of human behavior…

The whole world decided in the 1970s not to pursue the technology of germ-line genetic engineering of humans, and that decision has stood.

People similar to you were saying in the 1950s and later that it was inevitable that nuclear weapons would be used in anger in massive attacks.

Although the people in charge are tentatively for AI "progress", if that ever changes, they can and will put a stop to large AI training runs and make it illegal for anyone they don't trust to teach, learn or publish about fundamental algorithmic "improvements" to AI. Individuals and groups pursuing "improvements" will not be able to accept grant money or investment money or generate revenue from AI-based services.

That won't stop all research on such improvements (because some AI researchers are very committed), but it will slow it down to a rate much much slower than the current rate (because the current fast rate depends of rapid communication between researchers who don't each other well, and if communicating about the research were to become illegal, then a researcher can communicate only with those researchers he knows won't rat him out) essentially stopping AI "progress" unless (unluckily for the human species) at the time of the ban, the committed researchers were only one small step away from some massive algorithmic improvement that can be operationalized using the compute resources at their disposal (i.e., much less than the resources they have now).

Will the power elite's attitude towards AI change? I don't know, but if they ever come to have an accurate understanding of the situation, they will recognize that AI "progress" is a potent danger to them personally, and they will shut it down.

It's not a situation like the industrial revolution in England in which texile workers were massively adversely affected (or believed they were) but the people running England were mostly insulated from any adverse effects. In the current situation, the power elite is definitely not insulated from severe adverse consequences if an AI lab creates an AI that is much more competent that the most competent human institutions (e.g., the FBI) and the lab fails to keep the AI under control. And it will fail if it were to use anything like the methods and bodies of knowledge AI labs have been using up to now. And there are very bright people with funding doing their best to explain that to the elite.

Those of you who want AI "progress" to continue until the world is completely transformed need to hope that the power elite are collectively too stupid to recognize a potent short-term threat to their own survival (or the transformation can be completed before the power elite wake up and react). And in my estimation, that is not inevitable.

Post reply on HN