Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

531–540 of 716 posts

Re: Anthropic drops flagship safety pledge

#531

Post by amelius »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

> Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible" ones.

Reminds me of:

https://en.wikipedia.org/wiki/Paradox_of_tolerance

which has the same kind of shitty conclusion.

Re: Anthropic drops flagship safety pledge

#532

Post by PowerElectronix »

ACCount37 wrote: post #487
PowerElectronix wrote: post #464

Earlier quoted context omitted.

Same with everything, right? You could say the same with nukes, electricity, internet, the computer, etc... But if you look at it without paying attention to the "ultimate tool for humanity" hype, it doesn't really look that much of a threat or a salvation. It won't end civilization for dropping the guardrails, but it will surely enable bad actors to do more damage than before (mass scams, blackmail, deepfake nudes,…

Very few things are as powerful and dangerous as AI. AI at AGI to ASI tier is less of "a bigger stick" and more of "an entire nonhuman civilization that now just happens to sit on the same planet as you". The sheer magnitude of how wrong that can go dwarfs even that of nuclear weapon proliferation. Nukes are powerful, but they aren't intelligent - thus, it's humans who use nukes, and not the other way around. AI can…

I think we are giving too much credit to what is a bunch of bayesian filters under a trenchcoat.

Re: Anthropic drops flagship safety pledge

#533

Post by ACCount37 »

visarga wrote: post #493
observationist wrote: post #474

Earlier quoted context omitted.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…

John Good's quote is pretty myopic, it assumes machines make better machines based on being "ultraintelligent" instead of learning from environment-action-outcome loop. It's the difference between "compute is all you need" and "compute+explorative feedback" is all you need. As if science and engineering comes from genius brains not from careful experiments.

At sufficient levels of intelligence, one can increasingly substitute it for the other things.

Intelligence can be the difference between having to build 20 prototypes and building one that works first try, or having to run a series of 50 experiments and nailing it down with 5.

The upper limit of human intelligence doesn't go high enough for something like "a man has designed an entire 5th gen fighter jet in his mind and then made it first try" to be possible. The limits of AI might go higher than that.

Re: Anthropic drops flagship safety pledge

#534

Post by SecretDreams »

SoftTalker wrote: post #530
SecretDreams wrote: post #518

Earlier quoted context omitted.

> If AI tech goes very well The IF here is doing some very heavy lifting. Last I checked, for profit companies don't have a good track record of doing what's best for humanity.

For profit companies do have a good track record of doing what's best for profit. If their AI creates a world where human intelligence, labor, and money are worthless, or where their creations take control of those things instead of them having control, that's not a very good outcome for them.

> If their AI creates a world where human intelligence, labor, and money are worthless, or where their creations take control of those things instead of them having control, that's not a very good outcome for them.

You would think that, but a lot of kings and people in power have been able to achieve something similar over our humanity's history. The trick is to not make things "completely worthless". Just to increase the gap as much as (in)humanly possible while marching us towards a deeper sense of forced servitude.

Re: Anthropic drops flagship safety pledge

#535

Post by coldtea »

ACCount37 wrote: post #414
coldtea wrote: post #330

Earlier quoted context omitted.

In general public benefit corporations and non-profits should have a very modest salary cap for everybody involved and specific public-benefit legally binding mission statements. Anybody involved should also be prohibited from starting a private company using their IP and catering to the same domain for 5-10 years after they leave. Non-profits where the CEO makes millions or billions are a joke. And if e.g. your miss…

"A very modest salary cap" works if your mission is planting trees. Not so much if what you're building is frontier AI systems.

If a non-profit can't attract people not motivated except by profit, perhaps it shouldn't exist.

Re: Anthropic drops flagship safety pledge

#536

Post by coldtea »

ACCount37 wrote: post #462
the_bear wrote: post #425

Earlier quoted context omitted.

I think that's the point though. The AI companies can't compete without hiring very talented employees and raising lots of money from investors. Neither the employees nor investors would participate if there weren't the potential for making mountains of money. So these AI companies fundamentally can't be non-profits or true B-corps (I realize that's a vague term, but the it certainly means not doing whatever it takes…

To me, it feels like saying "you can't be a public benefit corporation unless all the labor involved in delivering that public benefit is cheap". Which just doesn't seem like it should be true? Sure, some "public benefit" missions could scale sideways and employ a lot of cheap labor, not suffering from a salary cap at all. But other missions would require rare high end high performance high salary specialists who are…

>But other missions would require rare high end high performance high salary specialists who are in demand - and thus expensive. You can't rely on being able to source enough altruists that will put up with being paid half their market worth for the sake of the mission.'

That's exactly what a non-profit should be able to rely on. And not just "half their market worth", but even many times less.

Else we can just say "we can't really have non-profits, because everybody is a greedy pig who doesn't care about public benefit enough to make a sacrifice of profits - but still a perfectly livable salary" - and be done with it.

Re: Anthropic drops flagship safety pledge

#537

Post by goodmythical »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

Isn't curing cancer just as dangerous as a nuclear bomb? Especially considering some of the gene-therapies under consideration? Because you can bet that a non-negligable portion of research in this space is being funded by governments and groups interested in application beyond curing cancer. (Autism? Whiteness? Jewishness? Race in general? Faith in general? Could china finally cure western greed? Maybe we can slip some extra compliancy in there so that the plebia- ah- population is easier to contr- ah- protect.)

Curing all cancers would increase population growth by more than 10% (9.7-10m cancer related deaths vs current 70-80m growth rate), and cause an average aging of the population as curing cancer would increase general life expectancy and a majority of the lives just saved would be older people.

We'd even see a jobs and resources shock (though likely dissimilar in scale) as billions of funding is shifted away from oncologists, oncology departments, oncology wards, etc. Billions of dollars, millions of hospital beds, countless specialized professionals all suddenly re-assigned just as in AI.

Honestly the cancer/nuclear/tech comparison is rather apt. All either are or could be disruptive and either are or could be a net negative to society while posing the possibility of the greatest revolution we've seen in generations.

Re: Anthropic drops flagship safety pledge

#538

Post by mullingitover »

kunai wrote: post #428
gordian-mind wrote: post #410

Earlier quoted context omitted.

Participating in the economic life before re-allocating that wealth produced to philanthropic activities sounds pretty good. Modern concentrated wealth is hardly misanthropic, since it's mostly private equity, that is, companies with people and jobs.

Except this is not the age of the Rockefellers or the Carnegies, who, despite being far more philanthropic than modern-day billionaires, drew ire from every corner of society for their wealth accumulation. It wasn't until the New Deal that the balance shifted. Unconstrained accumulation of capital into the hands of the few without appropriate investment into labor is illiberal and incompatible with democracy and true…

> Those of us who are capitalists see surplus value as a compromise to ensure good economic growth.

I think the problem is that every system of economics requires ignoring human nature in order to believe it possibly can work. In order to believe that capitalism doesn't lead to despotic rule you have to ignore the fact that civilizations love a good hierarchy far more than they love justice and fairness.

You can make any system of economics work if you figure out how to deal, head on, with the particular human nature factor that it tries to ignore.

Re: Anthropic drops flagship safety pledge

#539

Post by Cipater »

bn_layc wrote: post #429
Schlagbohrer wrote: post #404

Earlier quoted context omitted.

Pete Hegseth also threatened to take, by dictat, everything Anthropic has. He can do that with the Defense Industrial Act or whatever its called if he designates them as critical to national defense.

He seems to be the driving force behind all this. Mediocrities are attracted to AI like moths. The press always say "the Pentagon negotiates". Does any publication have an evidence that it is "the Pentagon" and not Hegseth? In general, I see a lot of common sense from the real Pentagon as opposed to the Secretary of War. I hope Westpoint will check for AI psychosis in their entrance interviews and completely forbid A…

Military academy boards have been purged and stacked with loyalists.
Post reply on HN