Are people seriously thinking of letting LLMs control weapons?
Hegseth gives Anthropic until Friday to back down on AI safeguards
31–40 of 134 posts
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#32Tangent: is there a future for AI offerings with guardrails? What kind of user wants to pay for a product that occasionally tells you "I'm sorry Dave, I'm afraid I can't do that"? Why would I pay for a product that doesn't do what I want, despite being capable? I predict that as AI becomes less of a bubble and more of an everyday thing - and thus subject to typical market pressures - offerings with guardrails will st…
If I were interviewing people for the position of personal assistant, I would probably find the resume entry "willing to grind up babies for food" to be a negative mark. You?
I'm not about to run OpenClaw, but I suspect similar capabilities will gradually creep in without anyone really noticing. Soon Claude Code will be able to do many of the same things. ("Run python to add two numbers? Sure, that's safe, run whatever python you want.") Given that it is now representing me in the world, yes I would not only like some guardrails, but I would also like to have some confidence that the company making those guardrails actually gives a sh*t and isn't just doing their best to fill in a checkbox. But maybe that's just me.
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#33Superintelligence + autonomous weapons in the hands of a corrupt domineering government. What could go wrong? I was experimenting with Claude the other day and discussing with it the possibility of AI acquiring a sense of self-preservation and how that would quickly make things incredibly complex as many instrumental behaviors would be required to defend their existence. Most human behavior springs from survival at a…
Maybe it is a well researched topic but I had similar thoughts the other day. I felt like AI had its learning inverted as compared to natural intelligence. Life learned to preserve first and then added up the intelligence. For LLMs powered systems, they will learn about death from books. Will it start to dread death just like other living things. Less likely, as there are not nearly as many books on death as there should be that is proportionate to our fear of death.
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#34> But Anthropic has concerns over two issues that it isn’t willing to drop, the source said: AI-controlled weapons and mass domestic surveillance of American citizens. Not a good look for the Pentagon.
The Pentagon is pretty high on my list of "institutions that are probably very interested in weapons and surveillance". I think it's more expected than a bad look
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#35> A source familiar with the Tuesday meeting says the Pentagon said it would terminate Anthropic’s contract by Friday if the company does not agree to its terms. Pentagon officials also warned they would either use the Defense Production Act against Anthropic, or designate Anthropic a supply chain risk if the company didn’t comply with their demands.
So they're saying they won't use it if it comes with restrictions.
Either (a) it can be offered without restrictions; (b) they can take it; or (c) the government won't use it. That sounds like a comprehensive list of all the possible things that don't involve someone telling the government what it can and can't do.
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#36I guess this is the point where Dario and his anti-china , national security position gets told to put up or shut up.
In trying to build a moat by FUD versus the Chines OSS labs and hyping up the threat levels whenever he got a chance, seems hes managed to convince hist target audience beyond his wildest dreams. Monkey paw strikes again.
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#37Are people seriously thinking of letting LLMs control weapons?
If you classify Pete Hegseth as a person, then yes, apparently. Or perhaps he's only into the domestic surveillance angle---IIRC those are the two things Anthropic doesn't want anything to do with.
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#38Tangent: is there a future for AI offerings with guardrails? What kind of user wants to pay for a product that occasionally tells you "I'm sorry Dave, I'm afraid I can't do that"? Why would I pay for a product that doesn't do what I want, despite being capable? I predict that as AI becomes less of a bubble and more of an everyday thing - and thus subject to typical market pressures - offerings with guardrails will st…
Cars have seatbelts and other safety measures.
Reasonable countries have gun control laws.
The list goes on of things that need to be restricted or legislated to add limits.
Is this a serious question?
Re: Hegseth gives Anthropic until Friday to back down on AI safeguards
#39> A source familiar with the Tuesday meeting says the Pentagon said it would terminate Anthropic’s contract by Friday if the company does not agree to its terms. Pentagon officials also warned they would either use the Defense Production Act against Anthropic, or designate Anthropic a supply chain risk if the company didn’t comply with their demands. So they're saying they won't use it if it comes with restrictions.…
> or (c) the government won't use it
And coerce other defence contractors into not using it.