Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI companies want you to be afraid of them

bbc.com

Post reply on HN
Print view View original

61–70 of 230 posts

Re: Why AI companies want you to be afraid of them

#61

Post by stuaxo »

codingdave wrote: post #57
boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

Yes, and it has been said since day one of LLMs that all we need to do is keep things that way - no action without human intervention. Just like it was said that you should never grant AI direct access to change your production systems. But the stories of people who have done exactly that and had their systems damaged and deleted show that people aren't trying to even keep such basic safety nets in place. AI is getti…

Eh, these same people will attach openclaw to production systems soon and destroy their own companies.

Re: Why AI companies want you to be afraid of them

#64

Post by fnoef »

ctoth wrote: post #59
boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

> doesn't change the fact that it's software that requires human interaction to work. Have you ever seen Claude Code launch a subagent? You've used it, right? You've seen it launch a subagent to do work? You understand that that is, in fact, Claude Code running itself, right?

My Linux server runs a cron job, that can spin off a thread and even use other ~apps~ tools. Did I invent AGI?

Re: Why AI companies want you to be afraid of them

#65

Post by b00ty4breakfast »

Imnimo wrote: post #31

My read is not so much "if we say this is dangerously powerful, it will make people want to buy our product", but rather that there is a significant segment of AI researchers for whom x-risk, AI alignment, etc. is a deal-breaker issue. And so the Sam Altmans of the world have to treat these concerns as serious to attract and retain talent. See for example OpenAI's pledge to dedicate 20% of their compute to safety res…

Maybe Altman specifically is only paying lip service to this stuff, but when a company like Anthropic is like "BRO MYTHOS IS TOO DANGEROUS BRO WE CANT EVEN RELEASE IT BRO JUST TRUST US BRO", my bullshit detector is beeping too loud to ignore. It's very obviously a publicity stunt, because if it were actually that dangerous you wouldn't be making such a press release, you'd be keeping your mouth shut and working to make it safe.

Re: Why AI companies want you to be afraid of them

#66

Post by ctoth »

fnoef wrote: post #64
ctoth wrote: post #59

Earlier quoted context omitted.

> doesn't change the fact that it's software that requires human interaction to work. Have you ever seen Claude Code launch a subagent? You've used it, right? You've seen it launch a subagent to do work? You understand that that is, in fact, Claude Code running itself, right?

My Linux server runs a cron job, that can spin off a thread and even use other ~apps~ tools. Did I invent AGI?

Does your Linux server decide what processes it should launch at what time with a theory of what will happen next in order to complete a goal you specified in natural language? If so yes, I reckon you sure have!

Re: Why AI companies want you to be afraid of them

#67

Post by gdulli »

SpicyLemonZest wrote: post #11

> It's a strange way for any company to talk about its own work. You don't hear McDonald's announcing that it's created a burger so terrifyingly delicious that it would be unethical to grill it for the public. > Here's one theory. But the author never gets back to this! It's the main observation the theory has to account for; why don't we see other companies speak this way, if it's such an effective strategy for defl…

If McDonald's food was featured in sci fi movies about being able to end humanity through war, that's when this would apply and they'd cultivate fear of that nonsense to distract from their food being shitty and overpriced and unhealthy.

Re: Why AI companies want you to be afraid of them

#68

Post by echelon »

ctoth wrote: post #59
boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

> doesn't change the fact that it's software that requires human interaction to work. Have you ever seen Claude Code launch a subagent? You've used it, right? You've seen it launch a subagent to do work? You understand that that is, in fact, Claude Code running itself, right?

All AI requires steering as the results begin to decohere and self-enshittify over time.

AI in the hands of an expert operator is an exoskeleton. AI left alone is a stooge.

Nobody has built an all-AI operator capable of self-direction and choices superior to a human expert. When that happens, you'd better have your debts paid and bunker stocked.

We haven't seen any signs of this yet. I'm totally open to the idea of that happening in the short term (within 5 years), but I'm pessimistic it'll happen so quickly. It seems as though there are major missing pieces of the puzzle.

For now, AI is an exoskeleton. If you don't know how to pilot it, or if you turn the autopilot on and leave it alone, you're creating a mess.

This is still an AI maximalist perspective. One expert with AI tools can outperform multiple experts without AI assistance. It's just got a much longer time horizon on us being wholly replaced.

Re: Why AI companies want you to be afraid of them

#69

Post by MSFT_Edging »

notrealyme123 wrote: post #8
detectivestory wrote: post #4

Earlier quoted context omitted.

That's true but in reality I think people are far more afraid of AI in terms of how it is being used in warfare and policing. Automatic target detection and deployment of drones, or even how it might simply make their role at work redundant etc

I am to be honest not sure what I am more scared. AI shaping warfare Vs. Using AI to justify outrageous warfare

We sadly don't need AI to justify outrageous warfare. You just need to remember when the US invaded Iraq over WMDs, including a full investigation into the WMDs that never found any. We then invaded anyway, to the detriment of everyone except defense contractors.

Re: Why AI companies want you to be afraid of them

#70

Post by balls187 »

ctoth wrote: post #66
fnoef wrote: post #64

Earlier quoted context omitted.

My Linux server runs a cron job, that can spin off a thread and even use other ~apps~ tools. Did I invent AGI?

Does your Linux server decide what processes it should launch at what time with a theory of what will happen next in order to complete a goal you specified in natural language? If so yes, I reckon you sure have!

Claude does not have a "theory" of anything, and I'd argue applying that mental model to LLM+Tools is a major reason why Claude can delete a production database.
Post reply on HN