Live data from Hacker News

Why AI companies want you to be afraid of them

bbc.com

221–230 of 230 posts

Re: Why AI companies want you to be afraid of them

#221

Earlier quoted context omitted.

I work in security, and I think it's marketing BS meant to drive FOMO until proven otherwise. You cannot take any claims from these people seriously, they lie constantly.

> I work in security Doing what? School admins work in security.

Your mom, primarily.

Also general blue team shit and appsec.

Re: Why AI companies want you to be afraid of them

#222

Earlier quoted context omitted.

> I work in security Doing what? School admins work in security.

Your mom, primarily. Also general blue team shit and appsec.

lol.

Aren’t you convinced by the posts by security researchers (and more to the point, non-security-researchers) claiming semiautonomous (or better) 0day discovery with these tools?

Haven’t seen enough of them?

Help me understand.

Re: Why AI companies want you to be afraid of them

#223
post #182
post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

My observation is that the true believers really don't want to think of models as an inert pile of weights. There's some mysticism attached to imagining it's the ship's computer from Star Trek, HAL-9000 or C-3PO. A file loaded into memory and executed over is just so... _pedestrian_.

Canonically, the Star Trek computers have pretty much always been just computers, not themselves sentient because the software running on them just isn't.

I'm still not sure if HAL-9000 was supposed to be conscious or just an interesting plot device with a persona as superficial as LLMs are dismissed as today.

LLMs could definitely play the part of all three of your examples, given the flaws they showed on-screen. Could even do a decent approximation of Data (though perhaps not Lore without some jailbreaking).

Still weird that even the best of them isn't really ready to be KITT.

Re: Why AI companies want you to be afraid of them

#224

Earlier quoted context omitted.

Your mom, primarily. Also general blue team shit and appsec.

lol. Aren’t you convinced by the posts by security researchers (and more to the point, non-security-researchers) claiming semiautonomous (or better) 0day discovery with these tools? Haven’t seen enough of them? Help me understand.

Why? I'm clearly not going to convince you lol. You convince me I should.

Re: Why AI companies want you to be afraid of them

#225

Earlier quoted context omitted.

Exxon has never bragged to investors that they'd burn so much oil, civilization would collapse from climate change. They've always talked about how great fossil fuels are for the economy and our living standards. It makes no sense to sell apocalypse to investors either.

They're selling FOMO to investors. "Last chance to jump on the AI train, invest into your future robot overlord or be turned into biodiesel for datacenters in the future."

There's no reason to think an out-of-control ASI would spare its investors.

Re: Why AI companies want you to be afraid of them

#226

Earlier quoted context omitted.

They're selling FOMO to investors. "Last chance to jump on the AI train, invest into your future robot overlord or be turned into biodiesel for datacenters in the future."

There's no reason to think an out-of-control ASI would spare its investors.

There's no reason to think it wouldn't. Shouldn't you hedge your bets?

Also, you can probably make a shitton of money as an out-of-control-AI-investor while the world is in the process of being destroyed.

Re: Why AI companies want you to be afraid of them

#227

Earlier quoted context omitted.

There's no reason to think an out-of-control ASI would spare its investors.

There's no reason to think it wouldn't. Shouldn't you hedge your bets? Also, you can probably make a shitton of money as an out-of-control-AI-investor while the world is in the process of being destroyed.

There are all sorts of things you could do that might make an AI like you, and none of them have more justification than any other. This is not an argument AI firms are making.

I agree that short-term greed is driving investment, but it would drive just as much investment if AI companies were not warning of apocalypse. Probably it would drive even more, because there'd be less risk of regulatory interference, and more future profit to discount into the present.

So why are they making those warnings? It doesn't benefit them. The simplest explanation is that this stuff actually is dangerous, and people who know that are worried.

Re: Why AI companies want you to be afraid of them

#229
post #213

Earlier quoted context omitted.

They’re not clueless they just don’t have a memory and they don’t have judgement. They create the illusion of being able to make decisions but they are always just following a simple template.They do not consider nuance, they cannot judge between two difficult options in a real sense. Which is why they can delete prod databases and why they cannot do expert level work

>they cannot do expert level work Well this is just factually incorrect considering they are currently on par with grad students in some areas of mathematics.

Not sure if you are being pedantic but mathematics is quite different from other fields because it is highly structured, reasoning is explicit and it contains a dense volume of high level training data. Results are able to be verified easily due to its structure.

Even then, they are most effective in assisting and are not able to produce results independently. If you have proof otherwise I would love to read up on it

Re: Why AI companies want you to be afraid of them

#230

Earlier quoted context omitted.

There's no reason to think it wouldn't. Shouldn't you hedge your bets? Also, you can probably make a shitton of money as an out-of-control-AI-investor while the world is in the process of being destroyed.

There are all sorts of things you could do that might make an AI like you, and none of them have more justification than any other. This is not an argument AI firms are making. I agree that short-term greed is driving investment, but it would drive just as much investment if AI companies were not warning of apocalypse. Probably it would drive even more, because there'd be less risk of regulatory interference, and mor…

> So why are they making those warnings? It doesn't benefit them.

Because "we built a chatbot that can generate technical debt" is not a good proposition for investors. "Invest into our AI before it takes over the world and fires all knowledge workers" is.

> The simplest explanation is that this stuff actually is dangerous, and people who know that are worried.

LMAO. Please.

Post reply on HN