Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI companies want you to be afraid of them

bbc.com

Post reply on HN
Print view View original

121–130 of 230 posts

Re: Why AI companies want you to be afraid of them

#121

Post by georgemcbay »

add-sub-mul-div wrote: post #96
dataviz1000 wrote: post #93

Earlier quoted context omitted.

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

If you told a programmer 30 years ago that someday we'd switch from a deterministic to nondeterministic paradigm for programming computers, they'd ask if we'd put lead back in the drinking water.

> they'd ask if we'd put lead back in the drinking water.

With Lee Zeldin heading the EPA is anyone sure we won't?

Re: Why AI companies want you to be afraid of them

#122

Post by autoexec »

SpicyLemonZest wrote: post #11

> It's a strange way for any company to talk about its own work. You don't hear McDonald's announcing that it's created a burger so terrifyingly delicious that it would be unethical to grill it for the public. > Here's one theory. But the author never gets back to this! It's the main observation the theory has to account for; why don't we see other companies speak this way, if it's such an effective strategy for defl…

> why don't we see other companies speak this way

They do. Every company who promised us that their shitty cell phone app or website was going to change the world and revolutionize and disrupt industry/society was guilty of the same thing. They just usually focused their ridiculous levels of hype on the positives. The goal was the same. "Our technology is going to change the world so investors had better give us cash or else they will be left behind" is still the message.

I think this is just an advancement of what we saw with self-driving cars and how companies were pushing narratives around how every trucker will be out of work (this still hasn't happened) or how no individuals would own a car again while deflecting from things like how badly their cars performed in snow/rain or in anything other than very carefully controlled and mapped out conditions.

Re: Why AI companies want you to be afraid of them

#123

Post by ben_w »

AndrewKemendo wrote: post #107
stuaxo wrote: post #61

Earlier quoted context omitted.

Eh, these same people will attach openclaw to production systems soon and destroy their own companies.

Sounds like a pretty efficient self correcting mechanism I’m not sure what the problem is there

Normalisation of deviance is the problem: https://en.wikipedia.org/wiki/Normalization_of_deviance

Remember that these models are getting better; this means they get trusted with increasingly more important things by the time an error explodes in someone's face.

It would be very bad if the thing which explodes is something you value which was handed off to an AI by someone who incorrectly thought it safe.

AI companies which don't openly report that their AI can make mistakes are being dishonest, and that dishonesty would make this normalization of deviance even more prevelant than it already is.

Re: Why AI companies want you to be afraid of them

#124

Post by CamperBob2 »

stuaxo wrote: post #61
codingdave wrote: post #57

Earlier quoted context omitted.

Yes, and it has been said since day one of LLMs that all we need to do is keep things that way - no action without human intervention. Just like it was said that you should never grant AI direct access to change your production systems. But the stories of people who have done exactly that and had their systems damaged and deleted show that people aren't trying to even keep such basic safety nets in place. AI is getti…

Eh, these same people will attach openclaw to production systems soon and destroy their own companies.

The problem is, out of ten companies who take this approach, nine will indeed destroy themselves and one will end up with a trillion-dollar market cap. It will outcompete hundreds of companies who stuck with more conservative approaches. Everybody will want to emulate company #10, because "it obviously works."

I don't see any stabilizing influences on the horizon, given how much cash is sloshing around in the economy looking for a place to land. Things are going to get weird, stupid, and chaotic, not necessarily in that order.

Re: Why AI companies want you to be afraid of them

#125

Post by airstrike »

dataviz1000 wrote: post #93
boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

While you're at it, I'll take a pair of unicorns too if you can find them.

Re: Why AI companies want you to be afraid of them

#126

Post by munk-a »

add-sub-mul-div wrote: post #96
dataviz1000 wrote: post #93

Earlier quoted context omitted.

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

If you told a programmer 30 years ago that someday we'd switch from a deterministic to nondeterministic paradigm for programming computers, they'd ask if we'd put lead back in the drinking water.

We'd just explain that management told us we had to and then they'd understand.

Re: Why AI companies want you to be afraid of them

#127

Post by otabdeveloper4 »

ctoth wrote: post #98
simonw wrote: post #76

Earlier quoted context omitted.

I don't think subagents are representative of anything particularly interesting on the "agents can run themselves" front. They're tool calls. Claude Code provides a tool that lets the model say effectively: run_in_subagent("Figure out where JWTs are created and report back") The current frontier models are all capable of "prompting themselves" in this way, but it's really just a parlor trick to help avoid burning mor…

The mechanism being simple is the interesting part. If one large complex goal can be split into subgoals and the subgoals completed without you, then you need a lot fewer humans to do a lot more work. The OP says AI requires human interaction to work. This simply isn't true. You know yourself that as agents get more reliable you can delegate more to them, including having them launch more subagents, thereby getting m…

You misunderstand.

The only reason to launch subagents is to avoid poisoning the LLM's already small context window with unrelated tokens.

It doesn't make the LLM smarter or more capable.

Re: Why AI companies want you to be afraid of them

#128

Post by ambicapter »

deepsquirrelnet wrote: post #50

This is my own take, directly related to this that I posted a little while back. The one thing that I think the article missed is the geopolitical angle they’re also working: * We need to completely deregulate these US companies so China doesn't win and take us over * We need to heavily regulate anybody who is not following the rules that make us the de-facto winner * This is so powerful it will take all the jobs (an…

The tech broligarchs learned from their algorithms that fear sells whatever they want, and they carried that lesson into their "thought leadership".

Re: Why AI companies want you to be afraid of them

#129

Post by trolleski »

boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

Just call the errors 'consciousness' and keep selling those tokens! Let the Spineless Generation have their last bubble!

Re: Why AI companies want you to be afraid of them

#130

Post by throwaway27448 »

dataviz1000 wrote: post #93
boh wrote: post #35

I think the big secret is that AI is just software. In the same way that a financial firm doesn't all of sudden make a bunch of money because Microsoft shipped an update to Excel, AI is inert without intention. If there's any major successes in AI output it's because a person got it to do that. Claude Code is great, but it will also wipe out a database even though it's instructed not to (I can confirm from experience…

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

What would a verifier even look like without having all of the same problems that the chatbot itself does? Are humans themselves not the cheap verifiers?
Post reply on HN