Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI companies want you to be afraid of them

bbc.com

Post reply on HN
Print view View original

141–150 of 230 posts

Re: Why AI companies want you to be afraid of them

#141

Post by Terr_ »

chis wrote: post #37
Imnimo wrote: post #31

My read is not so much "if we say this is dangerously powerful, it will make people want to buy our product", but rather that there is a significant segment of AI researchers for whom x-risk, AI alignment, etc. is a deal-breaker issue. And so the Sam Altmans of the world have to treat these concerns as serious to attract and retain talent. See for example OpenAI's pledge to dedicate 20% of their compute to safety res…

Yeah I just don't buy that it would somehow help AI companies for everyone to be existentially afraid of their technology. It seems much more reasonable to think that they really believe the things they're saying, than that it's some kind of 4d chess. Additionally Dario has just been really accurate with his predictions so far. For instance in early 2025 he predicted that nearly 100% of code would be written with AI…

> Yeah I just don't buy that it would somehow help AI companies for everyone to be existentially afraid of their technology.

It makes more sense if one breaks that "everyone" into subgroups. A good first-pass split would be "investors" versus "everyone else."

From their perspective: Rich Investor Alice rushing over with bags of money because of FOMO >>> Random Person Bob suffers anxiety reading the news.

One can hone it a bit more by thinking about how it helps them gain access to politicians, media that's always willing to spread their quotes, and even just getting CEO Carol's name out there.

Re: Why AI companies want you to be afraid of them

#142

Post by com2kid »

add-sub-mul-div wrote: post #96
dataviz1000 wrote: post #93

Earlier quoted context omitted.

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

If you told a programmer 30 years ago that someday we'd switch from a deterministic to nondeterministic paradigm for programming computers, they'd ask if we'd put lead back in the drinking water.

It has always been non-deterministic but we relied on low level engineers who knew the dark magicks to keep the horrors at bay.

Bit flips in memory are super common. Even CPUs sometimes output the wrong answer for calculations because of random chance. Network errors are common, at scale you'll see data corruption across a LAN often enough that you'll quickly implement application level retries because somehow the network level stuff still lets errors through.

Some memory chips are slightly out of timing spec. This manifests itself as random crashes, maybe one every few weeks. You need really damn good telemetry to even figure out what is going on.

Compilers do indeed have bugs. Native developers working in old hairy code bases will confirm, often with stories of weeks spent debugging what the hell was going on before someone figured out the compiler was outputting incorrect code.

It is just that the randomness has been so rare, or the effects so minor, that it has all been, mostly, an inconvenience. It worries people working in aviation or medical equipment, but otherwise people accept the need for an occasional reboot or they don't worry about a few pixels in a rendered frame being the wrong color.

LLMs are uncertainty amplifiers. Accept a lot of randomness and in return you get a tool that was pure sci-fi bullshit 10 years ago. Hell when reading science fiction now days I am literally going "well we have that now, and that, oh yeah we got that working, and I think I just saw a paper on that last week."

Re: Why AI companies want you to be afraid of them

#143

Post by bryan0 »

dodu_ wrote: post #80
bryan0 wrote: post #25

> Why do AI companies want us to be afraid of them? ... According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they're already doing to the world. People seem unable to make up their mind if AI is very dangerous or is it not. I think what the AI companies and this author agree on, is that this technology is potentially extremely dangerous. AI im…

>So maybe instead of demonizing the AI companies, we have a nuanced debate about this tech and propose solutions that our best for our society? These are not mutually exclusive. Calling out the demonic behavior of trying to coerce people into using your product out of fear is not an indictment of the underlying technology itself.

One of the points I was trying to make is that the statement:

> trying to coerce people into using your product out of fear

is nonsense.

Everyone agrees that there are legitimate reasons to be fearful of this technology, this is not a fabrication, but we need to figure out how to proceed in a safe and constructive way.

What "coercion" is occurring here? Either you find the technology valuable and you want to pay for it, or you find it not useful (or worse harmful), and you do not want to pay for it.

Maybe another way of putting it, what do you think the frontier AI companies should do in this situation? It seems that being straightforward with the dangers is correct thing to do, and probably being overly cautious is prudent. You could go further and argue they should slow down or stop development, but that is something that the govt should impose, we should not expect or trust the companies to do this themselves. Ironically, in the Anthropic / Pentagon case, we have Anthropic trying to pump the brakes and put up guardrails while the govt wants to go full-steam ahead with autonomous warfare.

The other issue with slowing down / pausing development is it requires an unheard of level of agreement, even with companies in China, or else it will probably not be effective. You could argue this is not even possible at this point.

Re: Why AI companies want you to be afraid of them

#144

Post by f1shy »

drBonkers wrote: post #97
dataviz1000 wrote: post #93

Earlier quoted context omitted.

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

Do you have any readings you recommend to start thinking in terms of non-deterministic algorithms and cheap verifiers?

Neurosymbolic programming

Re: Why AI companies want you to be afraid of them

#145

Post by xdavidliu »

throwaway27448 wrote: post #130
dataviz1000 wrote: post #93

Earlier quoted context omitted.

LLM models are a distribution. Unlike a python script or turning machine, a LLM model is capable of generating any series of tokens. Developers need stop reasoning about LLM agents as deterministic and to start to think about agents in terms of Monte Carlo and Las Vegas algorithms. It isn't enough to have an agents, it also requires a cheap verifier. If I was a Ph.D. student today, I'd probably do a thesis on cheap v…

What would a verifier even look like without having all of the same problems that the chatbot itself does? Are humans themselves not the cheap verifiers?

humans are probably the least cheap thing you can have in this context

Re: Why AI companies want you to be afraid of them

#146

Post by goatlover »

AndrewKemendo wrote: post #105
staminade wrote: post #87

AI company leaders didn't invent this concern about the potential dangers of AI, either as a cause of economic disruption, or as a potential extinction risk. Superintelligence was published in 2014, and even then it wasn't a new topic. Technologists, philosophers and science fiction authors have been discussing and speculating about AI risk for decades. Also, the idea that AI leadership seized on and amplified these…

Very well put and I think that covers pretty much everything that needs to be said here. In fact it has been AI people who have been leading discussions around AI ethics and the dangers of AI since 1955. This is not new and it is consistent. The new thing is that the average person is now entering into the debate around AI; And like pretty much everything else in the public sphere doing it with entirely no context. I…

> Most of the builders don’t care about this mess and are continuing to work like usual.

So they don't consider it an existential threat, unlike what the CEOs of companies raising hundreds of billions are saying.

Re: Why AI companies want you to be afraid of them

#147

Post by tikkabhuna »

AndrewKemendo wrote: post #107
stuaxo wrote: post #61

Earlier quoted context omitted.

Eh, these same people will attach openclaw to production systems soon and destroy their own companies.

Sounds like a pretty efficient self correcting mechanism I’m not sure what the problem is there

The problem is that destruction isn't contained to the company. If an AI agent exposes all company data and that includes PII or health information, that could have an impact on a large number of people.

Re: Why AI companies want you to be afraid of them

#148

Post by lofaszvanitt »

deepsquirrelnet wrote: post #50

This is my own take, directly related to this that I posted a little while back. The one thing that I think the article missed is the geopolitical angle they’re also working: * We need to completely deregulate these US companies so China doesn't win and take us over * We need to heavily regulate anybody who is not following the rules that make us the de-facto winner * This is so powerful it will take all the jobs (an…

Which trillion company is regulated in the US?

Re: Why AI companies want you to be afraid of them

#149

Post by gip »

deepsquirrelnet wrote: post #50

This is my own take, directly related to this that I posted a little while back. The one thing that I think the article missed is the geopolitical angle they’re also working: * We need to completely deregulate these US companies so China doesn't win and take us over * We need to heavily regulate anybody who is not following the rules that make us the de-facto winner * This is so powerful it will take all the jobs (an…

> We need to heavily regulate anybody who is not following the rules that make us the de-facto winner

How about building a multipolar world where different parts of the world (US/China/India/EU/Africa,..) get to build sovereign tech and have their own winners?

Re: Why AI companies want you to be afraid of them

#150

Post by bryan0 »

Tangurena2 wrote: post #136
bryan0 wrote: post #25

> Why do AI companies want us to be afraid of them? ... According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they're already doing to the world. People seem unable to make up their mind if AI is very dangerous or is it not. I think what the AI companies and this author agree on, is that this technology is potentially extremely dangerous. AI im…

> People seem unable to make up their mind if AI is very dangerous or is it not. This is a propaganda tactic. For decades, tobacco companies claimed that there was no evidence that smoking was bad for one's health. Then, only after losing dozens of lawsuits did the propaganda switch to "but everyone knew for 100+ years that smoking was lethal". One can read about it by reading Trust Us, We're Experts , or Toxic Sludg…

Please explain how this tactic relates here. In this case we have the AI companies saying this technology is potentially very harmful, in fact existential. This seems the complete opposite of what big tobacco did.

What I meant by

> People seem unable to make up their mind if AI is very dangerous or is it not.

Is that the article says 2 contradictory things:

1. AI companies are misleading us when they say their tech is dangerous and people should be afraid.

2. AI is currently very dangerous and people should be afraid.

Anecdotally, people on the internet (including HN), seem unable to agree on whether AI is real or overblown "hype".

Post reply on HN