Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

newyorker.com

Post reply on HN
Print view View original

151–160 of 164 posts

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#151

Post by vidarh »

jacobgold wrote: post #143
vidarh wrote: post #136

Earlier quoted context omitted.

You're misrepresenting what I wrote. I specifically pointed out that I switch languages without intent to do so. When you suggest that is a "superficial analogy" after you were the one pointing out LLMs switching language as something that sets them apart, you're seriously reaching. I can often pinpoint afterward what was likely the trigger: E.g. I used a word that is the same in two languages, and continue in the se…

That's a fair point. I agree that the behavior can look similar even when the mechanism is different. But in practice, all the analogies I've seen are in fact superficial, including this one. The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

Most of the time when I see an LLM abruptly switching languages it usually continues with something directly related to whatever triggered it.

I'm sure there are other failure modes where it may change topics too, just like humans also regularly digress when triggered by certain words etc.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#152

Post by Kim_Bruning »

lproven wrote: post #121
orangecat wrote: post #50

Earlier quoted context omitted.

You can call your little doggy "AI" if it makes you happy. Or you can keep calling them stochastic parrots as they solve decades-old open problems. The real question is how useful they are, and the answer "not at all" increasingly requires flat-earth levels of denial. it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of They sort of are. Thi…

> The real question is how useful they are No, it is not. > and the answer "not at all" increasingly requires flat-earth levels of denial. No, it does not. For me, after ~25 years in the skeptics movement, I think the parallels with supplementary, complementary and alternative medicine are most useful. I choose that term intentionally: its initials are S.C.A.M. and that's exactly what it is. As Tim Minchin and Alan K…

> The reason AI works is the reason that people see the face of Jesus in slices of toast, as someone said recently.

Apparently my unit tests can see faces in slices of toast.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#153

Post by TeMPOraL »

jacobgold wrote: post #145
vidarh wrote: post #137

Earlier quoted context omitted.

Not nearly all prompt injections are by any means that absolute unless starting from the exact same state. Many of them will also work only probabilistically unless you turn temperature to 0 for exactly that reason. And at the same time, whole books have been written about how reliably we can induce certain behaviours from humans. E.g. the Blue-seven phenomenon [1] - I've personally experienced that second hand and i…

Anyone is free to draw whatever analogies they want, but they either make sense or they don't. Outside of philosophical discussions or science fiction, comparing influencing humans to prompt injection is silly.

I wouldn't call it silly, because see the two as directly equivalent and fundamentally the same thing.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#154

Post by dTal »

dofm wrote: post #140
dTal wrote: post #138

Earlier quoted context omitted.

...did it really, though?

Yeah, it did. I don't think this is really in doubt. Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc. It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely th…

Looks more like standard decision-washing finger pointing to me. "AI did it" is both hypey and also conveniently distracts from an uglier reality:

>Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike.

>The building in Minab had been classified as a military facility in a Defense Intelligence Agency database that, according to CNN, had not been updated to reflect that the building had been separated from the adjacent Islamic Revolutionary Guard Corps compound and converted into a school, a change that satellite imagery shows had occurred by 2016 at the latest. A chatbot did not kill those children. People failed to update a database, and other people built a system fast enough to make that failure lethal.

https://www.theguardian.com/news/2026/mar/26/ai-got-the-blam...

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#155

Post by dredmorbius »

dofm wrote: post #103
ACCount37 wrote: post #30

Earlier quoted context omitted.

Sure, just keep moving the goalposts. It's not a "real AI" because it can't take over the US military command and kick off WW3 and finish the survivors off with killer robots yet!

It can however select a girls school as a military target, and did.

AI seems to have been involved in the targeting process, so partial truth.

There may have been outdated information which incorrectly associated the prior use of the site "as either a factory or arms depot". See Wikipedia:

https://en.wikipedia.org/wiki/2026_Minab_school_attack#Analy...>

Citing "Iranian school was on U.S. target list, may have been mistaken as military site" (March 11, 2026) https://www.washingtonpost.com/national-security/2026/03/11/...>.

I'm not drawing conclusions one way or the other, but there do seem to have been multiple factors at play, and assigning full responsibility to use of AI seems suspect.

Which isn't the same as saying AI isn't at fault; e.g., an AI might challenge a dated assessment of a prospective target's role or status, as might a human-in-the-loop target assessment team and process.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#156

Post by [deleted] »

vidarh wrote: post #151
jacobgold wrote: post #143

Earlier quoted context omitted.

That's a fair point. I agree that the behavior can look similar even when the mechanism is different. But in practice, all the analogies I've seen are in fact superficial, including this one. The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

Most of the time when I see an LLM abruptly switching languages it usually continues with something directly related to whatever triggered it. I'm sure there are other failure modes where it may change topics too, just like humans also regularly digress when triggered by certain words etc.

[deleted]

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#157

Post by hbs18 »

TeMPOraL wrote: post #36
jacobgold wrote: post #18

Earlier quoted context omitted.

You can call your little doggy "AI" if it makes you happy. But when you call something "AI" and it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of.

That's a bad and tired example as it confuses people just as much as it does (or rather, did?) LLMs.

I never understood what the walk to car wash thing was supposed to prove. Was it supposed to be something to blow normies' minds with on social media? Woah dude, so like, chat gpt is not actually smart? That's crazy dude.

That experiment "proved" that LLMs are statistical text generators without a concept of meanings of words. Which is the same thing as "proving" that there aren't a million tiny humans inside your laptop doing the CPU's work by hand.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#158

Post by lproven »

Kim_Bruning wrote: post #152
lproven wrote: post #121

Earlier quoted context omitted.

> The real question is how useful they are No, it is not. > and the answer "not at all" increasingly requires flat-earth levels of denial. No, it does not. For me, after ~25 years in the skeptics movement, I think the parallels with supplementary, complementary and alternative medicine are most useful. I choose that term intentionally: its initials are S.C.A.M. and that's exactly what it is. As Tim Minchin and Alan K…

> The reason AI works is the reason that people see the face of Jesus in slices of toast, as someone said recently. Apparently my unit tests can see faces in slices of toast.

Good for you.

Now, shall we discuss the ecological and commercial cost of that?

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#159

Post by dofm »

dTal wrote: post #154
dofm wrote: post #140

Earlier quoted context omitted.

Yeah, it did. I don't think this is really in doubt. Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc. It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely th…

Looks more like standard decision-washing finger pointing to me. "AI did it" is both hypey and also conveniently distracts from an uglier reality: >Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike. >The building in Minab had be…

I am in no way absolving the people who set up the AI-powered tool that apparently acted without their oversight. But they set it up, they let it choose, and they didn’t countermand it. So the AI did, effectively, make the decision. And the suppliers of AI technology to these people should be on the hook as a result.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#160

Post by peterashford »

lproven wrote: post #122
peterashford wrote: post #107

Earlier quoted context omitted.

The field has been called Artificial Intelligence for what, 60 plus years now. Why is it a problem now?

Because we've spent something like 2 trillion dollars on it, as we hurtle into global climate collapse and WW3.

what has that got to do with what we name the field?
Post reply on HN