Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first big AI disaster is yet to happen

seangoedecke.com

Post reply on HN
Print view View original

121–130 of 154 posts

Re: The first big AI disaster is yet to happen

#121

Post by originalvichy »

ethbr1 wrote: post #113
beloch wrote: post #96

Earlier quoted context omitted.

This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because…

The IDF is using AI to whitewash ethics in the same way genai is using it to whitewash copyright. Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement. And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) ar…

This is the chilling part. When Mossad hunted down Eichmann, some of his defense could be summarized as the cliche of ”I was just following orders”. Now with this tooling, they are learning from the past, and centralizing the ”orders” into a non-human machine, which means only a few people (if ever) would stand trial if they were accused of crimes against humanity: minister of defense, some procurement guy, some technical lead. That’s it.

Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?

Re: The first big AI disaster is yet to happen

#122

Post by Zaylan »

I think the first real AI disaster probably won’t come from some kind of superintelligence. It’s more likely to happen in a very ordinary situation, where a system gives the wrong information, it causes real harm in the real world, and no one notices until it’s too late.

To me, the real risk is people handing over judgment to a system they don’t really understand, and then blaming it when things go wrong.

Re: The first big AI disaster is yet to happen

#123

Post by GJim »

protocolture wrote: post #43
michaelt wrote: post #7

> Chatbot sites like character.ai and Chai AI have been linked to user suicides, for understandable reasons: if you allow users near-unlimited control over chatbots, over time you’ll eventually end up with some chatbots who get into an “encouraging the user to self-harm” state. If cooking hadn't been invented 2.5 million years ago and instead you tried to invent it today, I wonder how many thousands of articles would…

Considering that governments are trying to remove gas stoves from peoples homes for safety reasons yes, you would probably have a hard time introducing them from scratch today.

[citation needed]

The phasing out of gas cooking is discussed in terms of reducing CO2 emissions, not safety.

Re: The first big AI disaster is yet to happen

#124

Post by roenxi »

simonw wrote: post #44
tonymet wrote: post #36

Covid response / lockdown would be the first epistemological catastrophe due to AI . Not strictly due to LLM, but broadly due to poorly configured and likely tampered-with diagnostics signals, wryly inference models that led to global triggering of failsafe's and lockdown response protocols. In the simplest terms, poor diagnostics were wildly extrapolated which triggered excessive response. Phony narratives were ampl…

Covid killed 1,193,165 people in the USA alone. Do you think the lockdowns failed to prevent that number from being significantly larger?

That seems quite possible; I haven't seen anyone waving evidence around that the US lockdowns helped. Have you got any figures estimating what order of magnitude difference they made?

Re: The first big AI disaster is yet to happen

#125

Post by nsonha »

chneu wrote: post #52
ks2048 wrote: post #48

If everyone no longer trusts any video or image they see (or has plausible deniability to dismiss it), that can lead to a terrible place without a "big disaster" ever taking place.

Wait what? You shouldn't trust anything you see online already. This has been true for 10+ years now.

You should not trust things that you can't find evidence for, yes. Imagine not being able to find evidence for any thing, ever, any more.

Re: The first big AI disaster is yet to happen

#126

Post by MotiBanana »

beloch wrote: post #96
cypherpunks01 wrote: post #46

AI-directed bombings of persons are already known to have occurred on a large scale, in Gaza. https://www.972mag.com/lavender-ai-israeli-army-gaza/ One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target…

This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because…

> The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas)

No such admission exists, and 20:1 is a made up number that can't be backed by any credible source.

Re: The first big AI disaster is yet to happen

#127

Post by michtzik »

GJim wrote: post #123
protocolture wrote: post #43

Earlier quoted context omitted.

Considering that governments are trying to remove gas stoves from peoples homes for safety reasons yes, you would probably have a hard time introducing them from scratch today.

[citation needed] The phasing out of gas cooking is discussed in terms of reducing CO2 emissions, not safety.

https://www.cpsc.gov/About-CPSC/Commissioner/Richard-Trumka/...

https://www.cnn.com/2023/01/09/business/gas-stove-ban-federa...

https://www.npr.org/2024/09/13/nx-s1-5003074/climate-gas-sto...

https://www.latimes.com/opinion/story/2023-01-15/gas-stoves-...

Re: The first big AI disaster is yet to happen

#128

Post by ninkendo »

reasonableklout wrote: post #67
LPisGood wrote: post #22

Earlier quoted context omitted.

I don’t think AI is meaningfully taking any jobs. Just like return to office mandates, it is just more cover to do layoffs in less than prosperous economic conditions.

It's not the same as "AI replacing humans", but seems clear that the launch of ChatGPT led to dramatic cuts of NLP researchers. [1][2] For instance, at HuggingFace: > A few days after that, Thom Wolf, who was my manager at Hugging Face and also one of the co-founders, messages me, “Hey, can you get on a call with me ASAP?” He told me that they had fired people from the research team and that the rest would either be…

It’s only natural that a breakthrough new way of doing $thing would lead to dramatic cuts in people who are experts in the old way of doing $thing, right? Doesn’t matter what $thing is. (Tons of examples, automated phone switches replacing telephone operators, online booking sites replacing travel agents, etc etc.)

Re: The first big AI disaster is yet to happen

#130

Post by simonw »

roenxi wrote: post #124
simonw wrote: post #44

Earlier quoted context omitted.

Covid killed 1,193,165 people in the USA alone. Do you think the lockdowns failed to prevent that number from being significantly larger?

That seems quite possible; I haven't seen anyone waving evidence around that the US lockdowns helped. Have you got any figures estimating what order of magnitude difference they made?

Here's a recent one (looks like it's gone through peer review now, there were earlier preprints that hadn't floating around): https://pubmed.ncbi.nlm.nih.gov/40468026/

The PDF is linked from https://www.researchgate.net/publication/387051732_Estimatin...

> This study uses influenza mortality reduction (IMR) as an indicator of the aggregate effect of non-pharmaceutical interventions (NPI's) on the spread of respiratory infections to assess their impact on COVID mortality. [...] A counterfactual model of no NPI's suggests they prevented 850,000 COVID related deaths in the United States.

Post reply on HN