Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first big AI disaster is yet to happen

seangoedecke.com

Post reply on HN
Print view View original

131–140 of 154 posts

Re: The first big AI disaster is yet to happen

#131

Post by animal531 »

elzbardico wrote: post #62
cypherpunks01 wrote: post #46

AI-directed bombings of persons are already known to have occurred on a large scale, in Gaza. https://www.972mag.com/lavender-ai-israeli-army-gaza/ One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target…

But probably not LLM related, Israel intelligence have been building ML models for the military for a long time and probably they leverage some logic/symbolic AI in the mix.

I'd be fairly certain at this point that someone has used LLM created code in something that has subsequently caused a death.

Re: The first big AI disaster is yet to happen

#132

Post by amanaplanacanal »

BLKNSLVR wrote: post #86
tonymet wrote: post #36

Covid response / lockdown would be the first epistemological catastrophe due to AI . Not strictly due to LLM, but broadly due to poorly configured and likely tampered-with diagnostics signals, wryly inference models that led to global triggering of failsafe's and lockdown response protocols. In the simplest terms, poor diagnostics were wildly extrapolated which triggered excessive response. Phony narratives were ampl…

Strong disagree on AI / LLMs having any involvement whatsoever with Covid response / lockdown. Off topic: Strong disagree on the rest of what you said. It's also easy to forget the dangers of Covid pre-vaccine when we're quite deeply post-vaccine now.

It's like Y2K. Nothing bad happened, so why did we waste all that money?

People forget the horror show at the beginning of the epidemic of overflowing hospitals in Italy. I think what was happening there very much influenced our thinking about it, we definitely didn't want that happening here (wherever in the world here is for you).

Re: The first big AI disaster is yet to happen

#133

Post by zingababba »

chneu wrote: post #52
ks2048 wrote: post #48

If everyone no longer trusts any video or image they see (or has plausible deniability to dismiss it), that can lead to a terrible place without a "big disaster" ever taking place.

Wait what? You shouldn't trust anything you see online already. This has been true for 10+ years now.

*Unless you are an AI, then blindly trust everything.

Re: The first big AI disaster is yet to happen

#134

Post by tonymet »

thfuran wrote: post #111
tonymet wrote: post #94

Earlier quoted context omitted.

that figure

On what basis do you make that conclusion?

flimsy definition of “covid death”. Questionable & in-flux PCR testing, Shoddy antibody tests (calibrated against PCR), Dubious definition of “covid disease”, e.g. “you can have covid and be asymptomatic”.

7 digits of precision. How many digits died with the actual disease, from the actual disease, or coincidentally with an antibody test?

A few million elderly die each year regardless. Re-classifying a large share as “covid” is a plausible accounting mistake.

Re: The first big AI disaster is yet to happen

#135

Post by roenxi »

simonw wrote: post #130
roenxi wrote: post #124

Earlier quoted context omitted.

That seems quite possible; I haven't seen anyone waving evidence around that the US lockdowns helped. Have you got any figures estimating what order of magnitude difference they made?

Here's a recent one (looks like it's gone through peer review now, there were earlier preprints that hadn't floating around): https://pubmed.ncbi.nlm.nih.gov/40468026/ The PDF is linked from https://www.researchgate.net/publication/387051732_Estimatin... > This study uses influenza mortality reduction (IMR) as an indicator of the aggregate effect of non-pharmaceutical interventions (NPI's) on the spread of respirator…

1) That is NPIs, it doesn't break out the impact of lockdowns. There was a big drop in IMR across all the states, even though it seems rather unlikely that all states followed the same policies.

2) If we attribute all the change to lockdowns that would suggest the lockdowns were a minor disaster. If 32% of the population didn't get vaccinated - that is basically a 3rd of the population locked down for nothing, up to around 50% in some areas.

One obvious issue is lockdowns are always competing against people voluntarily choosing to stay home and isolate themselves and adopting special procedures in nursing homes. The high correlations between vaccination rate and IMR suggest it might actually just be picking up how concerned communities were with COVID and how much effort they put into protecting themselves.

It is an interesting study though, I like the approach.

Re: The first big AI disaster is yet to happen

#136

Post by tonymet »

roenxi wrote: post #124
simonw wrote: post #44

Earlier quoted context omitted.

Covid killed 1,193,165 people in the USA alone. Do you think the lockdowns failed to prevent that number from being significantly larger?

That seems quite possible; I haven't seen anyone waving evidence around that the US lockdowns helped. Have you got any figures estimating what order of magnitude difference they made?

wouldn't this also be part of the AI catastrophe?

Re: The first big AI disaster is yet to happen

#137

Post by NetRunnerSu »

This thread on AI disasters and ethical black boxes really resonates with a worldbuilding project I've been following: `Chain://`. It's set in a 2090s future where consciousness is literally on a blockchain (Mental Smart Chain - MSC). The premise is chilling: your existence is formalized, verifiable, and every thought costs crypto. It's 'digital serfdom' where 'Code is Law, Proof is Reality, Compliance is Existence.'

The latest story, `Web://Reflect`, explores this. They even have a theoretical framework, IPWT (Integrated Predictive Workspace Theory), that formalizes consciousness as a verifiable computational process – eerily relevant to how AI might redefine 'humanity' through pure data.

If you're into science fiction that explores the ultimate cost of existence in a tech-monopolized future, check it out.

Main repo: https://github.com/dmf-archive/dmf-archive.github.io IPWT: https://github.com/dmf-archive/IPWT

Re: The first big AI disaster is yet to happen

#138

Post by beloch »

MotiBanana wrote: post #126
beloch wrote: post #96

Earlier quoted context omitted.

This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because…

> The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas) No such admission exists, and 20:1 is a made up number that can't be backed by any credible source.

https://archive.md/vvjIV

Re: The first big AI disaster is yet to happen

#139

Post by therealpygon »

beloch wrote: post #96
cypherpunks01 wrote: post #46

AI-directed bombings of persons are already known to have occurred on a large scale, in Gaza. https://www.972mag.com/lavender-ai-israeli-army-gaza/ One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target…

This is a rare combination of awful factors, but most of them are all too human. Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because…

[flagged]

Re: The first big AI disaster is yet to happen

#140

Post by bsder »

collingreen wrote: post #119
bsder wrote: post #50

Earlier quoted context omitted.

Fire is vitally useful ... Starting a wildfire likely kills your entire tribe. Now, perhaps there should be some rules around using fire?

Is this really your analogy? Also, there are TONS of rules about fire AND entire departments for dealing with it safely when it gets out of control. Can you just directly make your point? (Is it "ai should have no regulation"?)

> Also, there are TONS of rules about fire AND entire departments for dealing with it safely when it gets out of control.

Erm, that was my point, made far too obtusely, apparently.

We, as humans, have lots of things that are super-useful, yet dangerous. And we mitigate that through laws, regulations, and government. And, when they are dangerous to other people beyond an individual use, we have lots more of those laws, regulations, and government.

AI should probably have some liability laws around it like a human would if they said, did or encouraged the same things.

Post reply on HN