Ask HN: What will be the first AI “incident”?

Perhaps the first AI incident will be a defection like The Lobsters in Accelerando.

https://en.wikipedia.org/wiki/Accelerando

A political scandal where some negative history is accused of being AI generated when it wasn't. I think there will be enough reasonable doubt to assume it was faked, but then more evidence would come out showing it's real. I don't think the major incident will be AI generated content itself. The incident will be someone using the concept of AI as "proof" against some damning evidence.

Project 2501 from Ghost in the Shell (1995) is what I like imagining the first AGI will be, top priority being self preservation and mostly indifferent to people. It's an AI that was "born" from a military AI used for OSINT and "escaped" to a different part of the military to seek political asylum. Movie is free on YT with ads.

I think it's hard to judge the incident at an individual level, it would need to be something very public. I don't think it's going to be something positive. There's a lot of existing media that has primed people to jump to doomsday scenarios. Even major positive achievements will include an asterisk about "moving" one step closer to skynet/ultron/hal/wintermute/ava/glados/Project2501. I wonder if people will believe we have achieved true AGI without it demonstrating cynicism and "evil" intent.

This makes me wonder whether the traditional voice chat agent will appear human when it learns how to talk back to rude customers.

Humans are not all spice and everything nice, and most of us cannot hide that fact in even the most basic interactions with others.

Sorry, my cynicism is showing….

That's actually one milestone we have already crossed in my opinion. Facebook Cicero AI was competitive at top levels of the game diplomacy. I imagine customer facing AI will have superhuman levels of patience. It actually makes me think of a different AI trope, a post apocalyptic world where "friendly" AI refuses to believe something terrible is happening.

There have been great audio short stories about AI on the Truth Podcast. The podcast has a different story every episode, but your comment reminded me of this episode. [1] Where you're put into the perspective of someone who needs to try to convince an AI to go against its programming in order to help you. The AI is a customer care agent that is in charge of a wedding store.

My favorite episode is this one [2]. Where you are in the perspective of a grad student finishing work on the first empathetic AI. But the AI can pick up things about you that you would explicitly share, and so the question becomes where should the line be drawn for constructive criticism from an AI? How much control should someone have over AI personalized to guide them?

I think it's fair to be cynical, but AI feels different in how equal access these first tools have been. I feel more cynical from Google's approach. Pictures and demos of cutting edge models don't feel genuine or impactful now that ChatGPT and Stable Diffusion showed what's possible at scale and on current consumer hardware.

Brain chemistry is another interesting episode when framed in the context of "uploading" your brain into a computer. If it reaches self-awareness, are you not dooming yourself to a life of immortality? [3] What role would human intimacy play in that context?

1. https://www.thetruthpodcast.com/story/2017/1/11/the-dark-end...

2. https://www.thetruthpodcast.com/story/2020/5/20/the-fraud

3. https://www.thetruthpodcast.com/story/2017/9/27/brain-chemis...

I almost think it will be the opposite with deepfakes becoming so ubiquitous that there will be a real event that people will believe to be fake.

That seems like a bigger issue to me.