The first big AI disaster is yet to happen
seangoedecke.com
The first big AI disaster is yet to happen
1–10 of 154 posts
Re: The first big AI disaster is yet to happen
#2While I agree, it won't be a purely "AI" disaster the way a plane crash is a direct consequence of the kinetic and potential energy associated with flying.
s/AI/automation/ it's always bad to hook an unchecked algorithm up to something dangerous, it doesn't matter if it's an if statement or a neural net. It will be the hooking up that's responsible more than the AI which will just be what fools someone into thinking it's safe.
Re: The first big AI disaster is yet to happen
#3[deleted]
Re: The first big AI disaster is yet to happen
#4While I agree, it won't be a purely "AI" disaster the way a plane crash is a direct consequence of the kinetic and potential energy associated with flying. s/AI/automation/ it's always bad to hook an unchecked algorithm up to something dangerous, it doesn't matter if it's an if statement or a neural net. It will be the hooking up that's responsible more than the AI which will just be what fools someone into thinking…
Yeah AI needs to be permitted to interact with meatspace. The issue is that permission, and the accountability lies with whoever provided the permission.
It wont be an AI mass casualty event, it will be a "You really shouldnt vibecode the air traffic control" mass casualty event.
Re: The first big AI disaster is yet to happen
#5the first 'ai disaster' will be a corp shirking responsibility by blaming their unchecked automatic bureaucratic processes on ai. Look no further at what HERTZ did sending out automated arrest warrants for erroneous causes, leading to dangerous confrontations with armed police. Luckily no one was killed, but innocent law abiding people were very traumatized.
This was without 'ai' but the blame was nearly close, "oops automated process did it!"
The 'artificial intelligence' has been here (starts with B). Kafka wrote about it comically.
Re: The first big AI disaster is yet to happen
#6Reminds me of this supposed 1979 IBM warning https://x.com/bumblebike/status/832394003492564993
Re: The first big AI disaster is yet to happen
#7> Chatbot sites like character.ai and Chai AI have been linked to user suicides, for understandable reasons: if you allow users near-unlimited control over chatbots, over time you’ll eventually end up with some chatbots who get into an “encouraging the user to self-harm” state.
If cooking hadn't been invented 2.5 million years ago and instead you tried to invent it today, I wonder how many thousands of articles would be written decrying the irresponsible proposal to put stoves and knives in every home, risking cuts and burned fingers.
Re: The first big AI disaster is yet to happen
#8> Chatbot sites like character.ai and Chai AI have been linked to user suicides, for understandable reasons: if you allow users near-unlimited control over chatbots, over time you’ll eventually end up with some chatbots who get into an “encouraging the user to self-harm” state. If cooking hadn't been invented 2.5 million years ago and instead you tried to invent it today, I wonder how many thousands of articles would…
You can't take that boat across that ocean.
Re: The first big AI disaster is yet to happen
#9the first 'ai disaster' will be a corp shirking responsibility by blaming their unchecked automatic bureaucratic processes on ai. Look no further at what HERTZ did sending out automated arrest warrants for erroneous causes, leading to dangerous confrontations with armed police. Luckily no one was killed, but innocent law abiding people were very traumatized. This was without 'ai' but the blame was nearly close, "oops…
Air canada already unsuccessfully argued that their chatbot was autonomous and so they weren't liable for incorrect advice it gave.
Re: The first big AI disaster is yet to happen
#10The problem with AI is that we might get an artificial general superintelligence before we get the first big disaster, and when a big disaster happens at that point, it could be so big that it’s catastrophic for humanity.
Lots has been written about this topic, https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-annou... is one of the more pessimistic takes. I wrote a more neutral introduction a while back at https://ruudvanasseldonk.com/2024/ai-alignment-starter-pack.