I think the most plausible AI disaster is going to be of a Therac-25 nature. Some "everyone must use AI" shop will ship an AI-generated feature that some dev signed off on and it might kill people.
The first big AI disaster is yet to happen
71–80 of 154 posts
Re: The first big AI disaster is yet to happen
#72Earlier quoted context omitted.
Cooking has proven its utility a long time ago. AI girlfriends? Let's say, it's still on the fence.
> utility Also safety: Modern cooking usually occurs in a context of safety-measures developed over thousands of years of injuries, deaths, and burnt cities. Some are so pervasive and traditional that we barely notice, like "no fires unless you have a way to extinguish it in a hurry", or "if you wait too long to eat it is becomes dangerous again."
What about the 5 seconds rule?
Re: The first big AI disaster is yet to happen
#73Earlier quoted context omitted.
I don’t think AI is meaningfully taking any jobs. Just like return to office mandates, it is just more cover to do layoffs in less than prosperous economic conditions.
It's not the same as "AI replacing humans", but seems clear that the launch of ChatGPT led to dramatic cuts of NLP researchers. [1][2] For instance, at HuggingFace: > A few days after that, Thom Wolf, who was my manager at Hugging Face and also one of the co-founders, messages me, “Hey, can you get on a call with me ASAP?” He told me that they had fired people from the research team and that the rest would either be…
That’s very odd. You’d think a breakthrough and surge of funding would massive boost NLP research - unless those researchers who not focused on LLMs and the company just wanted to move all resources to that.
Re: The first big AI disaster is yet to happen
#74Earlier quoted context omitted.
AI may prevent lost jobs from coming back, even if it didn't cause the loss. Companies will be able to do more with less.
> Companies will be able to do more with less. I guess everyone on Hacker News should stop developing software then, since almost every piece of software I've ever worked on has allowed companies (and individuals) to do more with less.
In other words, if you can do more with less, you can do EVEN MORE with more!
Re: The first big AI disaster is yet to happen
#75The best comparable would be the first big computer disaster or the first big internet disaster. Do we have consensus on what that would be?
For acute disasters: maybe the Morris worm? https://en.wikipedia.org/wiki/Morris_worm Longer-term there's an argument to be made for political destabilization driven by social media. We are still seeing this play out.
That's a good one.
I always think of https://en.wikipedia.org/wiki/Therac-25
Re: The first big AI disaster is yet to happen
#76[flagged]
I honestly feel that LLMs have been a net negative for me rather than a positive, *especially* with software development. Most of the time I end up coaxing it through a series of prompts, only for there to be some bug in the code it generated for me that takes me longer to debug than if I had just read the relevant docs and examples myself. I’m sure there’s utility now, and I’m sure it’ll (ever more slowly, asymptoti…
Use it for non coding busy work. At my job we rely fully on AI commit messages, and pr descriptions. We can also automatically generate GitHub issues based off the context of a helpdesk ticket. Less clerical busy work is a nice win
Re: The first big AI disaster is yet to happen
#77"Shall we play a game?"
Re: The first big AI disaster is yet to happen
#78Covid response / lockdown would be the first epistemological catastrophe due to AI . Not strictly due to LLM, but broadly due to poorly configured and likely tampered-with diagnostics signals, wryly inference models that led to global triggering of failsafe's and lockdown response protocols. In the simplest terms, poor diagnostics were wildly extrapolated which triggered excessive response. Phony narratives were ampl…
Now I'm wondering if you used an LLM to generate this sci-fi scenario.
Re: The first big AI disaster is yet to happen
#79There's a town in northern Norway where the municipality used AI tools / LLMs to aid their process in restructuring schools. So they were going to close and merge a bunch of schools, and pointed to research that supported their plans. But it turns out that the AI they used, had hallucinated this research. The AI had correctly used relevant researchers, but had hallucinated the work itself. It wasn't until a investiga…
[deleted]
Re: The first big AI disaster is yet to happen
#80[flagged]