[deleted]
Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
41–50 of 87 posts
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#42Maybe this is peak AI panic? It seems wild that someone could unironically talk about tools “disempowering” their users. Like, I get it, C disempowers programmers by shielding them from assembly language, and Cuisinarts disempower chefs, and airplanes disempower travelers by not making them walk through each territory. But… isn’t that a pretty tortured interpretation of tool use? Doesn’t it lead to “the Stone Age was…
All of your examples are tools that incrementally improve upon what came before and, importantly, still require the user to have an understanding of and experience with the underlying skill.
The risk being posed here is that AI may not land as an incremental improvement that still requires the user to maintain some understanding of the underlying skill.
We aren’t quite there yet with the current LLMs. You still need to have a base level of knowledge to use them effectively. But if they were just a little bit better, hallucinated just a little bit less, the value of actually knowing things goes way down.
What would the incentive be to learn the underlying skill or area if the LLM can handle things just fine on its own? Why not just let the LLM figure it out and do it? And at that point, it ceases to be a tool and starts to be something you are completely dependent on. That is a risk.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#43I implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code erro…
people already do this with algorithms. IMO AI is just a continuation of that.
AI is genuinely just a rebranding of "algorithms". Like I get it, it's faster and it works by feeding data instead of feeding code... but code is data, data is code etcetc...
Jim Keller said it best, about every 10 orders of magnitude in available computation the paradigm of computing shifts to a higher level of mathematical building blocks. In the beginning we had pure logic. Then came addition and subtraction, then came vectors, then came matrices, and now we're at tensors, he believes the next building block is graphs.
This is citing from memory while I'm sleep-deprived, but I think the general idea holds. Approximately every 10x increase in computation, there's a paradigm shift in what is possible.
But it's just algorithms. Always has been, and still is.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#44Earlier quoted context omitted.
Calling humans a "tool" is metaphorical. Humans are not literally tools.
But they are in the abstract sense of something to employ to achieve a result. That's why consultants are hired, not for their humanity, but for their utility.
But people talking about tools usually mean tools in the concrete sense. Even very complicated tools in the concrete sense, e.g. compilers or CNC mills, only act as multipliers to the operator's ability. A consultant is an agent, not a tool, and this is a better comparison for modern AI. It's an important distinction because an agent can fail or act against your interests through no fault of your own. The ability multiplier is unpredictable and potentially negative.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#45While this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered Sta…
Kevin Kelly was on a recent episode of the Farnham Street podcast and was surprised by how much he anthropomorphized AI. Maybe that's not a surprise to people that know more about him, but it was to me.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#46Earlier quoted context omitted.
How many people have you talked to face to face about various existential risk scenarios? Have you gotten into probabilities and the logic involved? That’s what I’ve done and this is the level of rigor that is table stakes for calculating the cost-benefit over different possible outcomes.
> How many people have you talked to face to face about various existential risk scenarios A decent amount. I started off as a NatSec adjacent staffer and have helped build Cybersecurity and Defense Tech companies, so a lot of my peers and friends are working directly on US-China, US-Russia, Israel-Iran, Saudi-Iran, Saudi-Turkey, India-Pakistan, and India-China relations. These are all relations that could explode in…
>> I think the world is more likely to end due to bog standard conflicts escalating into an actual war. Not some sort of AGI/ASI going skynet
WOPPER, not skynet. You saw War Games right?
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#47Earlier quoted context omitted.
But they are in the abstract sense of something to employ to achieve a result. That's why consultants are hired, not for their humanity, but for their utility.
But people talking about tools usually mean tools in the concrete sense. Even very complicated tools in the concrete sense, e.g. compilers or CNC mills, only act as multipliers to the operator's ability. A consultant is an agent, not a tool, and this is a better comparison for modern AI. It's an important distinction because an agent can fail or act against your interests through no fault of your own. The ability mul…
Well I don't think getting hung up on such definitions will be fruitful. But here is the point i was trying to make: humans, as individuals and as collectives, do indeed have a lot of experience outsourcing intellectual jobs. They do this knowing full well that the "expert" they're employing is not a deterministic box, and may in fact be secretly working against their interests. None of those problems or potential issues is different if the expert is a human or a silicon agent.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#48Obviously scary AI future means we should probably give full regulatory capture to a handful of wealthy individual corporations. You know. For our own safety.
The following is some unstructured thoughts:
I know you are being facetious, but it is not obvious (to me at least) what the best approach is in this scenario where future AI capabilities and use is unclear. I don't know what the correct analogy for future AIs is because they don't exists yet. If AI have some offensive advantage then nuclear weapons might be the right analogy. The world would not be safer if everyone has their own nuclear weapon. If the harmless & helpful personal assistant analogy is correct, then we should democratize. But Biden's mandate that AI training runs of a certain size should be reported to the government (not in any way hindered) was (to me) just obviously good. Such a report would take at most the effort of one person-day (per Dario Amodei) and gives the government some overview of progress.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#49Bureaucratic systems have been able to fail like this for a long time: https://news.ycombinator.com/item?id=17350645
Now we have the tools to make it happen more often.
Re: Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
#50Earlier quoted context omitted.
> How many people have you talked to face to face about various existential risk scenarios A decent amount. I started off as a NatSec adjacent staffer and have helped build Cybersecurity and Defense Tech companies, so a lot of my peers and friends are working directly on US-China, US-Russia, Israel-Iran, Saudi-Iran, Saudi-Turkey, India-Pakistan, and India-China relations. These are all relations that could explode in…
>> I think the world is more likely to end due to bog standard conflicts escalating into an actual war. Not some sort of AGI/ASI going skynet WOPPER, not skynet. You saw War Games right?
Naw, was before my time. I did see Terminator though.
Did Wargames age well? I tried watching a couple iconic 80s movies like Scarface but I just couldn't stand it.