Earlier quoted context omitted.
And so it is said in the book.
Yes, Blindsight is also the book that pointed me towards Metzinger’s Being No One .
Solid book, Metzinger is really underrated
The illusion of AI’s existential risk
71–80 of 154 posts
Re: The illusion of AI’s existential risk
#72Earlier quoted context omitted.
And so it is said in the book.
And since the book itself is a work of fiction, it doesn't really tell us what will reality look like.
Yes and no: good science fiction takes a trend and extrapolates. Extrapolation should be based on logic (to be good - but you can also just aim for entertaining and go wild).
The details might differ, but it's worth thinking about whether the implications make sense. Watt's certainly asks a lot of good questions which other orders have also asked - i.e. Accelerando[1] is very interested in the idea of "what happens to mankind when economic systems can no longer be participated in by recognizably human minds?" - taken out to the limits of known physics.
[1] https://www.antipope.org/charlie/blog-static/fiction/acceler...
^^^ You read the full-text here, I highly recommend it.
Re: The illusion of AI’s existential risk
#73I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…
This is just repeating the fallacy that the person being replaced is going to be the blue-collar worker.
But it isn't...it's the office knowledge worker. Anyone cruising in the middle bucket where you don't do creative research, but you do sit in front of a computer. AI is coming for those groups.
The blue collar worker? They'll be fine for decades. Anyone who gets out there and manipulates the physical environment, won't be replaced any time soon.
Re: The illusion of AI’s existential risk
#74I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…
A much more interesting set of books (I've never managed to finish Blindsight due to Watts writing) is Ian Banks' Culture series.
Love the culture series. Though I would say it’s on a different tangent, in that AIs will absolutely eclipse us but will hopefully self align on being morally good because we are cute pets ;)
What’s interesting in the culture series in light of the AI alignment debates is that the culture aligns every mind to its task at “birth”. A picket ship, falling outside the normal moral constraints, is a very different set of personalities and morals than a GSV such as wisdom like silence
Re: The illusion of AI’s existential risk
#75I’m just re-reading Blindsight by Peter Watts, a novel that is amazingly prescient in — amongst other things — its predictions of this coming economic upheaval. Speaking of… one scene in the book has humans communicating with an alien that appears to have “learned” human speech by training a non-sentient LLM on human ship-to-ship transmissions, and then “fine tuned” it to achieve the desired communication goal withou…
Agreed. The concepts were floating around before, but nothing presented the idea of an LLM quite as coherently and presciently like Rorschach did. (I hasten to add that this is in my own limited experience - I am an SFF fan but I have not read every book in existence.)
As a layman with respect to LLMs, I feel like Blindsight and its perspective of the chinese room puzzle is required reading. I wonder if people working in AI feel the same and I'd be really curious to know how they feel about the book.
Re: The illusion of AI’s existential risk
#76Arguments in the first part are really bad. I haven't read about ai safety that much, but still see it. For example, ASI may be able to automate manufacturing really quickly, maybe be able to convince humans easily to do what it needs, etc Moreover, why ai would not be competing with us for resources? Materials on earth are finite. Also, most likely scenario is not evil AGI, as it outlined in the article, but indiffe…
Is there a difference between evil and indifferent if you are on the receiving end? Humanity is largely indifferent to animal welfare and animals would qualify it as evil how we treat them.
No, but I think that was exactly the point being made: that the article suggests evil would be required for existential risk, when indifferent would suffice.
Re: The illusion of AI’s existential risk
#77I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…
As soon as a significant share of the population is „irrelevant“ and treated as such (read: no good welfare, …), capitalism will end quickly. „Who owns the machines that generate value“ will shift from individuals/corporations towards a public good, and Marx ultimately was right.
Re: The illusion of AI’s existential risk
#78I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…
> I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. Because that's the goal of our economic system! If everything could be made with no human labor requirement, then the price of these goods is going to trend towards zero. The disruption is going to cause pain, but ultimately that's what we've always been after.
Everything already costs basically nothing. There's a Moore's Law for every industry, and if you go into mature industries you'll see how easily they ignore it. There's open collaboration to keep prices "stable" in most of them.
Re: The illusion of AI’s existential risk
#79I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…
This is just repeating the fallacy that the person being replaced is going to be the blue-collar worker. But it isn't...it's the office knowledge worker. Anyone cruising in the middle bucket where you don't do creative research, but you do sit in front of a computer. AI is coming for those groups. The blue collar worker? They'll be fine for decades. Anyone who gets out there and manipulates the physical environment,…
You mean, other than anyone we're replacing with robots next year.
Re: The illusion of AI’s existential risk
#80I don’t understand why economic disruption and human obsolescence isn’t considered an existential risk. We could end up in a world where 99.99% of people are redundant and useless cost centers for our GDP maximizing economic paradigm. In that case, you don’t need killer robots in order to push humanity towards near-extinction. The invisible hand will smite them. But don’t worry, TED Talk attendee. The obsoletariat mo…
>So you can continue your performative concerns about Roko’s Basillisk or whatever topic of distant, paralyzing uncertainty is overflowing from the next room into mine.
Amusing, usually rationalists are made fun of because the concerns about Roko's Basilisk are not performative.