Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Remarks on AI from NZ

nealstephenson.substack.com

Post reply on HN
Print view View original

101–110 of 117 posts

Re: Remarks on AI from NZ

#101

Post by jerjerjer »

thundergolfer wrote: post #13

> Speaking of the effects of technology on individuals and society as a whole, Marshall McLuhan wrote that every augmentation is also an amputation. Nice to see this because I drafted something about LLM and humans riffing on exactly the same McLuhan argument. Here it is: A large language model (LLM) is a new medium. Just like its predecessors—hypertext, television, film, radio, newspapers, books, speech—it is of obv…

> It took television about 50 years to hit stride and go beyond film, but it got there. Shows like The Wire, The Sopranos, and Mad Men achieved something not replaceable by the movie or the novel.

Can you elaborate on the differences between television and film? Especially considering the examples you cite. I'd agree that live broadcasting is a considerable departure from a film as a medium. Still, the shows you reference are very cinematic - longer, sure, but for me, they are as close to a film experience as possible.

Re: Remarks on AI from NZ

#102

Post by jerjerjer »

vermilingua wrote: post #60

> What people worry about is that we’ll somehow end up with AIs that can hurt us, perhaps inadvertently like horses, or deliberately like bears, or without even knowing we exist, like hornets driven by pheromones into a stinging frenzy. What endlessly frustrates me in virtually every discussion of the risks of AI proliferation is that there is this fixation on Skynet-style doomsday scenarios, and not the much more mu…

I want to see people who used LLMs all their conscious life enter the workforce. It's going to be amazing. I mean soon we'll see people who are essentially raised by LLMs.

Agreed, the takeover would be peaceful and even celebrated.

Re: Remarks on AI from NZ

#103

Post by jerjerjer »

yawnxyz wrote: post #15

> If AIs are all they’re cracked up to be by their most fervent believers, [our lives akin to a symbiotic eyelash mite's existence w/ humans, except we're the mites] like a possible model for where humans might end up: not just subsisting, but thriving, on byproducts produced and discarded in microscopic quantities as part of the routine operations of infinitely smarter and more powerful AIs. I kind of feel like we'r…

Wonder what Ted Kaczynski would make of LLMs.

Re: Remarks on AI from NZ

#104

Post by chubot »

graemep wrote: post #99
chubot wrote: post #44

Earlier quoted context omitted.

Yeah totally, I have read that the total biomass of cows and dogs dwarfs that of say lions or elephants Because humans like eating beef, and they like having emotional support from dogs That seems to be true: https://ourworldindata.org/wild-mammals-birds-biomass Livestock make up 62% of the world’s mammal biomass; humans account for 34%; and wild mammals are just 4% https://wis-wander.weizmann.ac.il/environment/weigh…

That is only mammalian biomass, though. > And then when say the Europeans got here, those animals were mostly gone ... their "biomass" just collapsed A lot of species had long been extinct, but the biomass of the remaining ones fell. Megafauna extinctions always follow 1. the mere arrival of humans and 2. agriculture and growth in human populations. Places the humans did not reach until later, kept a lot more megafau…

Yeah I actually meant that the first extinction happened 10,000 years ago when humans first arrived on the continent! Humans arriving is what caused the biomass of animals to collapse

And then Europeans arriving basically finished the job ... that one probably affected the plants more, due to agriculture. (but also the remaining animals)

Yeah New Zealand is a good example.

Re: Remarks on AI from NZ

#105

Post by vonneumannstan »

ViscountPenguin wrote: post #78
vonneumannstan wrote: post #28

Earlier quoted context omitted.

Unless you have a truly bastardized definition of ASI then there is undoubtedly nothing close to it on earth. No corporation or military or government comes close to what ASI could be capable of. Any reasonably smart person can identify errors that Militaries, Governments and Corporations make ALL THE TIME. Do you really think a Chimp can identify the strategic errors Humans are making? Because that is where you woul…

Do we know that Chimps can't identify some subset of human strategic errors? I'm not convinced that's the case. The idea of dumber agents supervising smarter ones seems relatively grounded to me, and forms the basis of OpenAIs old superalignment efforts (although I think that team might've been disbanded?)

Well they seemingly can't effectively combat any coordinated human activity so it's probably fair to say they indeed can't strategize against us effectively.

Re: Remarks on AI from NZ

#106

Post by vonneumannstan »

ayrtondesozzla wrote: post #97
vonneumannstan wrote: post #28

Earlier quoted context omitted.

Unless you have a truly bastardized definition of ASI then there is undoubtedly nothing close to it on earth. No corporation or military or government comes close to what ASI could be capable of. Any reasonably smart person can identify errors that Militaries, Governments and Corporations make ALL THE TIME. Do you really think a Chimp can identify the strategic errors Humans are making? Because that is where you woul…

> Unless you have a truly bastardized definition of ASI then there is undoubtedly nothing close to it on earth. No corporation or military or government comes close to what ASI could be capable of. This is glistening with religious fervour. Sure, they could be that powerful. Just like God/Allah/Thor/Superman could, too. I've no doubt that many rationalist types sincerely care about these issues, and are sincerely wor…

>This is glistening with religious fervour. Sure, they could be that powerful. Just like God/Allah/Thor/Superman could, too.

It's just straightforwardly following the definition of what an ASI would be, a strongly superhuman mind. Everything follows from that.

Re: Remarks on AI from NZ

#107

Post by vonneumannstan »

kmeisthax wrote: post #34
vonneumannstan wrote: post #29

Earlier quoted context omitted.

That's just because of a failure of imagination. The real world is not like Hollywood, get Terminator out of your head. A real AI take over is likely something we probably can't imagine because otherwise we would be smart enough to thwart it. It's micro drones injecting everyone on earth with a potent neurotoxins or a mirror virus that is dispersed into the entire atmosphere and kills everyone. Or its industrial AIs…

Microdrones and mirror life are still highly speculative[0]. Industrial waste heat is a threat to both human and AI (computers need cooling). And furthermore, those are harms we know about and can defend against. If AI kills us all, it's going to be through the most boring and mundane way possible, because boring and mundane is how you get people to not care and not fight back. In other words, the robot apocalypse wi…

>If AI kills us all, it's going to be through the most boring and mundane way possible, because boring and mundane is how you get people to not care and not fight back.

Human disempowerment is a different thing vs extinction I'd argue.

Anyway the boring human extinction scenarios we come up with probably just aren't close to what might actually happen but our lack of imagination doesn't make use safe.

>[0] I am not convinced that a mirror virus would actually be able to successfully infect and reproduce in non-mirror cells. The whole idea of mirror life is that the mirrored chemistry doesn't interact with ours.

Whether this or any other scenario humans come up with are actually feasible is completely irrelevant to the doom argument. People however hyperfocus on these things like why an ASI couldn't possibly build Nanobots or whatever they fixate on and it's just irrelevant to the core argument.

Re: Remarks on AI from NZ

#108

Post by vonneumannstan »

cadamsdotcom wrote: post #64
vonneumannstan wrote: post #24

It's a nice article but Neal like many others falls into the trap of seemingly not believing that intelligences vastly superior to Humans' across all important dimensions can exist and competition between minds like that almost certainly ends in Humanity's extinction. "I am hoping that even in the case of such dangerous AIs we can still derive some hope from the natural world, where competition prevents any one speci…

> almost certainly ends in humanity’s extinction. The Culture novels talk about super intelligent AIs that perform some functions of government, dealing with immense complexity so humans don’t have to. Doesn’t prevent humans from continuing to exist and being quite content in the knowledge they’re not the most superior beings in the universe. Why do you believe human extinction follows from superintelligence?

>The Culture novels talk about super intelligent AIs that perform some functions of government, dealing with immense complexity so humans don’t have to. Doesn’t prevent humans from continuing to exist and being quite content in the knowledge they’re not the most superior beings in the universe.

The minds in the Culture are formed with the explicit goal to maximise individual flourishing and minimise coercion, this is the reason why Humanity are treated the way they are in the Culture. It's not a base fact that Superintelligences would be so benevolent to other species and in fact they are probably hostile by default. The point is that Minds in the Culture are aligned.

I'm a fan of The Culture universe and the system the Minds use seems like one of the only ways for biological humans to have any relevance in the future.

>Why do you believe human extinction follows from superintelligence?

No, it follows from building unaligned superintelligence which is what we are seemingly on track to build, potentially in the near future.

Re: Remarks on AI from NZ

#109

Post by vonneumannstan »

vonneumannstan wrote: post #108
cadamsdotcom wrote: post #64

Earlier quoted context omitted.

> almost certainly ends in humanity’s extinction. The Culture novels talk about super intelligent AIs that perform some functions of government, dealing with immense complexity so humans don’t have to. Doesn’t prevent humans from continuing to exist and being quite content in the knowledge they’re not the most superior beings in the universe. Why do you believe human extinction follows from superintelligence?

>The Culture novels talk about super intelligent AIs that perform some functions of government, dealing with immense complexity so humans don’t have to. Doesn’t prevent humans from continuing to exist and being quite content in the knowledge they’re not the most superior beings in the universe. The minds in the Culture are formed with the explicit goal to maximise individual flourishing and minimise coercion, this is…

I think broadly my problem with your position is that it amounts to wishful thinking. Theres no reason to think we end up in a Culture future by default, quite the opposite. Theres many reasons to think the default for building superintelligence is bad for humanity and probably bad for the entire universe.

Alignment by default is to me a pipe-dream so if we want a future for humanity we need to forcefully fight for it.

Re: Remarks on AI from NZ

#110

Post by vonneumannstan »

tuatoru wrote: post #71
vonneumannstan wrote: post #24

It's a nice article but Neal like many others falls into the trap of seemingly not believing that intelligences vastly superior to Humans' across all important dimensions can exist and competition between minds like that almost certainly ends in Humanity's extinction. "I am hoping that even in the case of such dangerous AIs we can still derive some hope from the natural world, where competition prevents any one speci…

That's quite bold, accusing a successful science ficiton author of lacking imagination.

Even for the brightest minds, maybe especially for them, it’s incredibly difficult to imagine the possibility of something much smarter than all of humanity, let alone oneself.
Post reply on HN