Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

201–210 of 374 posts

Re: Among the A.I. doomsayers

#201

Post by hollerith »

bglazer wrote: post #113
jprete wrote: post #89

Earlier quoted context omitted.

How are you planning to prevent VCs and startups from giving the AI as many robots as it wants?

That’s a political question. It would require maintaining popular resistance to giving over all critical segments of the economy to AI. Of course that would be difficult, given the short term gains that would bring, but people seem pretty good at hanging on to ideology at their own expense.

Long ago, people a lot like you were saying we should go ahead with AI research, but we shouldn't do anything dangerous like giving the AI access to the internet or giving it the ability to write and run computer programs. (Giving it the ability to run code was back then considered dangerous because it might create an AI with the same goals it has, only more capable, then that second AI might use its novel capabilities to create an even more capable third AI, and so on.)

There was also the idea of building a question-answering machine that just answers questions, but doesn't plan or try to achieve any goals, so it would be safe (the theory went). But notice that one of the early things built on top of GPT was Auto-GPT, which tries to achieve goals (set it by the user) and if there was any "popular resistance" to Auto-GPT I am not aware of it.

But I guess you would reply that those were obviously bad ideas, whereas your idea is a good idea -- your idea of hoping that no one gives a dangerously-capable AI access to any robots. Not only it is a good idea (according to you) but it such an obviously good idea that popular resistance will be galvanized as soon as someone tries to give any AI that might be very capable access to any robots.

Re: Among the A.I. doomsayers

#202

Post by giantrobot »

reducesuffering wrote: post #157
i5heu wrote: post #151

Earlier quoted context omitted.

They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.

What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?

We don't give AP tests to humans to determine whether or not they are in fact an intelligent being. We give them to humans to show that over the course of about a decade and a half that human went from not realizing it had hands to ingesting a bunch of knowledge and had the capability to recall it answer questions.

ChatGPT was trained on hundreds of terabytes of information. I would hope that with all that effort it could answer AP test questions. That doesn't make it intelligent.

Re: Among the A.I. doomsayers

#203

Post by statuslover9000 »

antod wrote: post #162
sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

Part of me wonders if these people are intentionally framing the debate around ethics and potential risks as longer term extinction level problems to distract from the nearer term damage caused by them accelerating the economic inequality of the AI have-nots while they make themselves even richer.

The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species, unless we find a new economic system with human value at its core.

Re: Among the A.I. doomsayers

#204

Post by halflings »

reducesuffering wrote: post #150
i5heu wrote: post #146

Earlier quoted context omitted.

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.

Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"

Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble.

Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "donate money effectively" sense), but the movement has now morphed into a death cult obsessed by things like:

* OMG we're all going to die any time now (repeated every year since circa 2018)

* What is your pDoom? What are your timelines? (aka: what is your totally made up number that makes you feel like you're doing something rational/scientific)

I'm deep in the weeds w/ LLMs, e.g. I probably finetune an average of 1 model a day, and working with bleeding edge models... and AI safety just sounds so silly. Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when gradient descent was invented.

Re: Among the A.I. doomsayers

#205

Post by _y5hn »

toomuchdocs32 wrote: post #136
matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

Long time ago I was reading translated accounts of Rwandan Hutu's that had participated in the 1993 genocide. One in particular had stuck with me; one account of a man that had murdered his childhood friend. As he described it, standing there having gutting and dismembers the man he had grown up side by side with, he had a sense of exhilaration. With all the wealth he thought of the wealth he had now, a tin roof, cat…

Compassion is like a muscle that can be learned to stretch. However, the AI alignment problem is parallel to the human-alignment problem.

Re: Among the A.I. doomsayers

#206

Post by halflings »

reducesuffering wrote: post #155
tivert wrote: post #148

Earlier quoted context omitted.

So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman ( https://blog.samaltman.com/the-merge ) I would say it is.

Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people! Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of c…

This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yudkowsky also believes in, FYI)

Re: Among the A.I. doomsayers

#207

Post by fragmede »

giantrobot wrote: post #202
reducesuffering wrote: post #157

Earlier quoted context omitted.

What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?

We don't give AP tests to humans to determine whether or not they are in fact an intelligent being. We give them to humans to show that over the course of about a decade and a half that human went from not realizing it had hands to ingesting a bunch of knowledge and had the capability to recall it answer questions. ChatGPT was trained on hundreds of terabytes of information. I would hope that with all that effort it…

what's your definition of intelligence that you're using here. there are various takes on what it means, so I'd like to know your take on it.

That it's able to spit out essays makes it something, intelligence is too loosely defined to describe it for everybody, so we need to develop new words and meanings and new language for what it does. It doesn't think, but it does dot products to tensors to get create its output.

Re: Among the A.I. doomsayers

#208

Post by mitthrowaway2 »

ToucanLoucan wrote: post #200
mitthrowaway2 wrote: post #198

Earlier quoted context omitted.

> We don't even know if AGI is possible. My response to a very similar statement is here: https://news.ycombinator.com/item?id=39612641 > That's just... not how it's going to be done. I agree. I think there are some critical ingredients missing. Obviously the weights need to be able to update to new data in a semi-online fashion, for example. But I think there are less ingredients missing than there were a decade ago…

> Reading this is like hearing "there is no evidence that heavier-than-air flight is even possible" being spoken, by a bird. This is a vivid bit of rhetoric to underscore this point, but if you think about it for any length of time it starts to fall apart really, really quickly. The Wright brothers and the dozens if not hundreds of inventors forgotten who came before them drew upon the physics of what they observed i…

Maybe by "we don't know if it's possible" you meant "we don't presently have a step-by-step plan to implement AGI using the tools we already have in our toolbox, and no new ones"? If so then I certainly agree.

But when you look at, say, Sora's videos... How certain are you that there's no path between that and the human visual cortex? How certain are you that they aren't solving at least some of the same problems in structurally analogous ways? Given that nature built a visual cortex (more than once!) by applying survival pressure and turning a crank, just how hard can it be? When we apply a bunch of optimization pressure and billions of dollars turning cranks and something eerily similar pops out, that tells me that there's just less magic to our own brain than we thought. Like when spent centuries wondering if our solar system is unique, and then we finally put Kepler up there and it turns out that planets are everywhere you look.

Re: Among the A.I. doomsayers

#209

Post by hyperadvanced »

roarcher wrote: post #168
reducesuffering wrote: post #157

Earlier quoted context omitted.

What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?

I call it an LLM, not AGI. Consider this scenario: - Ask ChatGPT to produce some output meeting certain criteria - ChatGPT responds, but the output does not meet the criteria in one or more obvious ways - Point this out and ask for a corrected version - ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake. - Repeat steps 3 and 4 ad nauseam. I…

I’d say that LLM’s inability to handle DSL’s in particular is an interesting question on the topic of knowledge and language. Should a generally intelligent entity be able to quickly figure out a new dialect with little background? If I were to air-drop you into an extremely foreign nation, how long would it take you to organically decode the local language and be able to meaningfully craft expressions in it? I’m not sure if humans are “generally intelligent” but a common bar for AGI is for it to be able to beat or meet average humans at normal tasks. I’m not sure if something like writing perfect k8s yaml should be a requirement, though I do agree that LLM’s inability to do word puzzles or relatively straightforward math should disqualify them.

Re: Among the A.I. doomsayers

#210

Post by neta1337 »

digitalsalvatn wrote: post #166

ai is going to bring about the singularity. when robots can do everything, no one needs to work, and the humans get to do whatever they want and pursue their passions.

Ah, the passionate binge watchers and recreational drug users, who doesn’t know them. Many people have no passion after work.
Post reply on HN