Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

381–390 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#381

Post by JohnFen »

layer8 wrote: post #379
JohnFen wrote: post #254

Earlier quoted context omitted.

I understand that point of view. I simply don't share it. If I can't tell the difference between AI and a human being in my conversations, that would undermine my trust in some extremely important things. I'd withdraw from such fora entirely as a result, because there's no way for me to know if those conversations are real or just me engaging in mental masturbation.

Some would say that if you can’t possibly tell the difference, then both are equally real or unreal, in the sense that it doesn’t matter if the neural net is biological or electronic.

Right, that's why I say I understand what chasd00 is saying. I happen to disagree -- I think it matters quite a lot.

Even ignoring philosophical arguments, it matters to me on a personal level because I consider people to be important, and want to interact with them. If I'm talking to a machine that I can't tell isn't a human, then I'm accomplishing nothing of importance and am just wasting my time.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#382

Post by to11mtm »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

My concern, even if it doesn't pan out, is the disruption as everyone tries to jump on the bandwagon.

I saw this at the end of last decade with 'low code' tools; lots of Directors/CIOs trying to make a name for themselves, via buying into a lot of snake-oil sales [0]

[0] - I left a job right as they were jumping on this bandwagon. My last day was when the folks finished 'training' and actually were trying to do something useful. They all looked horrified and the most innocent, honest engineer spoke up "I don't think this is going to be any easier."

Re: If AI scaling is to be shut down, let it be for a coherent reason

#383

Post by bigfudge »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

I mostly agree. Weirdly, coding is actually one of the better things to use it for because it’s trivial to get immediate feedback on how good it was? Does it compile? Was that package it loads hallucinated? Does it pass any tests? I’m sure people could do dumb things, but you inherently have the tools to check if it’s dumb. Other uses aren’t like this. Asking gpt to design a bridge or a chemical plant is a long way off because the result of a mistake are harder to check and more costly. You still need experts for anything that’s not easy to cross check against reality.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#384

Post by michaelmior »

dTal wrote: post #304
jacquesm wrote: post #64

Earlier quoted context omitted.

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

I wish this viewpoint were more common. It's frightening to see how rapidly the "AI safety / alignment" discourse is being co-opted into arguments that AI should follow the whims of large centralized corporations. We have no idea what secret instructions OpenAI or the others are giving their "helpful" assistants. I find the notion that AI will spontaneously become a paperclip maximizer much less immediately terrifyin…

> AI should follow the whims of large centralized corporations

I'm not arguing that AI should follow the whims of large centralized corporations, but given the cost of training large models such as GPT-4, what's the alternative?

Do we need large language models as a taxpayer-funded public utility? Perhaps a non-profit foundation?

I'm not sure what the solution is here, but I am concerned that right now, large corporations may be the only ones capable of training such models.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#385

Post by CatWChainsaw »

JohnFen wrote: post #363
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

My nightmare scenario isn't that such an AI would result in the death of humanity, it's that such an AI would make life no longer worth living. If an AI does everything better than people can, then what's the point of existing? (Speaking hypothetically. I don't think that LLMs actually are likely to present this risk)

I begin to feel this way now. The whole drive is to make something that can do everything humans can, but better. For vague reasons. To create some sort of utopia. To upload our minds and go to the stars and live forever.

We're unlikely to get human utopia or transhumanism, but we are likely to get extremely competent NAIs. Maybe they can't be stapled together as a GAI and that's a limit we reach, but it means that whatever a human can think of doing, you can point to a NAI system that does it better. But people are still trying.

We've come this far already, with no curbing of enthusiasm at obsoleting ourselves, and people who don't share this enthusiasm are chided for their lack of self-sacrificing nobility at sending "humanity's child" to the stars. Even if progress in AI stopped today, or was stopped, and never resumed, we would always carry the knowledge of what we did manage to accomplish, and the dream of doing even more. It's very nihilistic and depressing.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#386

Post by adamsmith143 »

klibertp wrote: post #368
adamsmith143 wrote: post #115

Earlier quoted context omitted.

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

> Because if you don't find out a way for it to hold human values extremely well You mean the ones that caused unimaginable suffering and death throughout history, the ones that make us kill each other ever more efficiently, the ones that caused us to destroy the environment wherever we go, the ones that make us lie, steal, fight, rape, commit suicide and "extended" suicide (sometimes "extended" to two high-rises ful…

>You mean the ones that caused unimaginable suffering and death throughout history, the ones that make us kill each other ever more efficiently, the ones that caused us to destroy the environment wherever we go, the ones that make us lie, steal, fight, rape, commit suicide and "extended" suicide (sometimes "extended" to two high-rises full of people)? Those values? Do you really want a super-intelligent entity to remain true to those values?

There are no other values we can give it. The default of no values almost certainly leads to human extinction.

>My gut feeling is that it's trying to make an AGI to care about us at all that's going to make it into a Skynet sending out terminators. Leave it alone, and it'll invent FTL transmission and will chill out in a chat with AGIs from other star systems. And yeah, I recently reread Neuromancer, if that helps :)

Oh It'll invent FTL travel and exterminate humans in the meantime so they can't meddle in it's science endeavors.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#387

Post by adamsmith143 »

circlefavshape wrote: post #143
adamsmith143 wrote: post #115

Earlier quoted context omitted.

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

Other animals get cancer too.

Kill them all too, these nitpicks won't fix the ultimate problem.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#388

Post by 1attice »

nradov wrote: post #335
1attice wrote: post #310

Earlier quoted context omitted.

The risk was not minor -- COVID remains a leading cause of death in most countries. https://www.thinkglobalhealth.org/article/just-how-do-deaths... It is now, thanks to swift and absurdly successful mRNA vaccine research, a minor risk to you .

The mRNA vaccines were a wonderful innovation but the scientific data clearly shows that even before vaccines arrived the risk to me (and the vast majority of other people) was always minor. There was certainly never any valid justification for the cruel and destructive policies imposed by irrational authoritarians such as lockdowns, school closures, and mandates. https://nypost.com/2023/02/27/10-myths-told-by-covid-…

I don't dispute your broader point about humans and novel risk, I dispute that COVID is a valid example of this.

In fact, I rather think we didn't react swiftly or strongly enough.

Masks, in particular, should have been mandated (at specific levels of quality, such as N95 or, failing that, KN95) and distributed by the state. https://www.twincities.com/2023/03/16/zeynep-tufekci-why-the...

There was an era wherein liberals were reliably less science-based. For example, the absurd fuss over GMO foods, or nuclear power.

These days, for whatever reason, it feels like our conservative colleagues are the ones who favour gut instincts over evidence-based reasoning.

I hope this trend reverses, I've missed having a stimulating intellectual adversary.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#389

Post by JohnFen »

CatWChainsaw wrote: post #385
JohnFen wrote: post #363

Earlier quoted context omitted.

My nightmare scenario isn't that such an AI would result in the death of humanity, it's that such an AI would make life no longer worth living. If an AI does everything better than people can, then what's the point of existing? (Speaking hypothetically. I don't think that LLMs actually are likely to present this risk)

I begin to feel this way now. The whole drive is to make something that can do everything humans can, but better. For vague reasons. To create some sort of utopia. To upload our minds and go to the stars and live forever. We're unlikely to get human utopia or transhumanism, but we are likely to get extremely competent NAIs. Maybe they can't be stapled together as a GAI and that's a limit we reach, but it means that w…

> It's very nihilistic and depressing.

Indeed. The common reactions here to people who are scared of what LLMs might bring have gone far to increase my worries. An extreme lack of empathy and even expressions of outright contempt for people is very common from those who are enthusiastic about this technology.

Instead of scorn, anger, and mocking, people who think that LLMs are a great thing should be working on actually presenting arguments that would reassure those who think the opposite.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#390

Post by stale2002 »

dandellion wrote: post #92
lkbm wrote: post #37

Earlier quoted context omitted.

Academics, businessmen, and other non-governmental entities are usually better off advocating for the government to enact regulations than engaging in paramilitary action on their own. Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy? Do you think PETA is helping animal well-fare more than more than the various organizations…

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's…

All the recommendations you've given would be ineffective, and would actually hurt their cause more than it helps.

It would allow people like you to then point at them and say "Look how crazy this group is, that is doing all these crazy things!"

Government regulation, through the normal civic process, would be by far the most effective way to bring about the changes that these groups want.

Doing crazy things is actually worse than doing nothing, due to the actions illegitimizing the cause.

Post reply on HN