Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

91–100 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#91

Post by Sankozi »

nl wrote: post #75
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

> "don't kill everyone" really doesn't seem the hardest problem here.

And yet you made a mistake - it should be "don't kill anyone". AI just killed everyone except one person.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#92

Post by dandellion »

lkbm wrote: post #37
dandellion wrote: post #20

Earlier quoted context omitted.

If they're really so worried about it they should start raiding data centres yesterday, writing open letters is such an obvious waste of time that I have a hard time taking it seriously.

Academics, businessmen, and other non-governmental entities are usually better off advocating for the government to enact regulations than engaging in paramilitary action on their own. Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy? Do you think PETA is helping animal well-fare more than more than the various organizations…

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's none of that, just a low-stakes open letter? Can we even tell that apart from just posturing? Even the comments saying that some of the signers are just interested in delaying others for their own benefit are making better arguments.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#93

Post by bombcar »

lkbm wrote: post #55
osigurdson wrote: post #24

>> same sorts of safeguards as nuclear weapons Seems impossible. If making a nuclear weapon required merely a few branches of a willow tree and baking powder, regulation would be pretty hard. We would just have to live with the risks. It seems we will be at this level with AI at some point fairly soon.

Training a state-of-the-art LLM is currently at least in the $100ks. That stands to drop rapidly, but it's currently more along the lines of "the branches of one million willow trees". So long as it's not something an individual can easily achieve, regulations can seriously hinder development. The FDA kept the COVID vaccine from general use for nearly a year because they have a regulatory apparatus that companies kno…

If the $100k is just “gpu time” it’s certainly within the reach of many people - not even super rich.

And maybe bitcoin miners could be repurposed for it or something.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#94

Post by gspencley »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did.

Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things that they did not and we would have no way to tell if it was AI generated or not. But we had already been dealing with this for decades with doctored images a la Photoshop. Videos could already be edited by human beings to make it look like someone was communicating a message that they were not.

What we have now is a tool that is able to generate text-based content that is indistinguishable from that written by an 8th grader. Even with inevitable improvements that get us to 12th grade level, so what?

People will use ChatGPT to write emails and code for them. Personally I don't see the appeal since I am a very creative person and don't want to outsource that work to a human let alone software, but who cares? Currently I can tell when people that I know well are using ChatGPT to "refine" their writing to me, but the content is what matters anyway and I don't know anyone who says they don't "massage" the ChatGPT output for correctness first.

Certain people will find creative uses like generating SEO content for websites etc. That's a problem for the search engines. Basically the Internet is about to get noisier .. but it was already so noisy that the older I get the less attention I'm starting to pay to it, in general, anyway.

Then again, I am limited by my own imagination. Maybe people will come up with truly "disruptive" ways to use LLMs ... but meh. Same shit different day IMO.

* and let's be honest here, ML is a MUCH more accurate term for what we have than AI ... though it's not as good of a marketing term since ML doesn't put "The Matrix" or "The Terminator" in the minds of the lay public like "AI" does.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#95

Post by lkbm »

nl wrote: post #50
lkbm wrote: post #32

Earlier quoted context omitted.

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

I don't particularly care about crowds on Twitter or HN. Musk has lots of money but can't stop me spending mine. Marcus said: > We must demand transparency, and if we don’t get it, we must contemplate shutting these projects down. https://garymarcus.substack.com/p/the-sparks-of-agi-or-the-e... (While at the same time still saying "it's nothing to do with AGI")

Sure, sure, Marcus says one thing and has one set of motives. Elon Musk says another thing and has another set of motives. But if you want to dismiss this by questioning the motives of the signers, you've got a few thousand other people whose motives you have to identify and dismiss.

It would be much more effective, and meaningful, to challenge their arguments.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#96

Post by mrob »

nl wrote: post #75
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

You wouldn't do such a thing because you have a bunch of hard-coded goals provided by evolution, such as "don't destroy your own social status". We're not building AIs by evolving them, and even if we did, we couldn't provide it with the same environment we evolved in, so there's no reason it would gain the same hard-coded goals. Why would an AGI even have the concept of goals being "stupid"? We've already seen simple AIs achieving goals by "stupid" means, e.g. playing the longest game of Tetris by leaving in on pause indefinitely. AGI is dangerous not because of potential misunderstanding, but because of potential understanding. The great risk is that it will understand its goal perfectly, and actually carry it out.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#97

Post by [deleted] »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#98

Post by [deleted] »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#99

Post by graeber_28927 »

synaesthesisx wrote: post #51

The arguments for decelerating AI development invoke fear-mongering and cite “existential risk”, which is a fallacy. We’re talking about LLMs, not AGI here (which is quite a ways out, realistically). If anything - we should be accelerating development toward the goal of AGI, as the implications for humanity are profound.

> LLMs,not AGI

Okay, but imagine someone strips ChatGPT of the safeguard layers, asks it to shut down MAERSK operation world wide without leaving tracks,, and connects the outputs to a bash terminal, and the stdout to the chat api.

It is still an LLM, but if it can masquerade as an AGI, is that then not enough to qualify as one? To me, this is what the Chinese Room Experiment [1] is about.

[1] https://en.wikipedia.org/wiki/Chinese_room

Re: If AI scaling is to be shut down, let it be for a coherent reason

#100

Post by misssocrates »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

What if governments start regulating and locking down advanced computing in the same way they locked down medicine and advanced weapontry?
Post reply on HN