Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

171–180 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#171
post #155

Earlier quoted context omitted.

We know how they work, that is true. We don't know why they work, because if we could, then we could extrapolate what happens when you throw more compute at them, and no one would have been surprised about the capabilities of GPT-N+1. Also no one would have been caught with their pants down by seeing people jailbreak their models. To illustrate it in a different way: on a mechanistic level, we know how animal brains…

I don't need examples. It's simply how they work. This is why they hallucinate. A LLM is fundamentally a mathematical function (albeit a very complex one, with billions of terms (a.k.a parameters or weights)). The function does one thing and one thing only: it takes a sequence of tokens as input (the context), and it emits the next token(word)[1]. This is a stateless process: it has no "memory" and the model paramete…

Responding here since it won't let me continue your thread any more.

No, there's a fundamental misunderstanding here. I'm not saying the model will tell you the truth about its internal state if you ask it (it absolutely will not.)

I'm saying it has no internal state, and no inner high level processes at all other than it's pre-baked, immutable parameters.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#172

It seems as though Scott just rejects the idea of the singularity entirely. If an AI gets advanced enough to improve itself, it seem entirely reasonable that it would go from laughable to godlike in a week. I don’t know if the singularity is near, inevitable at some point, or really even possible, but it does seem like something that at least could happen. And if it occurs, it will look exactly as what he describes n…

>it seem entirely reasonable that it would go from laughable to godlike in a week.

Not if it has a power cord.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#173
post #153

Earlier quoted context omitted.

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

Other fads have the disadvantage of being easily identifiable and avoidable. But AI chips away at what it means to be human. Now imagine if every comment in this thread is not a real person, but made by a LLM. Get any Truman show vibes?

> Get any Truman show vibes?

No.

It's an interesting thought experiment but it changes nothing. Not for me, anyway. Commenting on these threads is an activity that I do for my own entertainment and "mental flexing." If it turns out that I'm not talking to real people then it doesn't make much of a difference because I don't actually perceive these messages as anything other than pixels on a screen anyway.

I hope that doesn't sound "cold" but I come from a generation that was born before the Internet existed. I was about 10 years-old when we got our first modem-equipped computer and that was still early for most people (1992). Having those experiences early on with "early world-wide-web" meant that even though you knew you were talking to "real people" ... there was no real time chat, or video streaming or voice-over-ip etc. ... and so everyone's messages to each other were always pure text all of the time. And no one dared ever give their real name or identity online. You had no idea who you were talking to. So the web forced you to think of communication as just text between anonymous entities. I never got over that mental model, personally. Maybe a little bit with close friends and family on Facebook. But I'm not much of a social media user. When it comes to twitter and forums ... you all might as well be AI bots anyway. Makes no difference to me!

EDIT (addendum):

It's interesting, the more I think about your comment and my response the more I realize that it is THE INTERNET, still, that has fundamentally changed the nature of what it means to be human and to relate to others.

If you see interactions between people online as "real", meaningful human interactions, no different than relating to people in the physical / offline world, then it must be somewhat disturbing to think that you might be caught up in a "meaningful" relationship with someone who is "faking it." But that reminds me of romance scams.

For 18 years I ran a high traffic adult website and I would have men email me from time to time to share their stories about scammers luring them into relationships using doctored video, images and false identities etc. These men got completely wrapped up in the fantasy that they were being sold and it cost many of them their life savings before they finally realized it was a lie. I felt awful for them and quite sympathetic but at the same time wondered how lonely I would personally have to be to lose my skepticism of what I was being told if an attractive young woman were to express an interest in me out of nowhere.

ML will undoubtedly be used by scammers and ne'er-do-wells as a way to do their misdeeds more efficiently. But I think that the remedy is education. I miss the days when people were a bit weary of what they did, said or uploaded to the interwebs. I don't see the problem with trusting what you read online a little bit less.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#174
post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US. And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up. Surely Elon has no reservations about putting premature s…

>last decade of his business failures

What world are you living in if you think Tesla and SpaceX have been in the past 10 years business failures?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#175
post #73
post #27

Earlier quoted context omitted.

I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?

Putin already has access to the atomic bomb and hasn’t done much with it lately. So perhaps something can limit actors.

The biggest differences here are the potential scale, deniability, etc. "We didn't poison the American water system. It did it to itself!"

Re: If AI scaling is to be shut down, let it be for a coherent reason

#176
post #171
post #155

Earlier quoted context omitted.

I don't need examples. It's simply how they work. This is why they hallucinate. A LLM is fundamentally a mathematical function (albeit a very complex one, with billions of terms (a.k.a parameters or weights)). The function does one thing and one thing only: it takes a sequence of tokens as input (the context), and it emits the next token(word)[1]. This is a stateless process: it has no "memory" and the model paramete…

Responding here since it won't let me continue your thread any more. No, there's a fundamental misunderstanding here. I'm not saying the model will tell you the truth about its internal state if you ask it (it absolutely will not.) I'm saying it has no internal state , and no inner high level processes at all other than it's pre-baked, immutable parameters.

Then you did not read my post carefully enough. The question was not about "internal state" but "inner workings". The model clearly does something. The problem is that we don't know how to describe in human terms what happens between the matrix multiplication and the words it spits out. Whether it has state is completely irrelevant.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#177
post #72

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

Sometime I listen to 90’s AM conspiracy theory radio for fun (Coast to Coast AM). One thing that’s struck me is how much fear there was about the Human Genome Project and designer babies being right around the corner (Gattaca was 1997, for example). Maybe that will come to pass someday. But at the moment, it still seems a long ways off. A lot of groups have some new technology they’re scared of. Tech folks have latch…

The technology for designer babies is here; polygenic embryo selection could do it right now, people just aren't going all the way for various reasons (concern about regulation).

Re: If AI scaling is to be shut down, let it be for a coherent reason

#178
post #75

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

This is known as the orthogonality thesis -- goals are orthogonal to intelligence. Intelligence is the ability to plan and act to achieve your goals, whatever they are. A stupid person can have a goal of helping others, and so can the smartest person on earth -- it's just that one is better. Likewise, a stupid person can have a goal of becoming wealthy, and so can a smart person. The smart person is Jeff Bezos or Bill Gates.

There are very smart people who put all their intelligence into collecting stamps, or making art, or acquiring heroin, or getting laid, or killing people with their bare hands or doing whatever they want to do. They want to do it because they want to. The goal is not smart or stupid, it just is. It may be different from your goal, and hard to understand. Now consider that an AI is not even human. Is it that much of a stretch to imagine that it has a goal as alien, or more, than the weirdest human goal you can think of?

*edit - as in this video: https://www.youtube.com/watch?v=hEUO6pjwFOo

Re: If AI scaling is to be shut down, let it be for a coherent reason

#179
post #56
post #32

Earlier quoted context omitted.

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…

There are researches working on the specific problem of AI safety, and I consider them to be the experts of this field, regardless of the fact that probably no university is currently offering a master's degree specifically on AI safety. Whether one or more of these researchers are in favor of the ban, I don't know.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#180
post #93
post #55

Earlier quoted context omitted.

Training a state-of-the-art LLM is currently at least in the $100ks. That stands to drop rapidly, but it's currently more along the lines of "the branches of one million willow trees". So long as it's not something an individual can easily achieve, regulations can seriously hinder development. The FDA kept the COVID vaccine from general use for nearly a year because they have a regulatory apparatus that companies kno…

If the $100k is just “gpu time” it’s certainly within the reach of many people - not even super rich. And maybe bitcoin miners could be repurposed for it or something.

This may be in the $10^7 category now, but is there any reason to believe it will never be $10^3?

Oddly the most pressing concern is “increased productivity”.

Post reply on HN