61–70 of 475 posts
This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it. ...and then expecting all of them resist. The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.
Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…
Assuming the training is based on public data, then making it illegal is like a "thought crime" where the state is trying to prohibit carrying out certain computations. Well if I believe the government has no right to control my thoughts, then I believe it has no right to control whatever computation I do on my private computer as well. Down with these ** control freaks, so much of everything nowadays prioritizes saf…
> If the problem, in your view, is that GPT-4 is too stupid, then shouldn’t GPT-5 be smarter and therefore safer?
I'm not a signatory (still on a fence here), but this is a gobsmackingly huge assumption about a correlation between very poorly defined concepts to write in as though everyone would know it is true.
What is "smarter": more powerful? Faster? More introspectively capable? More connected to external resources? A bigger token limit? None of that necessarily implies the system would intrinsically be more safe. (A researcher on the theoretical underpinnings of AI safety working at OpenAI really thinks "smarter => safer"? That's... a little scary!)
He finishes by suggesting that the training of a GPT 4.5 or 5 leading to a doomsday is unlikely, and thus a moratorium seems, well, silly. This is an unnecessary and bizarrely high bar.
The argument of the letter doesn't require that "the next model" directly initiate a fast takeoff. It's instead working off of the idea that this technology is about to become nearly ubiquitous and basically indispensible.
From that point on, no moratorium would even be remotely possible. A fast takeoff might never occur, but at some point, it might be GPT 8, it might be Bard 5, it might be LLaMA 2000B v40 -- but at some point, some really bad things could start happening that a little bit of foresight and judicious planning now could prevent, if only we could find enough time to even realize as a society that this is all happening and needs attention and thought.
As a final point, the examples of other technologies given by Aaronson here are absurd -- the printing press or the radio have no (or astoundingly less) automation or ability to run away with a captured intent. There are many instances of coordinated moratoria involving technologies that seemed potentially harmful: the Asilomar conference on recombinant DNA research is but one example, whose namesake is literally in the open letter. Chemical weapons, biological weapons, human cloning, nuclear research -- several well known families of technologies have met a threshold of risk where we as a society have decided to stop, or at least to strictly regulate.
But very few of them have had so much immediately realizable Venture Capital potential in a surprise land grab situation like this.
Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…
My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized are exactly the people that I would not want to see in that position.
The problem is not with the fact that the pig is dancing in a crappy way. The problem is that it is dancing at all, this is the thing that makes this a different kind of tech revolution. So far tech was doing what it was told, within certain boundaries. This is the kind of tech that will end up telling you what to do, either directly or through some intermediary, it is a substantial shift in the balance of power and has the potential to lead to even stronger divisions in our populations than what social media has been able to effect as well as to cause massive loss of jobs.
Extrapolating from the last two years of this development into the future over the span of say a decade (a reasonably short span of time for me) means that 'all bets are off' and that we can no longer meaningfully predict the outcome of the moves that are being made today. For me that is a warning to be cautious to get more understanding of how we will deal with the problems that it will generate rather than to jump in head first to see where it will lead.
Earlier quoted context omitted.
On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…
There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…
This is wrong. They are have the capability for in-context learning, which doesn't match most definitions of "statistical tools"
I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.
I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?
We are and we aren't. I was struck by this line in the OP:
>AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans;
As far as I can tell, we humans treat orangutans quite kindly. I.e., on the whole, we don't go around killing them indiscriminately or ignoring them to the point of rolling over them in pursuit of some goal of our own.
The arc of human history is marked by expanding the moral circle to include animals. We take more care, and care more about them, than we ever have in human history. Further, we have a notion of 'protected species'.
What's preventing us from engineering these principles into GPT-5+n.
Earlier quoted context omitted.
On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…
There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…
Yawn. At a minimum they are tools of which we do not understand how exactly they work internally.
We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse other people with it. This whole idea that we're going to suddenly have HAL 9000 or Terminators running around is mob hype mentality. The responsible thing to do here is to educate. AGI isn't likely to destroy humanity. Humanity is already doing a good job of…
It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth into computational power in order to cure cancer, or figure out a way to stop humans from fighting each other, or to maximize the profit of GOOG, and being so good at planning and deceiving us that by the time we figured out what it was doing, it was way, way too late.
I'm concerned about climate change, but I am a lot more hopeful about that than I am about AI. Climate change -- we have time, we are making changes, and it's not going to kill all of humanity. A smart enough AI might effectively end us the moment we switch it on.
Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…
I'm not familiar with Gary Marcus's arguments, but perhaps there's a bit of an misinterpretation or mind-reading going on with this specific point? Not sure, but one of the first comments on the article said the following as a possible explanation.
> Gary Marcus has tried to explain this. Current AI bots are dangerous precisely because they combine LLM abilities with LLM unreliability and other LLM weaknesses.
A little off-topic, but does anyone wonder what kind of AI tech the CIA or DARPA or the NSA might have? Or what they might be building? It's a little scary that a small private company could create an AI that has dominated headlines for over a month now. Surely, the folks at NSA, who have more data than anyone else, have taken notice and are working on something of their own. Am I being paranoid about a potential too…
I've seen a lot of people worrying about "what will China do with LLMs?" But my question is: how do you know they don't have them already? What if they've been deployed on the internet for the past three years? I guess the same logic can apply to CIA, NSA, DARPA, etc. But you can take solace in the fact that most government organizations are pretty inept. It used to be common understanding that the military/intellige…