If AI scaling is to be shut down, let it be for a coherent reason
scottaaronson.blog
If AI scaling is to be shut down, let it be for a coherent reason
1–10 of 475 posts
Re: If AI scaling is to be shut down, let it be for a coherent reason
#2> ... I was deeply confused, until I heard a dear friend and colleague in academic AI, one who’s long been skeptical of AI-doom scenarios, explain why he signed the open letter. He said: look, we all started writing research papers about the safety issues with ChatGPT; then our work became obsolete when OpenAI released GPT-4 just a few months later. So now we’re writing papers about GPT-4. Will we again have to throw our work away when OpenAI releases GPT-5? I realized that, while six months might not suffice to save human civilization, it’s just enough for the more immediate concern of getting papers into academic AI conferences.
In other words, the people who wrote and are signing open letters to slow down AI scaling appear to be more concerned with their inability to benefit from and control the dialog around AI scaling than any societal risks posed by these advances in the near term. Meanwhile, to the folks at organizations like Microsoft/OpenAI, Alphabet, Facebook, etc., the scaling of AI looks like a shiny rainbow with a big pot of gold -- money, fame, glory, etc. -- on the other side. Why would they want to slow down now?
Re: If AI scaling is to be shut down, let it be for a coherent reason
#3> On the other hand, I’m deeply confused by the people who signed the open letter, even though they continue to downplay or even ridicule GPT’s abilities, as well as the “sensationalist” predictions of an AI apocalypse.
Says the quantum computing professor turned so-called 'AI safety employee' at O̶p̶e̶n̶AI.com who would rather watch an unregulated hallucination-laden language model run off the rails to be sold as the new AI snake-oil than to actually admit about the huge risks of GPT-4's black-box nature, poor explainability and transparent reasoning methods that is explained in the letter.
Once again, he hasn't disclosed that he is working for O̶p̶e̶n̶AI.com again. I guess he has a large amount of golden handcuffs to defend with another total straw-man of an argument.
Re: If AI scaling is to be shut down, let it be for a coherent reason
#4> ... I was deeply confused, until I heard a dear friend and colleague in academic AI, one who’s long been skeptical of AI-doom scenarios, explain why he signed the open letter. He said: look, we all started writing research papers about the safety issues with ChatGPT; then our work became obsolete when OpenAI released GPT-4 just a few months later. So now we’re writing papers about GPT-4. Will we again have to throw…
>In other words, the people who wrote and are signing open letters to slow down AI scaling appear to be more concerned with their inability to benefit from and control the dialog around AI scaling than any societal risks posed by these advances in the near term.
That's just a joke the author makes. He is not seriously suggesting this is the case.
Re: If AI scaling is to be shut down, let it be for a coherent reason
#5> On the other hand, I’m deeply confused by the people who signed the open letter, even though they continue to downplay or even ridicule GPT’s abilities, as well as the “sensationalist” predictions of an AI apocalypse. Says the quantum computing professor turned so-called 'AI safety employee' at O̶p̶e̶n̶AI.com who would rather watch an unregulated hallucination-laden language model run off the rails to be sold as th…
> Once again, he hasn't disclosed that he is working for O̶p̶e̶n̶AI.com again.
From the article:
> ... and while I’ve been spending the year at OpenAI to work on theoretical foundations of AI safety, I’m going to answer strictly for myself.
(Not to say that OpenAI's name isn't dumb, or that there won't be issues from people directly plugging LLMs into important decisions.)
Re: If AI scaling is to be shut down, let it be for a coherent reason
#6If the government tries to prohibit training these models, I think we should find a way to keep it going somehow. Yes, civil disobedience.
Re: If AI scaling is to be shut down, let it be for a coherent reason
#7I agree with eg Andrew Ng that the letter is anti-innovation. It’ll be interesting seeing people who argue against this letter later hypocritically argue for closing down open source models though.
Re: If AI scaling is to be shut down, let it be for a coherent reason
#8If the government tries to prohibit training these models, I think we should find a way to keep it going somehow. Yes, civil disobedience.
Why?
Re: If AI scaling is to be shut down, let it be for a coherent reason
#9> On the other hand, I’m deeply confused by the people who signed the open letter, even though they continue to downplay or even ridicule GPT’s abilities, as well as the “sensationalist” predictions of an AI apocalypse. Says the quantum computing professor turned so-called 'AI safety employee' at O̶p̶e̶n̶AI.com who would rather watch an unregulated hallucination-laden language model run off the rails to be sold as th…
> Readers, as they do, asked me to respond. Alright, alright. While the open letter is presumably targeted at OpenAI more than any other entity, and while I’ve been spending the year at OpenAI to work on theoretical foundations of AI safety, I’m going to answer strictly for myself.
Re: If AI scaling is to be shut down, let it be for a coherent reason
#10This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it.
...and then expecting all of them resist.
The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.