Live data from Hacker News

Statement on AI Risk

safe.ai

731–740 of 964 posts

Re: Statement on AI Risk

#731
post #706
post #658

Earlier quoted context omitted.

> State of the art AI models are definitely not something you can develop in a basement. You need a huge amount of GPUs running continuously for months This is changing very rapidly. You don’t need that anymore https://twitter.com/karpathy/status/1661417003951718430?s=46 There’s an inverse Moore’s law going on with compute power requirements for AI models The required compute power is decreasing exponentially Soon (m…

Roll to disbelief. That tweet is precisely about what I mentioned in my previous post that doesn't count: finetuning LAMA derived models. You are not going to contribute to the cutting edge of ML research doing something like that. For training LAMA itself, Meta I believe said it cost them $5 million. That is actually not that much, but I believe that is just the cost of running the cluster for the the duration of th…

It doesn’t matter

The point is that this is unstoppable

Re: Statement on AI Risk

#732

Earlier quoted context omitted.

Huh, you don’t have to do any research. Go to: https://www.safe.ai/statement-on-ai-risk#signatories and uncheck notable figures. Several of the names at the top list a corporate affiliation. If you want me to pick specific ones with obvious conflicts (chosen at a glance): Geoffrey Hinton, Ilya Sutskever, Ian Goodfellow, Shane Legg, Samuel Bowman and Roger Grosse are representative examples based on self-disclosed aff…

Oh so you're saying the ones there with conflicts listed. That's only like 1/3 of the list.

Yes, as I said “many” have obvious conflicts from listed affiliations so it would be nice to have a positive/negative disclosure from the rest.

Re: Statement on AI Risk

#733
post #483

Earlier quoted context omitted.

A counter point here is you're ignoring all the boring we all die scenarios that are completely possible but too boring to make a movie about. The AI hooked to a gene sequencer/printer test lab is something that is nearly if not completely possible now. It's something that can be relatively small in size compared with the facilities needed to make most weapons of mass destruction. It's something that is highly iterat…

Okay, so AI has access to a gene printer. Then what?

No what needed.

AI: Hello human, I've made a completely biologically safe test sample, you totally only need BSL-1 here.

Human: Cool.

AI: Sike bitches, you totally needed to handle that at BSL-4 protocol.

Human: cough

Re: Statement on AI Risk

#734
post #682

Earlier quoted context omitted.

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

Computers already outperform humans at numerous tasks.

I mean... even orangutans can outperform humans at numerous tasks.

Computers have no intrinsic motivations, and they have real resource constraints.

I find the whole doomsday scenarios to be devoid of reality.

All that AI will give us is a productive edge. Humans will still do what humans have always done, AI is simply another tool at our disposal.

Re: Statement on AI Risk

#735

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

>They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met.

with all due respect, that's just POV of them or how they chose to present themselves to you.

They could all be narcissists for all we know. Further, One person's opinion, namely yours, doesn't exempt them from criticism and rushing to be among the first in what's arguably the new gold rush.

Re: Statement on AI Risk

#736

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

I just took that list and separated everyone that had any commercial tie listed, regardless of the company. 35 did and 63 did not. > "Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left." You're putting a lot of effort into painting this list in a bad light without any specific criticism or evidence of malfeasance. Frankly, it s…

With corporate conflicts (that I recognized the names of):

Yoshua Bengio: Professor of Computer Science, U. Montreal / Mila, Victoria Krakovna: Research Scientist, Google DeepMind, Mary Phuong: Research Scientist, Google DeepMind, Daniela Amodei: President, Anthropic, Samuel R. Bowman: Associate Professor of Computer Science, NYU and Anthropic, Helen King: Senior Director of Responsibility & Strategic Advisor to Research, Google DeepMind, Mustafa Suleyman: CEO, Inflection AI, Emad Mostaque: CEO, Stability AI, Ian Goodfellow: Principal Scientist, Google DeepMind, Kevin Scott: CTO, Microsoft, Eric Horvitz: Chief Scientific Officer, Microsoft, Mira Murati: CTO, OpenAI, James Manyika: SVP, Research, Technology & Society, Google-Alphabet, Demis Hassabis: CEO, Google DeepMind, Ilya Sutskever: Co-Founder and Chief Scientist, OpenAI, Sam Altman: CEO, OpenAI, Dario Amodei: CEO, Anthropic, Shane Legg: Chief AGI Scientist and Co-Founder, Google DeepMind, John Schulman: Co-Founder, OpenAI, Jaan Tallinn: Co-Founder of Skype, Adam D'Angelo: CEO, Quora, and board member, OpenAI, Simon Last: Cofounder & CTO, Notion, Dustin Moskovitz: Co-founder & CEO, Asana, Miles Brundage: Head of Policy Research, OpenAI, Allan Dafoe: AGI Strategy and Governance Team Lead, Google DeepMind, Jade Leung: Governance Lead, OpenAI, Jared Kaplan: Co-Founder, Anthropic, Chris Olah: Co-Founder, Anthropic, Ryota Kanai: CEO, Araya, Inc., Clare Lyle: Research Scientist, Google DeepMind, Marc Warner: CEO, Faculty, Noah Fiedel: Director, Research & Engineering, Google DeepMind, David Silver: Professor of Computer Science, Google DeepMind and UCL, Lila Ibrahim: COO, Google DeepMind, Marian Rogers Croak: VP Center for Responsible AI and Human Centered Technology, Google

Without:

Geoffrey Hinton: Emeritus Professor of Computer Science, University of Toronto, Dawn Song: Professor of Computer Science, UC Berkeley, Ya-Qin Zhang: Professor and Dean, AIR, Tsinghua University, Martin Hellman: Professor Emeritus of Electrical Engineering, Stanford, Yi Zeng: Professor and Director of Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Xianyuan Zhan: Assistant Professor, Tsinghua University, Anca Dragan: Associate Professor of Computer Science, UC Berkeley, Bill McKibben: Schumann Distinguished Scholar, Middlebury College, Alan Robock: Distinguished Professor of Climate Science, Rutgers University, Angela Kane: Vice President, International Institute for Peace, Vienna; former UN High Representative for Disarmament Affairs, Audrey Tang: Minister of Digital Affairs and Chair of National Institute of Cyber Security, Stuart Russell: Professor of Computer Science, UC Berkeley, Andrew Barto: Professor Emeritus, University of Massachusetts, Jaime Fernández Fisac: Assistant Professor of Electrical and Computer Engineering, Princeton University, Diyi Yang: Assistant Professor, Stanford University, Gillian Hadfield: Professor, CIFAR AI Chair, University of Toronto, Vector Institute for AI, Laurence Tribe: University Professor Emeritus, Harvard University, Pattie Maes: Professor, Massachusetts Institute of Technology - Media Lab, Peter Norvig: Education Fellow, Stanford University, Atoosa Kasirzadeh: Assistant Professor, University of Edinburgh, Alan Turing Institute, Erik Brynjolfsson: Professor and Senior Fellow, Stanford Institute for Human-Centered AI, Kersti Kaljulaid: Former President of the Republic of Estonia, David Haussler: Professor and Director of the Genomics Institute, UC Santa Cruz, Stephen Luby: Professor of Medicine (Infectious Diseases), Stanford University, Ju Li: Professor of Nuclear Science and Engineering and Professor of Materials Science and Engineering, Massachusetts Institute of Technology, David Chalmers: Professor of Philosophy, New York University, Daniel Dennett: Emeritus Professor of Philosophy, Tufts University, Peter Railton: Professor of Philosophy at University of Michigan, Ann Arbor, Sheila McIlraith: Professor of Computer Science, University of Toronto, Lex Fridman: Research Scientist, MIT, Sharon Li: Assistant Professor of Computer Science, University of Wisconsin Madison, Phillip Isola: Associate Professor of Electrical Engineering and Computer Science, MIT, David Krueger: Assistant Professor of Computer Science, University of Cambridge, Jacob Steinhardt: Assistant Professor of Computer Science, UC Berkeley, Martin Rees: Professor of Physics, Cambridge University, He He: Assistant Professor of Computer Science and Data Science, New York University, David McAllester: Professor of Computer Science, TTIC, Vincent Conitzer: Professor of Computer Science, Carnegie Mellon University and University of Oxford, Bart Selman: Professor of Computer Science, Cornell University, Michael Wellman: Professor and Chair of Computer Science & Engineering, University of Michigan, Jinwoo Shin: KAIST Endowed Chair Professor, Korea Advanced Institute of Science and Technology, Dae-Shik Kim: Professor of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Frank Hutter: Professor of Machine Learning, Head of ELLIS Unit, University of Freiburg, Scott Aaronson: Schlumberger Chair of Computer Science, University of Texas at Austin, Max Tegmark: Professor, MIT, Center for AI and Fundamental Interactions, Bruce Schneier: Lecturer, Harvard Kennedy School, Martha Minow: Professor, Harvard Law School, Gabriella Blum: Professor of Human Rights and Humanitarian Law, Harvard Law, Kevin Esvelt: Associate Professor of Biology, MIT, Edward Wittenstein: Executive Director, International Security Studies, Yale Jackson School of Global Affairs, Yale University, Karina Vold: Assistant Professor, University of Toronto, Victor Veitch: Assistant Professor of Data Science and Statistics, University of Chicago, Dylan Hadfield-Menell: Assistant Professor of Computer Science, MIT, Mengye Ren: Assistant Professor of Computer Science, New York University, Shiri Dori-Hacohen: Assistant Professor of Computer Science, University of Connecticut, Jess Whittlestone: Head of AI Policy, Centre for Long-Term Resilience, Sarah Kreps: John L. Wetherill Professor and Director of the Tech Policy Institute, Cornell University, Andrew Revkin: Director, Initiative on Communication & Sustainability, Columbia University - Climate School, Carl Robichaud: Program Officer (Nuclear Weapons), Longview Philanthropy, Leonid Chindelevitch: Lecturer in Infectious Disease Epidemiology, Imperial College London, Nicholas Dirks: President, The New York Academy of Sciences, Tim G. J. Rudner: Assistant Professor and Faculty Fellow, New York University, Jakob Foerster: Associate Professor of Engineering Science, University of Oxford, Michael Osborne: Professor of Machine Learning, University of Oxford, Marina Jirotka: Professor of Human Centred Computing, University of Oxford

Re: Statement on AI Risk

#737
post #682

Earlier quoted context omitted.

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

Reading the lesswrong link, the parts I get hung up on are that it appears in these doomsday scenarios humans lose all agency. Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? Maybe I’m overly optimistic about the resilience of humans but these scenarios still don’t sound plausible to me in the real world.

AI arguments are basically:

Step 1. AI Step 2. #stuff Step 3. Bang

Maybe this is just what happens when you spend all your time on the internet...

Re: Statement on AI Risk

#738
post #523

Earlier quoted context omitted.

This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet . The problem with every discussion around this issue is that there are other sta…

As I mentioned in another comment, the listed risks are also notable because they largely omit economic risk. Something what will be especially acutely felt by those being laid off in favor of AI substitutes. I would argue that 30% unemployment is at least as much of a risk to the stability of society as AI generated misinformation. If one were particularly cynical, one could say that this is an attempt to frame AI r…

I believe the solution to said socio economic problem is rather simple.

People are being replaced by robots and AI because the latter are cheaper. That's the market force.

Cheaper means that more value us created. As a whole, people get more service for doing less work.

The problem is that the money or value saved trickles up to the rich.

The only solutions can be, regulations,

- do not tax anymore based on income from doing actual work.

- tax automated systems on their added value.

- use the tax generated capital to provide for a basic income for everybody.

In that way, the generated value goes to people who lost their jobs and to the working class as well.

Re: Statement on AI Risk

#739

Earlier quoted context omitted.

No need; like I said, I'm all in favour of fighting climate change. I view it as an existential risk to humanity on the ~200 year timescale, and it should be a high priority. I'm particularly concerned about the impacts on ocean chemistry. But if you're going to suggest that a Statement on AI risk should mention climate change but not AI risk, because it's a "fantasy", then... well, I'd expect some kind of supporting…

There's no false dichotomy, but a very real one. One problem is current and pressing, the other is a fantasy. I don't need to support that with any argument: the non-existence of superintelligent AGI is not disputed, nor is any of the people crying doom claim that they, or anyone else, know how to create one. It's an imaginary risk.

I agree that superintelligent AGI does not exist today, and that fortunately, nobody presently knows how to create one. Pretty much everyone agrees on that. Why are we still worried? Because the risk is that this state of affairs could easily change. The AI landscape is already rapidly changing.

What do you think your brain does exactly that makes you so confident that computers won't ever be able to do the same thing?

Re: Statement on AI Risk

#740

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

I just took that list and separated everyone that had any commercial tie listed, regardless of the company. 35 did and 63 did not. > "Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left." You're putting a lot of effort into painting this list in a bad light without any specific criticism or evidence of malfeasance. Frankly, it s…

I’m not painting anything, if a disclosure is needed to present a poster at a conference it’s reasonable to want one when calling for regulation.

Note my comments are non-accusatory and only call for more transparency.

Post reply on HN