Live data from Hacker News

Statement on AI Risk

safe.ai

871–880 of 964 posts

Re: Statement on AI Risk

#871

Earlier quoted context omitted.

The median person in field thinks 5-10%, not 2-5%, and median timelines are shorter than 40 years. But this is all a distraction, since unaligned ASI is the default case absent significant efforts (that we aren't currently making), and trying to evaluate risk by averaging out the views of people who haven't actually explored the arguments very carefully (vs. just evaluating the arguments yourself) is a doomed effort.

> The median person in field thinks 5-10%, not 2-5% The median person in the study here, under a particular definition was 5-10%, other comparable studies have found 2%, and similar questions using arguably better definitions in the same study found lower percentages. > median timelines are shorter than 40 years. The median person suggested a 50% chance in 39 years. > since unaligned ASI is the default case I challen…

I work in the space (doing tech stuff that isn't direct research). The best argument I've seen for alignment by default is something like "morality comes from training data, and therefore the fact that LLM training data sets contain human moral intuitions will mean that an ASI stemming from such a training regime will share enough human values" (Quintin Pope believes something like this, as far as I can tell), which is deeply unconvincing, since it contradicts the evidence we _do_ have from human & animal value formation.

Happy to entertain other arguments that alignment-by-default is reasonable; most arguments I've seen are much worse than that one. I haven't seen many people make an active case for alignment-by-default, so much as leave open a whole bunch of swath of uncertainty for unknown unknowns.

Re: Statement on AI Risk

#872

Earlier quoted context omitted.

In her field doesn't mean that's what she researches, LLMs are loosely in her field but the methods are completely different. Computational linguistics != deep learning. Deep learning does not directly use concepts from linguistics, semantics, grammars or grammar engineering, which is what Emily was researcing for the past decades. It's the same thing as saying a number theorist and a set theorist are in the same fie…

They are what she researches though. She has published research on them. LLMs don't directly use concepts from linguistics but they do produce and model language/grammar; it's entirely valid to use techniques from those fields to evaluate them, which is what she does. In the same vein, though a self-driving car doesn't work the same way as a human driver does, we can measure their performance on similar tasks.

Hmm I looked into it, and looked at papers/pdfs in google scholar's advanced search with her as an author that mentioned LLMs or GPT in the past 3 years. Every single one was a criticism about how they couldn't actually understand anything (e.g. "they're only trained on form" and "at best they can only understand things in a limited well scoped fashion") and that linguistic fundamentals for NLP was more important.

Good to know my hunch was correct

Re: Statement on AI Risk

#873

Earlier quoted context omitted.

Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

You're a priori writing off my comment as fruitless because of your emotions and not because you actually have given it deep thought and carefully reached the conclusion that social feedback is somehow bad. Also, the notion that "people's work" is inherently worthy of respect is just nonsensical. I do shoddy work all the time. Hell, you just casually dismissed my internet comment work as shallow and told me not to do…

> Don't you think that this is all a bit anti-intellectual?

Quite rich considering what the GP post was.

Re: Statement on AI Risk

#874

Earlier quoted context omitted.

> It seems problematic to conclude there is no real risk purely on the basis of how software companies act. That is not the only basis. Another is the fact their lines of reasoning are literal fantasy. The signatories of this "statement" are steeped in histories of grossly misrepresenting and overstating the capabilities and details of modern AI platforms. They pretend to the masses that generative text tools like Ch…

> Another is the fact their lines of reasoning are literal fantasy. Isn't this also to be expected at this stage of development? i.e. if these concerns were not "fantasy", we'd already be experiencing the worst outcomes? The risk of MAD is real, and yet the scenarios unleashed by MAD are scenarios that humankind has never seen. We still take the the risk seriously. And what of the very real impact that generative AI…

> The risk of MAD is real, and yet the scenarios unleashed by MAD are scenarios that humankind has never seen. We still take the the risk seriously.

Yeah, because nuclear weapons are real and the science behind them is well-understood. Super intelligent AI is not real, and it is nowhere near becoming real. It is a fantasy fueled by science-fiction and wishful thinking.

> And what of the very real impact that generative AI is already having as it exists in production today?

This is a real concern, but it is not what is meant by "existential risk of AI". Losing jobs does not threaten our existence; it just means we'll need to figure out different ways to build society.

> The fact that it's so difficult to define lies at the center of what makes it so risky.

The fact that it's so difficult to define lies at the center of what makes it so profitable for many of these people.

Re: Statement on AI Risk

#875
post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI. This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders. But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit. My predictio…

> My prediction is that there will be copyright enforcement mandated by law in all GPUs.

Oh this is a nifty idea. Hadn't thought of regulation in this manner. Seems like it would be pretty effective too.

Re: Statement on AI Risk

#876
post #324

Earlier quoted context omitted.

Yes, I see a huge problem. Preventing damage is an order of magnitude more difficult than causing it.

There are an order of magnitude more people with orders of magnitude more resources who don't want to die than those who do.

But it just takes one unforeseen covid-ebola variant to slip through the blind spots of protection-oriented people to be ruinous for humanity.

Re: Statement on AI Risk

#877
post #865

Earlier quoted context omitted.

Not attributing anything, just pointing out an alternative explanation to the one that it is given in the comment that it is responding to However, you seem to be attributing super-human capabilities to a whole group of people for some reason

One does not need to be have super-human capabilities to foresee potential risks of the technology one is an expert in. Yes, the current version of AI is not capable of large scale harm by itself yet, but the plausible trajectory is worth warning about. Gordon Moore did make fairly accurate predictions after all.

> One does not need to be have super-human capabilities to foresee potential risks of the technology one is an expert in

You are right. And no one is arguing the contrary

We can all see the risks

The thing is, what do we do about it

Regulating AI will only serve to help the big players and keep others out

Re: Statement on AI Risk

#878
post #841

Earlier quoted context omitted.

There is a way, in my opinion: distribute AI widely and give it a diversity of values, so that any one AI attempting takeover (or being misused) is opposed by the others. This is best achieved by having both open source and a competitive market of many companies with their own proprietary models.

Like giving nuclear weapons to everyone. What could go wrong?

AI isn't nuclear weaponry. It's bits, not atoms. It's more like encryption.

Re: Statement on AI Risk

#879
post #776

Earlier quoted context omitted.

I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI. This is hardly the first time in history a new technological advancement may be used for nefarious purposes. It’s a discussion worth having as AI advances but if [insert evil actor] wants to cause harm there are many cheaper and easier ways to do this right now. To come out and say we need gove…

> I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI. That's what interesting to me. People fearmongering about bioengineering and GMO's were generally dismissed as being anti-science and holding humankind back (or worse, that there opposition to progress meant they had blood on their hands). Yet many of the people who mocked them proved themsel…

Counterpoint: CRISPR only reignited what was already a real fear of reduced difficulty and costs of engineering deadly pathogens.

In fact, what you and GP wrote is baffling to me. The way I see it, biotech is stupidly obviously self-evidently dangerous, because let's look at the facts:

- Genetic engineering gets easier and cheaper and more "democratized"; in the last 10 years, the basics were already accessible to motivated schools and individual hobbyists;

- We already know enough, with knowledge accessible at the hobbyist level, to know how to mix and match stuff and get creative - see "synthetic biology";

- The substrate we're working with is self-replicating molecular nanotechnology; more than that, it's usually exactly the type that makes people get sick - bacteria (because they're most versatile nanobots) and viruses (because they're natural code injection systems).

Above is the "inside view"; for "outside view", I'll just say this: the fact that "lab leak" hypothesis of COVID-19 was (or still is?) considered to be one of the most likely explanations for the pandemics already tells you that the threat is real, and consequences are dire.

I don't know how can you possibly look at that and conclude "nah, not dangerous, needs to be democratized so the Evil Elites don't hoard it all".

There must be some kind of inverse "just world fallacy" fallacy of blaming everything on evil elites and 1%-ers that are Out To Get Us. Or maybe it's just another flavor of the NWO conspiracy thinking, except instead the Bildenbergs and the Jews its Musk, Bezos and the tech companies.

Same is, IMHO, with AI. Except that one is more dangerous because it's a technology-using technology - that is, where e.g. accidentally or intentionally engineered pathogens could destroy civilization directly, AI could do it by using engineered pathogens - or nukes, or mass manipulation, or targeted manipulation, or ... countless other things.

EDIT:

And if you ask "why, if it's really so easy to access and dangerous, we haven't already been killed by engineered pathogens?", the answer is a combination of:

1. vast majority of people not bearing ill intent;

2. vast majority of people being not interested and not able to perform (yet!) this kind of "nerdy thing";

3. a lot of policing and regulatory attention given to laboratories and companies playing with anything that could self-replicate and spread rapidly;

4. well-developed policies and capacity for dealing with bio threats (read: infectious diseases, and diseases in general);

5. this being still new enough that the dangerous and the careless don't have an easy way to do what in theory they already could.

Note that despite 4. (and 3., if you consider "lab leak" a likely possibility), COVID-19 almost brought the world down.

Re: Statement on AI Risk

#880
post #850

Earlier quoted context omitted.

I don't understand this argument (the "Terminator" scenario). AI could conceivably self replicate and evolve as software but it still needs hardware to run on, power, networking etc etc. There's no better way to kill that than to start a nuclear war or engineer some super virus that kills everyone. It's hard to see any scenario where AI could become a dominant force without significant human collaboration. Perhaps so…

Sure. I recommend reading superintelligence by Nick Bostrom. But I think the key failure here, in your thinking, is that you can't conceive of something , and that therefore something can't happen, and you're doing that in the context of things that are smarter than you . You being able to conceive of it is simply not required for it to happen. Also, Colossus: The Forbin Project was a great movie from 1970 on this su…

Superintelligence and Life 3.0 seem to come up as recurring references in the discussion. I've only read synopses of both but frankly I find the argument that melevolent AI "escape" could occur without being noticed and thwarted a bit far fetched.

There's a good counterargument here[1] that seems reasonable: "One of the features of intelligence explosion that most preoccupies Bostrom and Yudkowsky is that it’s not a problem that we get to have many attempts at. In the Terminator movies, humans don’t get to approach a newly self-aware Skynet and request a do over. One minute Skynet is uncomplainingly complying with all human directives. The next, it’s nuking us. I suspect that we are likely to have plenty of opportunities for do overs in our attempts to make autonomous AIs. Autonomy is not an all-or-nothing proposition. The first machine agents are likely to be quite clumsy. They may be capable of forming goals in respect of their world but they won’t be particularly effective at implementing them. This gives us plenty of opportunity to tweak their programming as they travel the path from clumsy to sophisticated agency"

[1] https://jetpress.org/v26.1/agar.htm

Post reply on HN