Live data from Hacker News

Statement on AI Risk

safe.ai

921–930 of 964 posts

Re: Statement on AI Risk

#921
post #912

Earlier quoted context omitted.

No I'm not arguing that. My point is that if an AI is trying to take over the world we can just turn it off, regardless of power budgets. If it runs on millions of hacked devices, how do you turn it off? The same way any botnet gets turned off: virus scanners clean it up, the C&C servers get taken down, etc. This is not a new problem. The usual response to this is to claim that superintelligence will develop so fast…

So how do you turn off terrorists? I mean we had a global war on terror that we've spent billions of dollars on, and they are still out there? You keep acting like AGI is just going to be some other program. And yes, we do not have AGI yet. No planning, no continuous chain of thinking/learning at this point. But thinking that anti-virus would have a chance in hell again AGI is pretty weak when AV tends to fall apart…

AGI will be a computer program, unless you're imagining some entirely non-computer based form of artificial intelligence. And it will therefore obey the rules of normal computer programs, like vulnerability to SIGKILL.

Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere.

Terrorists aren't programs or even a well defined group, so I don't see the analogy.

Re: Statement on AI Risk

#922
post #776

Earlier quoted context omitted.

> I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI. That's what interesting to me. People fearmongering about bioengineering and GMO's were generally dismissed as being anti-science and holding humankind back (or worse, that there opposition to progress meant they had blood on their hands). Yet many of the people who mocked them proved themsel…

Counterpoint: CRISPR only reignited what was already a real fear of reduced difficulty and costs of engineering deadly pathogens. In fact, what you and GP wrote is baffling to me. The way I see it, biotech is stupidly obviously self-evidently dangerous , because let's look at the facts: - Genetic engineering gets easier and cheaper and more "democratized"; in the last 10 years, the basics were already accessible to m…

Great points. Will just add a point 1.5: There's usually an inverse correlation between ill intent and competence, so the subset of people who both want to cause harm to others on a mass scale and who are also able to pull it off is small

Re: Statement on AI Risk

#923
post #781

Earlier quoted context omitted.

I don't think "generally being right" is the same thing as "literally never getting anything wrong". Not every specific claim in "An Inconvenient Truth" was correct. That doesn't tell us much about whether Al Gore was "generally right" about climate change. His opponents at the time were mostly claiming that it either wasn't happening at all, or wasn't largely the result of human activities. What do you think is the…

I thought someone might say that. This thread is really a perfect demonstration of my point. Our society is so in thrall to self-proclaimed intellectuals that you can literally make a movie presenting falsifiable claims with 100% confidence, people can say at the time "this is absurd and will not happen", you can spend years attacking those critics, it can then not happen and still you will have an army of defenders…

Well, obviously what I say must be wrong if you guessed that someone might say it.

Indeed, someone might say "95%" because they want to make the same sort of impression as if they said "100%" but to be able to hide behind it if they're wrong. Or, y'know, they might say "95%" because they've thought about the strength of the evidence and expect to be right about such things about 95% of the time.

(I'm not sure how relevant any of this is to "An Inconvenient Truth" since you say it makes its claims with 100% confidence. I haven't watched the movie. I don't know exactly what it claims how confidently. It's basically a work of propaganda and I would expect it to overstate its claims whether the underlying claim is largely right or total bullshit or somewhere in between.)

Of course I don't think counting smart people is the only way to find out what's true. It couldn't be; you need some engagement with the actual world somewhere. Fortunately, there are plenty of people engaging with the actual world and reporting on what they find. It turns out that those people almost all seem to agree that climate change is real and a lot of it is caused by human activities.

Of course they could be wrong. And you've looked at the evidence, so no doubt you know better than they do. But ... in that case, this is a field so confusing that most people who dedicate their whole careers to investigating it end up with badly wrong opinions. If so, then why should I trust that your looking at the evidence has led you to the right answer? For that matter, why should you trust that? Shouldn't you consider the possibility that you can go astray just as you reckon those "smart people" did?

If I really wanted to be sure about this, then indeed I wouldn't go counting smart people. I would go into the field myself, study it at length, look at the underlying data for myself, and so forth. But that would mean abandoning the career I already have, and taking at least several years of full-time work before arriving at an opinion. So instead I look to see who seems to be (1) expert and (2) honest, and see what range of opinions those people have.

I find that the great majority of experts think anthropogenic climate change is real and a big deal. They could be wrong or lying or something. Do they look less expert than the people saying the opposite? No, it mostly seems like the people with the best credentials are on the "orthodox" side. Do they look less honest? Hard to tell for sure, but there sure seem to be a lot of people on the "unorthodox" side who just happen to be funded by the fossil fuel industry, and I don't see any strong financial incentive in the opposite direction for the "orthodox" folks.

What if I look at some of the particular claims they make? Some of them are really hard to evaluate without those several years of full-time work. But e.g. 10-15 years ago pretty much everyone on the "unorthodox" side was pushing the idea that warming had stopped, because if you look at the temperature graphs from 1998 onwards there was little or no upward trend. The people on the "orthodox" side replied that when you have signal plus lots of noise you will inevitably get periods that look that way. I did some simpleminded simulations and verified that the "orthodox" folks are right about the statistics. And there's a reason why this argument has disappeared into the memory hole: looking at the graph now no one would suggest that it's flat since 1998.

My impression from the limited amount of "looking at the evidence" I've done myself is that, while the "orthodox" folks haven't been infallible, they've done better than the "unorthodox". For instance, since we're looking at things produced by political figures rather than scientists, here https://web.archive.org/web/20071015042343/http://www.suntim... is an article by James Taylor of the Heartland Institute criticizing "An Inconvenient Truth". Claim 1: Gore says glaciers are shrinking but an article in the Journal of Climate says Himalayan glaciers are growing. Truth: (1) Taylor's alleged quotation isn't from that article but from something else Taylor himself wrote; (2) what the article (https://journals.ametsoc.org/view/journals/clim/19/17/jcli38...) actually says is that in one particular region summer temperatures are falling while winter temperatures rise, and the result is "thickening and expansion of Karakoram glaciers, in contrast to widespread decay and retreat in the eastern Himalayas". So: no, Gore isn't wrong to say glaciers are shrinking, but in one very particular place things work out so that the reverse happens, which is odd enough that someone bothered writing a paper about it. Claim 2: Kilimanjaro. Truth: Yup, Gore got that one wrong. Claim 3: Gore says global warming causes more tornadoes, and the IPCC says there's no reason to think it does. Truth: I dunno, but if the IPCC says that then this is specifically an argument about Al Gore rather than about climate orthodoxy. Claim 4: similar, for hurricanes instead of tornadoes. Truth: again, this seems to be specifically about Al Gore rather than about climate orthodoxy. (Looking at e.g. https://www.gfdl.noaa.gov/global-warming-and-hurricanes/ it seems that the conventional wisdom today is that yes, hurricanes are getting worse and are expected to continue gettings worse, but we don't know with much confidence exactly what the causes are.) Claim 5: Gore says that African deserts are expanding because of global warming, but in 2002 someone found them shrinking. Truth: the Sahel region of the Sahara desert had an extra-severe drought in the 1980s, after which in the short term it improved; this is like the "global warming hiatus" post-1998. The Sahara seems to have increased in size by about 10% over the last century, partly but not wholly because of climate change (https://www.jstor.org/stable/26496100). Claim 6: Gore says Greenland's ice is melting, but actually it's thinning at the edges and growing in the middle and the overall effect is that it's gaining a bit of mass. Truth: take a look at the graph at https://climate.nasa.gov/vital-signs/ice-sheets/; it oscillates within each year but there is an extremely clear downward trend over the entire time NASA's satellites have been measuring. Claim 7: similarly for the Antarctic. Truth: Look at another graph on that same page. There's more random variation here, and circa 2006 you could claim that there isn't a decrease, but the trend is extremely clear.

Gore doesn't come out of this looking anything like infallible, but he's done a lot better than Taylor. And, in general, this is the pattern I see: no one is perfect, especially people who aren't actually scientists, but the claims of the "orthodox" tend to hold up much better over time than those of the "unorthodox".

Re: Statement on AI Risk

#924

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

This is a bad take. The statement is signed by dozens of Academics who don't have much profit motive at all. If they did they wouldn't be academics, they could easily cash in by starting a company or joining one of the big players.

> The statement is signed by dozens of Academics who don't have much profit motive at all.

...right now.

This is the sort of pretexting you do to establish credibility as an "AI safety expert."

Re: Statement on AI Risk

#925

Earlier quoted context omitted.

I appreciate your reasonableness. I follow LW to some degree, but even the best of it (like the post you link) feels very in-group confirmation centric. That post is long and I have not read it all, but it seems to be missing any consideration of AGI upside. It’s like talking about the risk of dying in a car crash with no consideration of the benefits of travel. If I ask you “do you want to get in a metal can that ha…

While I agree that the rhetoric around AI Safety would be better if it tried to address some of the benefits (and not embody the full doomer vibe), I don't think many of the 'core thinkers' are unaware of the benefits in AGI. I don't fully agree with this paper's conclusions, but I think https://nickbostrom.com/astronomical/waste is one piece that embodies this style of thinking well!

Thanks for the link -- that is a good paper (in the sense of making its point, though I also don't entirely agree), and it hurts the AI risk position that that kind of thinking doesn't get airtime. It may be that those 'core thinkers' are aware, but if so it's counter-productive and of questionable integrity to sweep that side of the argument under the rug.

Re: Statement on AI Risk

#926

Earlier quoted context omitted.

Odd that you lump a person's choice of their own pronouns into a legitimate complaint that all she seems to have is prejudice and ad hominems.

I have no issue with her choice of pronouns. I just find it odd that she states them when ~100% of the population would infer them correctly from her name Emily (and picture). My guess is she put them there for ideological signaling.

> My guess is she put them there for ideological signaling.

Yes, and you attacking her for that choice is also ideological signaling. This isn't a case where one choice is totally devoid of signal and the other is pejorative-of-the-day.

Many people believe that using normative pronouns makes those who use different pronouns more comfortable. They may be right, wrong, misguided, whatever. But it's a well-meaning and harmless act, and extremely strange to mention in the same breath as overt racism.

Re: Statement on AI Risk

#927
post #880

Earlier quoted context omitted.

Sure. I recommend reading superintelligence by Nick Bostrom. But I think the key failure here, in your thinking, is that you can't conceive of something , and that therefore something can't happen, and you're doing that in the context of things that are smarter than you . You being able to conceive of it is simply not required for it to happen. Also, Colossus: The Forbin Project was a great movie from 1970 on this su…

Superintelligence and Life 3.0 seem to come up as recurring references in the discussion. I've only read synopses of both but frankly I find the argument that melevolent AI "escape" could occur without being noticed and thwarted a bit far fetched. There's a good counterargument here[1] that seems reasonable: "One of the features of intelligence explosion that most preoccupies Bostrom and Yudkowsky is that it’s not a…

How would you know if it is a good counter argument if you haven’t bothered to read the argument?

Re: Statement on AI Risk

#928

Earlier quoted context omitted.

In order to make your argument clear to people reading: If you are wrong there are no humans left. If I am wrong,inequality,societies will suffer like they have always in the hands of the strong.

Wouldn't you need to illustrate how likely each outcome is? I mean, there are lots of possible ways humans could be eradicated.

First of all, you agree that the probability is non zero right?

I am not a world known expert on xrisk to estimate this. You are not either. We have all these people claiming the probability is high enough. What else is ti be said? HNers shouldn't be reminded "trust in sciense".

Re: Statement on AI Risk

#929

Earlier quoted context omitted.

I’m not sure if extinction is a problem either. No one’s left to care about the issues, then.

How long would the currently 17 people in space survive if every human on planet earth was eradicated?

Dunno. I imagine they’d follow soon, though.

Re: Statement on AI Risk

#930

Earlier quoted context omitted.

It’s not just random text, they are predicting writings and documents produced by humans. They are “language” models. Language is used to say things about the world. This means that predicting language extremely well is best done through acquiring an understanding of the world. Take a rigorous, well-written textbook. Predicting a textbook is like writing a textbook. To write a good textbook, you need to be an expert…

>Predicting a textbook is like writing a textbook HN AGI discourse is full of statements like this (eg. all the stuff about stochastic parrots), but to me this seems massively non-obvious. Mimicking and rephrasing pre-written text is very different from conceiving of and organizing information in new ways. Textbook authors are not simply transcribing their grad school notes down into a book and selling it. They are s…

I took the liberty of asking GPT-4 to take the second statement and your response to it, and turning it into a fable:

"Once upon a time, in a village nestled by a grand mountain, lived a wise Sage. The Sage was known throughout the lands for his vast knowledge and understanding, for he had spent his life studying the texts of old and the secrets of the world.

In the same village, an Artisan, skilled in the craft of making extraordinary mirrors, lived. These were no ordinary mirrors, for they were said to reflect not just physical appearances, but also knowledge and experiences. Intrigued by the wisdom of the Sage, the Artisan decided to make a mirror that would reflect the Sage's knowledge.

After many days and nights of meticulous work, the Artisan finally crafted a mirror so clear and pure, it held a reflection of all the knowledge and wisdom the Sage had gathered throughout his life. The mirror could answer questions about the world, cite ancient texts, and reflect the wisdom it was imbued with.

Word quickly spread about the Sage's Mirror, and villagers began to claim, "This mirror is as wise as the Sage himself! It knows and understands as much as he does!"

However, a wise Old Woman of the village, known for her insightful observations, gently corrected them, "This mirror, as remarkable as it is, contains a reflection of the Sage's knowledge. It can share what the Sage knows but doesn't truly understand the way the Sage does."

The Old Woman continued, "The Sage has spent years learning, pondering, and experiencing life, which the mirror cannot replicate. The Sage's understanding implies the ability to think, reason, and learn in ways that the mirror, no matter how complete its reflection, simply cannot. The mirror's reflection is static, a snapshot of a moment in time, while the Sage's wisdom continues to grow and adapt."

The villagers learned a valuable lesson that day. They realized the mirror was an extraordinary tool that held vast knowledge, but it was not a substitute for the genuine understanding and wisdom of the Sage."

- Not too bad for a mirror.

I'd be interested to hear what you think is so special about human understanding? We also just absorb a lot of data and make connections and inferences from it, and spit it out when prompted, or spontaneously due to some kind of cognitive loop. Most of it happens subconsciously, and if you stop to observe it, you may notice that you have no control of what your next conscious thought will be. We do have a FEELING that we associate with the cognitive event of understanding something though, and I think many of us are prone to read a lot more into that feeling than is warranted

Post reply on HN