Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thousands of AI Authors on the Future of AI

arxiv.org

Post reply on HN
Print view View original

81–90 of 115 posts

Re: Thousands of AI Authors on the Future of AI

#81

Post by Vecr »

notahacker wrote: post #72
Vecr wrote: post #38

Earlier quoted context omitted.

I'm going to take that to mean "P(every last human dead) > 0.5" because I can't model situations like that very well, but if for some reason (see Thucydides Trap for one theory, instrumental convergence for another) the AI system thinks the existence of humans is a problem for its risk management, it would probably want to kill them. "All processes that are stable we shall predict. All processes that are unstable we…

If the AGI - which is for some reason always imagined as a singular entity - thinks humans are unpredictable and risky now, just imagine the unpredictability and risk involved in trying to kill all seven billion of us whilst keeping the electricity supply on...

It would have to prepare to survive a very large number of contingencies (preferably in secret) and then execute fait accompli with high tolerance to world model perturbations. It might find some other way to become independent from humans (I'm not a giga-doomer like Big Yud, ~13% instead of >99%, though I think he overstates it for (human) risk management reasons), but the probability is way too high to risk it. If a 1% chance of an asteroid (or more likely a comet, coming in from "behind" the sun) killing everyone is not worth it, neither is that same percentage for an AGI/ASI. I don't see the claimed upside unlike a lot of people, so it's just not worth it on cost/benefit. Edit: it's usually described as a single entity, because barring really out-there decision theory ideas, they're more of a risk to each other than humans are to them. It's not "well, if instrumental convergence is right, and they can't figure out morality (i.e. orthogonality thesis)", it's "almost certain conflict predicted".

Re: Thousands of AI Authors on the Future of AI

#82

Post by endisneigh »

golol wrote: post #77
endisneigh wrote: post #70

Earlier quoted context omitted.

I'm curious to hear your definition of AGI that hasn't already been met, given computers have been superior to humans at a large variety of tasks since the 90s.

- passing a hard Turing test, adversarial and with a duration of a few weeks and comparing with 10th percentile humans. - being a roughly human equivalent remote worker. - having robust common sense on language tasks - having robust common sense on video, audio and robotics tasks, basically housework androids (robotics is not the difficulty anymore). Just to name a few. There is a huge gap between what LLMs van do an…

I'm not sure these are good examples. What are the actual tasks involved? These are just as nebulous as "AGI".

I assure you computers already are superior to a human remote worker whose job it is to reliably categorize items or to add numbers. Look no further than the duolingo post that's ironically on the front page at the time of this writing with this very post.

computers have been on par with human translators at some languages since the 2010s. an hypothetical AGI is not god, it still would need exposure, similar to training with LLMs. We're already near the peak with respect to that problem.

I'm not familiar with a "hard turing test." What is that?

Re: Thousands of AI Authors on the Future of AI

#83

Post by staunton »

lainga wrote: post #4
bbor wrote: post #2

Very interesting, especially the huge jump forward in the first figure and a possible majority of AI researchers giving >10% to the Human Extinction outcome. To AI skeptics bristling at these numbers, I’ve got a potentially controversial question: what’s the difference between this and the scientific consensus on Climate Change? Why heed the latter and not the former?

A climate forcing has a physical effect on the Earth system that you can model with primitive equations. It is not a social or economic problem (although removing the forcing is). You might as well roll a ball down an incline and then ask me whether Keynes was right.

I would think any scenario where humans actually go extinct (as opposed to just civilization collapsing and the population plummeting, which would be terrible enough) has to involve a lot of social and economic modeling...

Re: Thousands of AI Authors on the Future of AI

#84

Post by endisneigh »

paxys wrote: post #78
endisneigh wrote: post #69

Earlier quoted context omitted.

What is your definition for AGI that isn't already met?

Intelligence involves self-learning and self-correction. AIs today are trained for specific tasks on specific data sets and cannot expand beyond that. If you give an LLM a question it cannot answer, and it goes and figures out how to answer it without additional help, that will be behavior that qualifies it as AGI.

by that definition what you realize is that it's the same as what I said since it can easily be reduced down to any thing any human can do, and your definition says AGI can go figure out how to do it. you extrapolate this onto future tasks and viola.

as I mention in another post, this is why I do not make any distinction between AGI and superintelligence. I believe they are the same thing. a thought experiment - what would it mean for a human to be superintelligent? presumably it would mean learning things with the least possible amount of exposure (not omniscience, necessarily).

Re: Thousands of AI Authors on the Future of AI

#85

Post by abeppu »

It seems wild to me that the 50% prediction for "unaided machines outperforming humans in every possible task" is 2047, but for "all human occupations becoming fully automatable" is 2116. That's multiple generations of people going to work (or _training_ to go to work) knowing that a machine could do it better. Someone born after machines can outperform humans at _everything_ could go to school, work a series of jobs, and reach retirement age, if the median predictions are right. What would have to be true of human institutions or the economy for that to be true?

Re: Thousands of AI Authors on the Future of AI

#86

Post by abeppu »

endisneigh wrote: post #12

Maybe I'm too pessimistic, but I doubt we will have AGI by even 2100. I define AGI as the ability for an intelligence that is not human to do anything any human has ever done or will do with technology that does not include itself* (AGI). * It also goes without saying that by this definition I mean to say that humanity will no longer be able to meaningfully help in any qualitative way with respect to intellectual tas…

There's a lot of cool work being done on embodied intelligence -- what makes you think that 76 years wouldn't be enough to create an embodied agent with relevant in-built constraints?

Re: Thousands of AI Authors on the Future of AI

#87

Post by glial »

throwup238 wrote: post #62
glial wrote: post #50

Earlier quoted context omitted.

> We don't need to simulate them. You might be right, but this is the kind of hubris that is often embarrassing in hindsight. Like when Aristotle thought the brain was a radiator.

If you have any evidence to the contrary, I would love to hear it because it would upend biology and modern medicine as we know it and we'd both win a Nobel prize. As long as it's modern scientific evidence and not a 2,300 year old anecdote, of course.

The role of astrocytes in neural computation is an example. For a long time, the assumption was that astrocytes were just "maintenance" or structural cells (the name "glia" comes from "glue"). Thus, they were not included in computational models. More recently, there is growing recognition that they play an important role in neural computation, e.g. https://picower.mit.edu/discoveries/key-roles-astrocytes

Re: Thousands of AI Authors on the Future of AI

#88

Post by staunton »

bbor wrote: post #2

Very interesting, especially the huge jump forward in the first figure and a possible majority of AI researchers giving >10% to the Human Extinction outcome. To AI skeptics bristling at these numbers, I’ve got a potentially controversial question: what’s the difference between this and the scientific consensus on Climate Change? Why heed the latter and not the former?

I think human extinction due to climate change is extremely unlikely. However, civilization collapsing is bad enough. We can't be certain whether or not that will actually happen but we do know it will if we do nothing. We even have a pretty good idea when, that's not too late yet, and we have an actionable scientific consensus about what to do about it.

In many ways AI risk looks like the opposite. It might actually cause extinction but we have no idea how likely that is and neither do we have any idea how likely any bad not-quite-extinction outcome is. The outcome might even be very positive. We have no idea when anything will happen and the only realistic plan that's sure to avoid the bad outcome is to stop building AI, which also means we don't get the potential good outcome, and there's no scientific consensus about that (or anything else) being a good plan because it's almost impossible to gather concrete empirical evidence about the risk. By the time such evidence is available, it might be too late (this could also have happened with climate change, we got lucky there...)

Re: Thousands of AI Authors on the Future of AI

#89

Post by idopmstuff »

bcrosby95 wrote: post #6
idopmstuff wrote: post #3

> If science continues undisrupted, the chance of unaided machines outperforming humans in every possible task was estimated at 10% by 2027, and 50% by 2047. Maybe I'm overly optimistic (or pessimistic depending on your point of view, I suppose), but 50% by 2047 seems low to me. That just feels like an eternity of development, and even if we maintain the current pace (let alone see it accelerate as AI contributes mor…

What is the "current pace". Last year? Last 5 years? Last 20 years? If you mean the last year, is that pace maintainable?

> If you mean the last year, is that pace maintainable?

That is the question, though I'd turn it around on you - over the course of human history, the speed of progress has been ever-increasing. As AI develops, it is itself a new tool that should increase the speed of progress. Shouldn't our base case be the assumption that progress will speed up, rather than to question whether it's maintainable?

Re: Thousands of AI Authors on the Future of AI

#90

Post by daxfohl »

jetrink wrote: post #45
sveme wrote: post #16

Does anyone know potential causal chains that bring about the extinction of mankind through AI? Obviously aware of terminator, but what other chains would be possible?

To borrow a phrase from Microsoft's history, "Embrace, Extend, Extinguish." AI proves to be incredibly useful and we welcome it like we welcomed the internet. It becomes deeply embedded in our lives and eventually in our bodies. One day, a generation is born that never experiences a thought that is not augmented by AI. Sometime later a generation is born that is more AI than human. Sometime later, there are no humans…

People wouldn't even get a vaccine injection because there were supposedly Bill Gates's microchips in them. Now you expect they'll be flocking to get these because they have Bill Gates's microchips in them?

Actually, I could see that happening.

Maybe it's time to give AGI a chance to run things anyway and see if it can do any better. Certainly it isn't a very high bar.

Post reply on HN