171–180 of 183 posts
Earlier quoted context omitted.
The exact details on how to make a Thermonuclear weapon are classified, do you think that is done so improperly?
Are you comparing building a nuclear bomb to running a piece of code?
Earlier quoted context omitted.
Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.
First you said it's clear, now you're saying it's highly speculative. Choose one.
Earlier quoted context omitted.
Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.
I don't think you should assume your position is the only valid one. I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.
So you are claiming that by making this statement you are not assuming that it is impossible to control an AGI?
Earlier quoted context omitted.
I don't think you should assume your position is the only valid one. I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.
Seems very unclear if you could "instruct" true AGI to do something. So you are claiming that by making this statement you are not assuming that it is impossible to control an AGI?
This is similar to having a very advanced bioengineering technology, where you could instantly change anything about your body. Would you make yourself smarter? Sure. Would you change who you are (e.g. turned off your instincts/desires)? Not so sure.
Earlier quoted context omitted.
To fully answer your last point. Sorry- this has grown extremelly long. I'm actually doing work right now, waiting for a long experiment to run so I got some time to burn :) So, my pet formula for deciding whether taking a risk is worth it is to multiply the probability of some adverse event occurring, let's call this event X, by the cost associated with that event, let's call it Y; so R = p(X) * Y, where R the risk…
Do you sincerely think that we are totally unable to predict when--or if ever--we might create general intelligence, something which stupid evolution has done through trial and error, but humans can attempt with striking brilliance and worldwide motivated research efforts? You can't make any rough prediction on whether we will be able to reach that point, or when it will happen?
Earlier quoted context omitted.
To fully answer your last point. Sorry- this has grown extremelly long. I'm actually doing work right now, waiting for a long experiment to run so I got some time to burn :) So, my pet formula for deciding whether taking a risk is worth it is to multiply the probability of some adverse event occurring, let's call this event X, by the cost associated with that event, let's call it Y; so R = p(X) * Y, where R the risk…
Do you sincerely think that we are totally unable to predict when--or if ever--we might create general intelligence, something which stupid evolution has done through trial and error, but humans can attempt with striking brilliance and worldwide motivated research efforts? You can't make any rough prediction on whether we will be able to reach that point, or when it will happen?
Yes.
>> You can't make any rough prediction on whether we will be able to reach that point, or when it will happen?
No.
Earlier quoted context omitted.
point is: AI is different than your usual game, in that the winner might appear randomly, and destroy the world if she makes a mistake. so i believe open ais points are warrented
> and destroy the world if she makes a mistake How would the world be destroyed? Does an example work without handwaving about recursive self-improvement and an imperative to optimize extremely literally? Can you give me a play by play of how a newly developed strong AI eradicates the human species quickly and thoroughly without us having any time to react? EDIT: In summation, there have been several downvotes, but t…
So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…
>> What is the responsible disclosure process for ideas and research around AI? Basically, we're so far away from AGI that there's no need to worry about disclosig anything. The recent advances in machine vision and speech processing are impressive, but only in the context of the last 50 years or so. A trully intelligent agent will need much more than this and there doesn't seem to be anyone alive today who knows how…
Interested in an off-ycombinator discussion?
Earlier quoted context omitted.
> Give me a credible scenario that brings us from strong AI to the annihilation of our species without handwaving about recursive self-improvement and selective idiocy and I'll reconsider my position. What kind of credible scenario are you looking for? You're obviously knowledgeable about the subject, I'm sure you've read various stories, but it's quite easy to dismiss them all as "not credible" or "just so stories".…
A credible scenario is one in which none of the parts of the story have a giant leap between them. For example, with your scenario: AGI gets built by a farming company Okay, sounds fine, keep going. gets programmed to farm more corn Yep, this makes sense to me. but there is no stop condition programmed in Definitely sounds like something that could happen, with you so far. so converts ever available land into corn fi…
Even if the smartest hypothetical AI can perfectly extrapolate the mental states of every human who ever lived, we still die in a cloud of nanobots if it isn't programmed ever-so-carefully to care about what we want.
Earlier quoted context omitted.
> Give me a credible scenario that brings us from strong AI to the annihilation of our species without handwaving about recursive self-improvement and selective idiocy and I'll reconsider my position. What kind of credible scenario are you looking for? You're obviously knowledgeable about the subject, I'm sure you've read various stories, but it's quite easy to dismiss them all as "not credible" or "just so stories".…
A credible scenario is one in which none of the parts of the story have a giant leap between them. For example, with your scenario: AGI gets built by a farming company Okay, sounds fine, keep going. gets programmed to farm more corn Yep, this makes sense to me. but there is no stop condition programmed in Definitely sounds like something that could happen, with you so far. so converts ever available land into corn fi…
Your argument, as I understand it, is that something intelligent and capable will only interpret its "instructions" absolutely literally. That's not the way I look at it.
It's not that another intelligence won't be able to understand what our "real goals" are. It's that it won't care - whatever it is programmed to do is, quite literally, its goal/value system.
I mean, we don't have to get exotic here - look at humans. We very clearly evolved to find sex pleasureable in order to spread our genes; just as clearly, many humans, while completely understanding the purpose of sex being pleasurable, continue to have sex without any attempt to spread their genes by using birth control.
And just as equally, while most other humans have more-or-less the same value system as I do, I think you'd be hard pressed to convince me that there have never been humans who have tried to do things I disagree with and wouldn't want to happen. And that's just humans. It's pretty clear that if certain people were more capable, a lot of bad things would've happened (at least from my point of view).
Basically, the idea that intelligence infers a value system is something I've been long convinced is untrue, and I'm not sure why you think otherwise.
> meanwhile 3) the collective capability of the human species is apparently insufficient to stop this from happening.
Well, I'm hoping we aren't. But in order to stop it, we need to do something, and clearly most people aren't even convinced there's a real threat. Hopefully, we can stop an unsafe AGI before it gets built, and also hopefully, we can stop it after it exists. But if you're just assuming that we'll be able to stop it after it has already started to do things against our interest, I think you're being optimistic - even humanity has already created weapons that, had we decided to really use them, could've destroyed all other humans before they had a chance to respond.
> But I take it about as seriously as alien life trying to invade us, a random meteorite killing us all, a solar flare destroying the Earth or malicious time travelers.
Some of those things are more fanciful. But, some of those are real threats, e.g. random meteorite. It has wiped out entire species before. I agree we shouldn't be very scared of a low probability event that we can't control, but that's the difference - we can control AGI, before it gets built, and it doesn't appear to be low probability. Not to mention, most people agree that we should get off this planet to protect against meteorites, too!
As for aliens - if you were truly convinced that aliens would visit the Earth in 200 years - are you really telling me it wouldn't change anything about what you think humanity should be doing? I for one think we would definitely be spending time in preparing for that. And that's more-or-less how I look at AGI - we're creating an alien that will visit us between 50-1000 years from now, and we're not really sure when exactly. What should we do to prepare?