Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's No Fire Alarm for Artificial General Intelligence

intelligence.org

Post reply on HN
Print view View original

191–200 of 209 posts

Re: There's No Fire Alarm for Artificial General Intelligence

#191

Post by sullyj3 »

fourfaces wrote: post #185

We don't need a fire alarm for AGI. The problem is not AGI. Machines will be motivated to do exactly what we tell them to do. It's called classical and operant conditioning. The problem is not AGI for the same reason that the problem is not knives, nuclear power, dynamite or gunpowder. The problem is us. The problem has always been us. Those who are running around screaming about the danger of AGI and why it should b…

Yes, an AI will do exactly what we tell it to do. The the incredible difficulty programmers have with writing bug-free code demonstrates that doing exactly what it's told isn't sufficient to guarantee it'll do what we want.

Classical and operant conditioning are psychological concepts that aren't applicable to non-humans.

Re: There's No Fire Alarm for Artificial General Intelligence

#192

Post by sullyj3 »

nurettin wrote: post #4

Why AGI and not "Artificial Consciousness" ? Is it because people think that consciousness is a by-product emerging from many kinds of pattern detection algorithms that suit all cases? (if so, what is the evidence for that?)

Because when I'm worried about a machine recycling the atoms that make up my body, I'm not particularly interested in whether it has conscious experience. There's no particular reason to suppose that something needs to be conscious in order to be very effective at achieving arbitrary goals.

Re: There's No Fire Alarm for Artificial General Intelligence

#193

Post by johansch »

gooseus wrote: post #163
johansch wrote: post #145

Earlier quoted context omitted.

That doesn't really make much sense to me, I'm afraid. Please bear with me, but my first guess is that you have invented a coping mechanism that allows you to deal with conflicting information in your mind. Are you by any chance religious? (I figure I should be allowed to ask such a normally speaking quite loaded question because of my previous statements, up the thread.)

No, I am not, by any chance, religious. I don't rule out the possibility of a God or God-like intelligence existing in the Universe, but that's simply as a result of the "absence of evidence is not evidence of absence" principle I hold to. I don't really see how religion factors into this though... I feel like I'm talking about a simple concept too. If I show you an encrypted message and show you that other people ca…

Well, my apoligies for being presumptuous then.

Re: There's No Fire Alarm for Artificial General Intelligence

#194

Post by akvadrako »

Smaug123 wrote: post #85
akvadrako wrote: post #39

Earlier quoted context omitted.

Intelligence is one of the most poorly defined words there is; I can't even begin to imagine what you have in mind. Consciousness is usually only one of two things; either "being aware" or "the experience of having an inner voice".

Neither of those is a concrete definition. Defining "consciousness" as "being aware" is just punting the question on to "how do you define the word AWARE?". And some people don't have an inner monologue. It's not specifically mentioned in http://slatestarcodex.com/2014/03/17/what-universal-human-ex... but it's of that same class of thing; there are certainly hits on Google for people claiming not to have an inner voi…

"Being aware" is a common phrase understood by most native english speakers as a synonym of "conscious". If you are asleep, you are not aware of much. If you get hit by a car as you cross the road it's probably because you weren't aware of it. You can do some actions, like driving, either consciously or unconsciously.

I didn't say that all "people" have inner voices (by which I meant inner life). But by that definition, it's a requirement to be conscious.

Re: There's No Fire Alarm for Artificial General Intelligence

#195

Post by rspeer »

blennon wrote: post #141
randomsearch wrote: post #132

Can someone please explain what has happened in ML or AI that makes AGI closer? Whilst some practical results (image processing) have been impressive, the underlying conceptual frameworks have not really changed for 20 or 30 years. We're mostly seeing quantitive improvements (size of data, GPGPU), not qualitative insights. ML in general is just applied statistics. That's not going to get you to AGI. Deep Learning is…

The biggest advance that I've seen towards AGI is the work using reinforcement learning, e.g. neural nets that learn to play video games through trial and error. There is an impressive repertoire of _behavior_ that emerges from these systems. This, in my opinion, has the greatest potential to take us another big step towards -- but not necessarily to -- AGI.

That advance happened by 1992, with TD-Gammon [1]. Our hardware and software are clearly much better now, but this seems like a solid example of what the GP said: the conceptual framework has stayed the same for 25 years.

[1] https://en.wikipedia.org/wiki/TD-Gammon

Re: There's No Fire Alarm for Artificial General Intelligence

#196

Post by rspeer »

Tossrock wrote: post #77

For me, the "smoke under the door" moment was Karpathy and Li's Deep Visual-Semantic Alignments for Generating Image Descriptions [1]. The almost perfectly grammatical machine-generated captions of photos were unnerving to me in a way that simple categorization was not. It somehow called to mind the image of a blank-eyed person speaking in a monotone while images flashed in front of them. What if they wake up? [1]: h…

These systems are lining up common phrases that appear in corpora of image captions, with visual patterns that they can be trained to recognize such as color, textures, and shapes. Nothing is "waking up".

Your astonishment at what these systems can do tells me that you may have looked at cherry-picked positive results. So here's an article I found that cherry-picks negative results instead: [1]

[1] https://gizmodo.com/this-neural-networks-hilariously-bad-ima...

Now of course this article is exaggerated too. Ideally, if a system is 95% accurate, you'd be looking at representative output from the system, with 95% good results and 5% bad ones, perhaps by running such a system yourself on a different set of images.

Re: There's No Fire Alarm for Artificial General Intelligence

#197

Post by Eliezer »

argonaut wrote: post #173
edanm wrote: post #142

Earlier quoted context omitted.

I think the most accurate answer is that we just don't know. Since we really don't know how an AGI could work, we have no idea which of the advances we've made are getting us closer, if at all. Is it just an issue of faster GPUs? Is the work done on deep learning advancing us? I don't think we'll know until we actually reach AGI, and can see in hindsight what was important, and what was a dead end. I do take exceptio…

It's funny that you mention Relu. People have recently trained Imagenet networks using sigmoid/tanh (e.g. the activations that were used decades ago) on GPUs and they train just fine. They train a bit slower is all. Not the breakthrough you're making it out to be. Relus were a very useful stop-gap in 2012 when GPUs weren't as fast.

Now that we know how to initialize the weights so as to have the layer activations be something like sane, yes, we can use sigmoid/tanh. If you don't know modern clever ways of initializing weights then multi-layered sigmoid/tanh causes your activations and gradients to die out fast in deep networks, and ReLU is a godsend.

Re: There's No Fire Alarm for Artificial General Intelligence

#198

Post by Veedrac »

robbensinger wrote: post #179
Veedrac wrote: post #176

Earlier quoted context omitted.

Those aren't exactly the questions I'm raising; I have no doubt that there exists some way to produce AGI. My concern is that it doesn't seem like the right question to ask, since history suggests that humans are much better at first building specialized devices, and when it comes to AI risk the only one that really matters is the first one built. I might have misunderstood your post, though.

The thing I'm pointing to is that there are certain (relatively) specialized tasks like 'par-human biotech innovation' that require more or less the same kind of thinking that you'd need for arbitrary tasks in the physical world. You may need exposure to different training data in order to go from mastering chemistry to mastering physics, but you don't need a fundamentally different brain design or approach to reason…

> The thing I'm pointing to is that there are certain (relatively) specialized tasks like 'par-human biotech innovation' that require more or less the same kind of thinking that you'd need for arbitrary tasks in the physical world.

The same way Go requires AGI, and giving semantic descriptions of photos requires AGI, and producing accurate translations requires AGI?

Be extremely cautious when you make claims like these. There are certainly tasks that seem to require being humanly smart in humanly ways, but the only things I feel I could convincingly argue being in that category involve modelling humans and having human judges. Biotech is a particularly strong counterexample, because not only is there no reason to believe our brand of socialized intelligence is particularly effective at it, but the only other thing that seems to have tried seems to have a much weaker claim at to intelligence yet far outperforms us: natural selection.

It's easy to look at our lineage, from ape-like creatures to early humans to modern civilization, and draw a curve on which you can place intelligence, and then call this "general" and the semi-intelligent tools we've made so far "specialized", but in many ways this is just an illusion. It's easier to see this if you ignore humans, and compare today's best AI against, say, chimps. In some regards a chimp seems like a general intelligence, albeit a weak one. It has high and low cognition, it has memory, it is goal-directed but flexible. Our AIs don't come close. But a chimp can't translate text or play Go. It can't write code, however narrow a domain. Our AIs can.

When I say I expect the first genuinely dangerous AI to be specialized, I don't mean that it will be specific to one task; even neural networks seem to generalize surprisingly well in that way. I mean it won't have the assortment of abilities that we consider fundamental to what we think of as intelligence. It might have no real overarching structure that allows it to plan or learn. It might have no metacognition, and I'd bet against it having the ability to convincingly model people. But maybe if you point it at a network and tell it to break things before heading to bed, you'd wake up to a world on fire.

Re: There's No Fire Alarm for Artificial General Intelligence

#199

Post by Balgair »

ciphergoth wrote: post #169
Balgair wrote: post #131

Earlier quoted context omitted.

> The superhuman fallacy really is the bane of AI Great point, I'll be stealing that one ;) One annoyance I have heard about SV is that all the companies are just trying to replace your Jewish Mom: Uber/Lyft is Mom's minivan, GrubHub/DoorDash/BlueKitchen is Mom's cooking, Google is Mom's encylopedia, Yelp is the synagogue's meeting hallway, Tinder is your Mom's yenta, etc. The examples abound in a non-B2B space. In t…

Bulverism is a poor substitute for actually thinking about the future. See Scott Alexander's excellent recent essay: http://slatestarcodex.com/2017/10/09/in-favor-of-futurism-be...

TLDR?

Re: There's No Fire Alarm for Artificial General Intelligence

#200

Post by YeGoblynQueenne »

sullyj3 wrote: post #190
YeGoblynQueenne wrote: post #181

I don't get this article. It keeps making the point that it's very hard to predict the future, even for specialists, then it uses this to argue that we should be preparing for AGI right now, precisely because we don't know if and when it will happen. Well, if you have no way to tell whether something is going to happen, or not, you don't prepare for it- because you can't justify spending the resources to prepare. Or…

I don't think the point of the article is to make a complete argument for acting now, but rather to counter a particular objection to acting now. MIRI has a fair amount of writing, which I would say in totality constitutes an argument for beginning work on AI safety ASAP. I don't think it's fair to expect them to reiterate the entire thing in every article they put out. That's what their FAQ is for.

Equally, I don't think it's fair that I should be expected to read "a fair amount of writing" before I can read this article which, itself, is a fair amount of writing. I can see that the author has some prior history with specific counters to his line of reasonsing, but i don't think I should need to go over the entire history before I can understand what he's talking about (and I do need to do that, then that's a problem of his text, not my reading; I'm reading an article on the web, after all, not a scientific publicaiton with references).

And I'm pretty sure I do understand what he's talking about. He's saying that the coming of AGI is entirely uncertain so we should act now. That's unreasonable in and of itself, and I don't need to go back and read all the history he's got with others to be able to tell that.

Opinion, anyway.

Edit: To reiterate why it's unreasonable- if we won't see AGI until it's already here, what, exactly, are we supposed to be preparing against? We won't know it even when we see it- so how will we know what to protect ourselves from?

Reasoning with uncertainty requires at least some amount of knowledge and the article goes to great lengths to point out there isn't any, in the case of AGI.

So there's no reasoning to be done, either. In that case, what are we talking about? "Beware of the unknown"? Well, OK. I don't know if the sky will fall on my head tomorrow so maybe I should stay home, just in case?

Post reply on HN