Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There’s No Fire Alarm for Artificial General Intelligence (2017)

intelligence.org

Post reply on HN
Print view View original

81–90 of 130 posts

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#81

Post by square_usual »

pavlov wrote: post #19
Eliezer wrote: post #12

Earlier quoted context omitted.

Here's an extensive reply to Chollet's essay, also by the author of "There's No Fire Alarm": https://intelligence.org/2017/12/06/chollet/ > ...some systems function very well in a broad variety of structured low-entropy environments. E.g. the human brain functions much better than other primate brains in an extremely broad set of environments, including many that natural selection did not explicitly optimize for. We…

Any graph of world GDP is meaninglessly short on the timescale of the Earth’s ecosystem. What looks like unstoppable success to us could be a blip in the broader context.

That is not a "practical" time scale though. Sure, if we wait long enough, GDP might flatten out, but on human time scales so far it has grown exponentially. A similar analogy can be made for AGI.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#82

Post by someguyorother »

nopinsight wrote: post #68

Our civilization and ability to dominate over other animals largely exist because of human intelligence. An AGI could be a great partner toward unprecedented prosperity, but an AGI not fully aligned with human values may also spur a major catastrophe. Two promising approaches toward AI Safety I have encountered: * Make sure the AI is not too certain of itself and will always defer to humans as the final judge. https:…

You're going to have to implement a value system into the AGI if it is a fully general AGI. Otherwise, it will just use its exceptional intelligence to defeat its own sensors, take control of its objective function, and essentially wirehead itself.

So an AGI without a value system is (mostly) harmless. If the AGI has a defined value system, it's a lot clearer what that value system is. No "whoops gonna destroy the world to make paperclips" will happen.

(A not-fully general AGI might not be smart enough to defeat its own sensors, yet still be smart enough to do lots of harm. The typical example is grey goo, which isn't smart at all, yet plenty harmful.)

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#83

Post by mehrdadn »

ajuc wrote: post #77
mehrdadn wrote: post #63

Earlier quoted context omitted.

Quite an interesting point I hadn't considered at all. On the one hand I'm wondering: what's your suggestion on how to address this with minimal changes to my criteria? On the other hand I'm wondering: well, if the analog of this is that AGI might get solved with more AGI, then that's only going to make me less likely to be worried in the first place!

To make it fair you can archive current spam data and see if future spam classifiers (not trained on that particular data) recognizes it correctly. > if the analog of this is that AGI might get solved with more AGI, then that's only going to make me less likely to be worried in the first place! that's going to end in arms race that leaves people unaware of what's even happening (aka singularity)

I had issues with just taking "current spam data, not trained on that particular data":

- It allows trainers to hard-code future rules based on their experience of what has passed through past filters, even if their model isn't technically trained on this dataset

- You might get similar emails sent to different mailboxes and the instances not included would still be allowed (and I don't really want to go down the rabbithole of defining a similarity metric between emails)

- I think I want to allow spammers to evolve their capabilities at least using current techniques, which we all presumably agree is "less than AGI". After all, intelligence implies adapting to a dynamic environment. It's not really going to feel like AGI (and certainly not going to make me worry) if it looks like AGI is trivial to outsmart by humans or less-than-AGI techniques.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#84

Post by throwaway2048 »

narag wrote: post #61
cwyers wrote: post #35

Earlier quoted context omitted.

> The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." As soon as an AI CEO starts trying to maximize a sin…

How about profit as a well-defined metric?

Plenty of examples of corporations run into the ground in the name of profit

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#85

Post by jonnypotty »

Artificial General Intelligence isnt a goal its a fear. We reduce everything that we try and program into computers into simple rules because complexity is too hard for us to think about. ML has allowed us to move up a level of complexity but we've had to give up understanding the mechanics of the machine as a consequence. AGI using machine learning techniques needs a function to optimise for. If someone can explain to me how you'd even, in theory, go about teaching an AI about EVERYTHING rather than one specific thing I'll take it seriously. We think the game of Go, because it displays chaotic and complex patterns and is difficult to predict, somehow this 'models' the real world. What it actually shows you is the bewildering complexity the universe is capable of given only very simple rules. The actual complexity of the world is on a different scale. Go has an aim, you can measure if you're getting better at it. General intelligence does not have a defined goal, understand everything perhaps? Even in theory how do you tell if you are closer to this goal or not. In a game of Go you 'nearly' won or lost. How do you get close to AGI and know that you were closer than the last time you tried? Modeling the state of a game of Go in a computer is trivial. What is the model you use to teach a computer about the world or the universe?

Imo not only is our software nowhere close to AGI, neither is our hardware or our ideas.

That said having a fire alarm to tell us the terminator is coming seems like a great idea in theory. As long as the sprinklers spray something that melts metal

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#86

Post by [deleted] »

ajuc wrote: post #57

If you told people in 90s that at one point in future AIs will be able to: - drive a car in traffic in almost all circumstances safer than majority of human drivers basing only on video input - take a video and replace faces from another video well enough that more than 90% of people are fooled - have 95%+ accurate OCR and speech recognition - predict people's preferences regarding music/movies/books better than any…

[deleted]

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#87

Post by ajuc »

jotm wrote: post #18
Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

I've been saying AI won't be Skynet launching the nukes, it will be a corporation maximizing productivity with complete disregard for human needs heh

This already happened in 19th century. That's why we need the state (which is another AI when you think about it, with some hardcoded rules and periodical review (elections)).

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#88

Post by ajuc »

WalterBright wrote: post #22
jotm wrote: post #18

Earlier quoted context omitted.

I've been saying AI won't be Skynet launching the nukes, it will be a corporation maximizing productivity with complete disregard for human needs heh

In a free market, a corporation has to please its customers in order to maximize profit. For example, Apple.

In doing so it often does things that customers wouldn't like if they knew about them.

For example child labour, slave labour, destroying environment, causing cancer, etc.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#89

Post by SCdF »

leereeves wrote: post #80
SCdF wrote: post #70

Earlier quoted context omitted.

> drive a car in traffic in almost all circumstances safer than majority of human drivers basing only on video input Has that happened? Don't self driving cars use a huge range of sensors, and are they actually legal / widely deployed anywhere at all? Are any for sale? I don't mean improved cruise control, I mean "OK computer, take me to work". > take a video and replace faces from another video well enough that more…

> haven't seen an example that was anything other than creepy, and really really obvious. Do you think I'm just really sensitive to this kind of thing, or have I not seen the really good examples? Here's a really good example (the video, not the audio, which the creators say was intentionally degraded): https://www.youtube.com/watch?v=l82PxsKHxYc

So my positive take is that this is the best (or worst depending on your perspective) I've seen for sure.

The negative take is that this still looks really fake to me. His eyes are dead, his head whips around weirdly, his neck undulates and flickers, his face doesn't appear attached to his head, etc. Also, due to how Obama looks he may be easier than other people (ie no hair).

Also, this isn't exactly what OP was talking about, though it's similar. They were talking about replacing faces, whereas this is (I presume) actually 100% Obama video from his various speeches reconstituted. So tbc my reaction was to the various videos I've seen of _that_, none of which have been remotely convincing.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#90

Post by shahbaby »

catalogia wrote: post #75
jinpan wrote: post #43

Earlier quoted context omitted.

Have you heard of this thing called quantum computing?

Are quantum computers even relevant to AGI? I'm under the impression that the belief that human brains are quantum computers is considered fringe in both computer science and particularly neuroscience, despite having a handful of high profile proponents (notably Penrose.)

People have a common tendency to connect one thing they don't understand with another they also don't understand.
Post reply on HN