Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There’s No Fire Alarm for Artificial General Intelligence (2017)

intelligence.org

Post reply on HN
Print view View original

21–30 of 130 posts

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#21

Post by andrewflnr »

Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

> A corporation is a system for maximizing a well-defined metric.

Interesting theory. My perspective is that once a corporation thinks of itself that way, it's already going downhill. Once a measure becomes a target, it ceases to be a good measure.

Now I assume you're imagining the well-defined metric to be profit, in attempt to dodge the aforementioned problem. However, even leaving aside the problem of short- vs long-term profit, I don't think that metric has a sufficiently well-defined link with actions to be optimized with anything we can build in the near term. There's too much "common sense" involved that, AFAICT, we still don't have any idea how to handle.

Ed: I guess I sort of did the thing the OP told me not to. But I still don't think the ability to phrase the problem of running a corporation as "optimize profit" makes it likely to be accomplished sooner than any other AGI application.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#22

Post by WalterBright »

jotm wrote: post #18
Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

I've been saying AI won't be Skynet launching the nukes, it will be a corporation maximizing productivity with complete disregard for human needs heh

In a free market, a corporation has to please its customers in order to maximize profit.

For example, Apple.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#23

Post by XorNot »

andrewflnr wrote: post #21
Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

> A corporation is a system for maximizing a well-defined metric. Interesting theory. My perspective is that once a corporation thinks of itself that way, it's already going downhill. Once a measure becomes a target, it ceases to be a good measure. Now I assume you're imagining the well-defined metric to be profit, in attempt to dodge the aforementioned problem. However, even leaving aside the problem of short- vs lo…

Investors would go nuts betting on AI corporations which burned short and bright with huge returns in the meantime.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#25

Post by satvikpendem »

Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

The book Blockchain Revolution covers this well. It postulates a decentralized autonomous organization (a bot, basically) that controls its money and can trade with other bots via smart contracts in cryptocurrency and self improve via machine learning. Its goal is to maximize profit, which it can do in various ways, both savory and unsavory to the human race.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#26

Post by jbay808 »

jkhdigital wrote: post #13
jp555 wrote: post #6

“Intelligence is situational — there is no such thing as general intelligence. Your brain is one piece in a broader system which includes your body, your environment, other humans, and culture as a whole. […] Currently, our environment, not our brain, is acting as the bottleneck to our intelligence.” “Human intelligence is largely externalized, contained not in our brain but in our civilization. We are our tools — ou…

Surprised he didn't mention the Flynn effect: https://en.wikipedia.org/wiki/Flynn_effect A book I recently read (Range, by David Epstein) seems to draw a connection through some other research that points to modernization and globalization as the drivers of this trend, in the sense that IQ tests measure pure abstract reasoning ability and we now live in a world dominated by abstraction so our brains get used to think…

> It is difficult to imagine anything resembling a "general" intelligence that isn't attached to a life-form[...]

How is that argument any stronger than these ones that (non-animal) heavier than air flight is impossible?

https://www.xaprb.com/blog/flight-is-impossible/

I think these are actually stronger arguments, because I can think of at least three very least significant material-science advantages that birds have even over modern technology that would, from a physics point of view, let me entertain the idea that a 19th century physicist could believe you'd at least need to be made of flesh to fly. I can't think of any comparable case for intelligence being restricted to biology.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#27

Post by _y5hn »

WalterBright wrote: post #22
jotm wrote: post #18

Earlier quoted context omitted.

I've been saying AI won't be Skynet launching the nukes, it will be a corporation maximizing productivity with complete disregard for human needs heh

In a free market, a corporation has to please its customers in order to maximize profit. For example, Apple.

Apple is a net negative in my perspective. They contribute negatively.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#28

Post by _y5hn »

Eliezer wrote: post #12
jp555 wrote: post #6

“Intelligence is situational — there is no such thing as general intelligence. Your brain is one piece in a broader system which includes your body, your environment, other humans, and culture as a whole. […] Currently, our environment, not our brain, is acting as the bottleneck to our intelligence.” “Human intelligence is largely externalized, contained not in our brain but in our civilization. We are our tools — ou…

Here's an extensive reply to Chollet's essay, also by the author of "There's No Fire Alarm": https://intelligence.org/2017/12/06/chollet/ > ...some systems function very well in a broad variety of structured low-entropy environments. E.g. the human brain functions much better than other primate brains in an extremely broad set of environments, including many that natural selection did not explicitly optimize for. We…

GDP is not an absolute measure.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#29

Post by jkhdigital »

jbay808 wrote: post #26
jkhdigital wrote: post #13

Earlier quoted context omitted.

Surprised he didn't mention the Flynn effect: https://en.wikipedia.org/wiki/Flynn_effect A book I recently read (Range, by David Epstein) seems to draw a connection through some other research that points to modernization and globalization as the drivers of this trend, in the sense that IQ tests measure pure abstract reasoning ability and we now live in a world dominated by abstraction so our brains get used to think…

> It is difficult to imagine anything resembling a "general" intelligence that isn't attached to a life-form[...] How is that argument any stronger than these ones that (non-animal) heavier than air flight is impossible? https://www.xaprb.com/blog/flight-is-impossible/ I think these are actually stronger arguments, because I can think of at least three very least significant material-science advantages that birds hav…

Yes, I realize that the phrase "difficult to imagine" is fun to pounce on. There's a big difference between "flight" and "general intelligence"; namely that the former is like Justice Potter Stewart's definition of pornography and the latter is... not.

Anyways, maybe what I'm suggesting is that if you can point to something and say "that is a (artificial?) general intelligence", then I suspect you will also feel compelled to say "that is (artificial?) life".

Definitely wandering off into the weeds here, but why not: I see evolution as a computational problem. In its most primitive, there is no computer, but any specific problem can still be solved with luck and memory. If you have a structure that recognizes a solution, then you just wait until you randomly bump into it. The challenges of evolution in a given environment are specific to that environment, and thus memory without a computer can go a really, really long way.

If human intelligence is general intelligence, it is still the logical progression of this chain of memory and (later on) compute operations. I can reason about the abstraction of general intelligence, but it feels like any concrete general intelligence will proceed through essentially the same steps. The general intelligence must reside in a concrete existence, and whatever form this takes I feel very strongly that it will be recognizable as what we call "life".

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#30

Post by reducesuffering »

jotm wrote: post #18
Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

I've been saying AI won't be Skynet launching the nukes, it will be a corporation maximizing productivity with complete disregard for human needs heh

It will be painfully obvious within the decade that this is exactly what's happening right now. Corporations are AI optimizing a single metric. They only use humans so much as they need them to improve the metric. As they get better, automation improves and humans are needed less and less. There's a reason so many feel like cogs in a machine.
Post reply on HN