There’s No Fire Alarm for Artificial General Intelligence (2017) • Hacker News Board

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#61

Post by narag » Thu, Jun 04, 2020, 8:07 AM UTC

cwyers wrote: post #35

Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

> The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." As soon as an AI CEO starts trying to maximize a sin…

How about profit as a well-defined metric?

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#62

Post by abainbridge » Thu, Jun 04, 2020, 8:07 AM UTC

ilaksh wrote: post #39

Hnrobert42 wrote: post #34

Earlier quoted context omitted.

Successfully respond to my basic questions for help at my bank website.

"Basic" is very broad. What questions?

I think the broadness is the problem. If it wasn't broad, you could hard code answers to all the questions. Examples of some basic questions that have no chance of working today: "Do you remember that discussion we were having last time?", "I've scanned the document you want, can I paste it straight into the chat window?", "Is PDF format OK?", "Can you look at page 2 in the PDF?".

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#63

Post by mehrdadn » Thu, Jun 04, 2020, 8:10 AM UTC

ajuc wrote: post #59

mehrdadn wrote: post #44

Earlier quoted context omitted.

Okay, sure. :-) Take the set of people [1] who: - Have had an email account for at least 5 years, - Have checked their spam folder at least once every month over the course of the past two years, and - Have marked a set of emails as spam on at least 1 day every three months over the course of the past two years. I'm very confident that, within the next 2 years, we will not have an accurate enough classifier that woul…

That's cheating because improving AI is working on both sides - it's making better classifiers but also better spam generators :) In the future with general AI classifying spam will be a hard problem even for people. Imagine you get a message looking like it's from your facebook friend with his catchphrases telling you about his last trip and saying how great that travel agency was :) Spam or not?

Quite an interesting point I hadn't considered at all. On the one hand I'm wondering: what's your suggestion on how to address this with minimal changes to my criteria? On the other hand I'm wondering: well, if the analog of this is that AGI might get solved with more AGI, then that's only going to make me less likely to be worried in the first place!

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#64

Post by Zaak » Thu, Jun 04, 2020, 8:26 AM UTC

jp555 wrote: post #6

“Intelligence is situational — there is no such thing as general intelligence. Your brain is one piece in a broader system which includes your body, your environment, other humans, and culture as a whole. […] Currently, our environment, not our brain, is acting as the bottleneck to our intelligence.” “Human intelligence is largely externalized, contained not in our brain but in our civilization. We are our tools — ou…

> “Recursively self-improving systems, because of contingent bottlenecks, diminishing returns, and counter-reactions […], cannot achieve exponential progress in practice. Empirically, they tend to display linear or sigmoidal improvement.”

Moore's Law. Use computers to make better computers. Exponential growth over more than 7 orders of magnitude and, although the growth rate is slowing, it hasn't run out of steam yet.

If AI eventually exhibits anywhere near that level of recursive self-improvement, godlike superintelligences lie in our future.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#65

Post by [deleted] » Thu, Jun 04, 2020, 8:27 AM UTC

pgt wrote: post #52

memexy wrote: post #41

Earlier quoted context omitted.

Proving theorems. I will nominate Cauchy's Residue Theorem. It has already been formalized so a neural network or any generally intelligent agent should be able to do the same: https://www.cl.cam.ac.uk/~wl302/publications/itp16.pdf . Complex analysis in general is one of the more fun parts of math and I doubt it's getting automated any time soon. It also has a lot of application in signal processing ( https://www.quo…

Depending on who's holding the hammer, I do fear the hammer.

[deleted]

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#66

Post by mehrdadn » Thu, Jun 04, 2020, 8:30 AM UTC

abainbridge wrote: post #62

ilaksh wrote: post #39

Earlier quoted context omitted.

"Basic" is very broad. What questions?

I think the broadness is the problem. If it wasn't broad, you could hard code answers to all the questions. Examples of some basic questions that have no chance of working today: "Do you remember that discussion we were having last time?", "I've scanned the document you want, can I paste it straight into the chat window?", "Is PDF format OK?", "Can you look at page 2 in the PDF?".

I think just "No" would work as an answer to all of those questions. (Tongue only halfway in cheek...)

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#67

Post by barry-cotter » Thu, Jun 04, 2020, 8:37 AM UTC

ajuc wrote: post #57

If you told people in 90s that at one point in future AIs will be able to: - drive a car in traffic in almost all circumstances safer than majority of human drivers basing only on video input - take a video and replace faces from another video well enough that more than 90% of people are fooled - have 95%+ accurate OCR and speech recognition - predict people's preferences regarding music/movies/books better than any…

> predict people's preferences regarding music/movies/books better than any human could

What recommender algorithm is this? Amazon certainly isn’t using it.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#68

Post by nopinsight » Thu, Jun 04, 2020, 8:42 AM UTC

Our civilization and ability to dominate over other animals largely exist because of human intelligence. An AGI could be a great partner toward unprecedented prosperity, but an AGI not fully aligned with human values may also spur a major catastrophe.

Two promising approaches toward AI Safety I have encountered:

* Make sure the AI is not too certain of itself and will always defer to humans as the final judge. https://www.ted.com/talks/stuart_russell_3_principles_for_cr...

* Get the AI to learn and internalize human values. A possible technique is Inverse Reinforcement Learning: https://thegradient.pub/learning-from-humans-what-is-inverse...

Both are being developed by Stuart Russell among others. See his book here: https://en.wikipedia.org/wiki/Human_Compatible. The latter approach was also discussed on several occasions by Ilya Sutskever.

Many believe AGI will not be realized for at least a few decades hence but no one can really be certain it will not be developed before then.

Since several different approaches together are better than one. Bright minds including those outside the field should propose more ideas for such a significant & unique problem of our time.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#69

Post by opportune » Thu, Jun 04, 2020, 8:48 AM UTC

andrewflnr wrote: post #21

Animats wrote: post #16

The decisive moment for AI will come when a program can run a corporation better than humans. That's not an unreasonable near-term achievement. A corporation is a system for maximizing a well-defined metric. Maximizing a well-defined metric is what machine learning systems do well. If a system listened in on all communication within a company, for traffic, sentiment analysis, who responds to whom how fast, and what c…

> A corporation is a system for maximizing a well-defined metric. Interesting theory. My perspective is that once a corporation thinks of itself that way, it's already going downhill. Once a measure becomes a target, it ceases to be a good measure. Now I assume you're imagining the well-defined metric to be profit, in attempt to dodge the aforementioned problem. However, even leaving aside the problem of short- vs lo…

But surely a general intelligence would recognize that maximizing short term profit at the expense of long term profits has its own set of trade offs. And indeed, that common sense or “je ne sais quoi” would also reasonably indicate general intelligence.

I think the “running a corporation” problem is actually a great example of how to measure a general intelligence not only because it has well defined success criteria but also because of the very open ended and general means by which it must be accomplished. Running a corporation requires communication, setting goals, creativity, number crunching, and lots of soft skills like branding or understanding how consumers/customers will react to things. But there is nothing uniquely human about it except that it needs to be able to make money from humans. Presumably if I’m paying for some well defined service like Search Ads I don’t particularly care if the corporation is run by a robot.

I do agree though that there is a bit of ambiguity here on what “run a corporation” means. I assume that with a lot of thought I could write a very procedural script that somehow incorporates, purchases a property, and rents it through a property management group.

Re: There’s No Fire Alarm for Artificial General Intelligence (2017)

#70

Post by SCdF » Thu, Jun 04, 2020, 8:50 AM UTC

ajuc wrote: post #57

If you told people in 90s that at one point in future AIs will be able to: - drive a car in traffic in almost all circumstances safer than majority of human drivers basing only on video input - take a video and replace faces from another video well enough that more than 90% of people are fooled - have 95%+ accurate OCR and speech recognition - predict people's preferences regarding music/movies/books better than any…

> drive a car in traffic in almost all circumstances safer than majority of human drivers basing only on video input

Has that happened? Don't self driving cars use a huge range of sensors, and are they actually legal / widely deployed anywhere at all? Are any for sale? I don't mean improved cruise control, I mean "OK computer, take me to work".

> take a video and replace faces from another video well enough that more than 90% of people are fooled

Are you talking about deepfakes? I haven't seen an example that was anything other than creepy, and really really obvious. Do you think I'm just really sensitive to this kind of thing, or have I not seen the really good examples?

> predict people's preferences regarding music/movies/books better than any human could

Why do you think that's happened? Spotify regularly recommends me things I don't like, but I've also never had a dedicated human butler that recommends me music so it's hard to compare, but but I don't think spotify is doing any better than last.fm was doing a decade ago (if I had to have an opinion I'd say it was worse).

---

I don't want to sound rude or that I'm jumping down your throat, but I don't really feel like we live in the future you're describing, at least not today.