Live data from Hacker News

Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

medium.com

21–30 of 92 posts

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#21
post #7

>"However, the current focus on doing AI research via the gathering of data, the deployment of “deep learning” infrastructure, and the demonstration of systems that mimic certain narrowly-defined human skills — with little in the way of emerging explanatory principles — tends to deflect attention from major open problems in classical AI. These problems include the need to bring meaning and reasoning into systems" I'd…

"When you get attacked in the forest by a guy in a clown costume with an axe you don't need to add that as a training input first before you make a run for it." Sure, because it's already a training input. We'd run because we recognize the axe, the signs of aggression, the horror movie trope of an evil clown, and so forth. We have to teach "stranger danger" to children. "There's no agency, liveliness, autonomy or lea…

>Sure, because it's already a training input

not in any meaningful sense, no. I can tell you, "if something's fishy about the situation, just leave". You can do this not because of some particular training inputs or examples I give you, but because you have common sense and a sort of personality and intuition for how to behave in the absence of data. If you told that sentence to a state of the art ML model you'd probably get "what fish?" as an answer.

>Well, that's at least in part because we design them that way

It's mostly because we have no idea how to design them anyway else. I think if anyone knew how to build complex agents with rich internal states that have the intent and communication abilities of humans we'd do that. It's not even really conceivable right now how you could have an ML type system that also can just directly adopt high level concepts dynamically just by communicating them.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#22
As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#23
post #22

As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?

This comparison is not perfect for various reasons. For example, the average firing rate of neurons is pretty low. Most attempts at comparison are done using FLOPS, e.g. https://www.openphilanthropy.org/research/how-much-computati.... You may prefer this comparison using Traversed Edges Per Second: https://aiimpacts.org/brain-performance-in-teps/.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#24
post #22

As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?

That’s only 3 orders of magnitude off from today’s largest models like PaLM (5x10^11 parameters), a gap that’s narrowed by 3 orders of magnitude just since 2019.

How far away do you think we are, exactly?

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#25
post #24
post #22

As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?

That’s only 3 orders of magnitude off from today’s largest models like PaLM (5x10^11 parameters), a gap that’s narrowed by 3 orders of magnitude just since 2019. How far away do you think we are, exactly?

Thank you for this information. I did not know this. But my view (I may be wrong), is that AGI is too resource-intensive to be within the reach of normal computing of the ordinary user for at least 2 decades.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#26
post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Biological brains have had a few billion years to optimize. Over the past decade or two, it's been increasingly apparent that the structure and algorithms that govern a particular neural net's behaviour are extremely important to its efficacy. We likely have a very warped view of what intelligence is, because the most prominent examples of it have been aggressively honed over an extremely long period of time to be go…

> Biological brains have had a few billion years to optimize.

Not just that, but it's grown on a body which has been optimized for survival during a few billion years; and that body is built on cells that have evolved to survive hostile environments, and those cells are built with self-replicating molecules, evolving from complex chemical reactions in several changing environments, that competed with and displaced other less-successful self-replicating molecules that disappeared.

Each of those layers provides a degree of adaptability and self-healing that is extremely hard to replicate. And if we managed to reverse-engineer and replicate one of those layers, it would still be missing all the layers below.

Our best hope to create fully independent agents will come from re-adapting and controlling biological entities, not from tools built from the ground up with current engineering techniques.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#27
post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Biological brains have had a few billion years to optimize. Over the past decade or two, it's been increasingly apparent that the structure and algorithms that govern a particular neural net's behaviour are extremely important to its efficacy. We likely have a very warped view of what intelligence is, because the most prominent examples of it have been aggressively honed over an extremely long period of time to be go…

Multi-cellular life has only been around for 600 million years or so.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#28
post #19
post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…

I think there's a lot of truth to this. I'm new to ML, still going through the ropes on some online courses, but already I can see that, once I get a bit of muscle memory in setting up models etc, there's a whole lot of power and efficiency to be unlocked by using simple models - specifically in CS/X and Marketing. Obviously model quality matters, so you have to have proper monitoring etc, but this stuff is low hanging fruit and should enable teams to be so much more efficient.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#29
post #19
post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…

> There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve.

https://en.wikipedia.org/wiki/Moravec%27s_paradox

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#30
post #19
post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…

I dont't think it's intuition. There's a whole field of junk economics dedicated to telling us that your position in the economic class hierarchy determines the automatability of your job. In general it goes unquestioned. A vast amount of capital is also deployed based upon this assumption.

This is an example paper that, for instance, mathematically blurred the distinction between offshoring and automation:

https://talkbusiness.net/2017/07/ball-state-study-automation...

There was another paper (that i cant find right now) that basically surveyed people about how creative they thought their job was and just assumed that creativity was inversely proportional to automatability.

Ironically I think a widespread belief in this myth helped, among other things, lead to the trucker shortage. Who wants to join a profession with a high barrier to entry that they believe will be automated soon?

Post reply on HN