Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence pioneer says we need to start over

axios.com

Post reply on HN
Print view View original

51–60 of 156 posts

Re: Artificial intelligence pioneer says we need to start over

#51

Post by fngl51 »

31reasons wrote: post #38
zebrafish wrote: post #34

I don't know. I think backprop is probably utilized a lot in biological networks. Isn't that why we take tests in school? Obviously backprop doesn't make sense in an unsupervised setting. There's no label to backprop on. But here's an example: I see a stove eye is black when cold, then when I see it turn red, I touch it. Ow, it hurts. That's supervised learning. Don't touch things that are glowing red when they don't…

I think Unsupervised Learning is fundamentally connected with the mission/goal of the AI. Whatever the mission is give to AI, it need to start learning information landscape by itself and make classification and use that knowledge to make predictions and act accordingly to optimized the outcome of the mission its given.

But, when you revise the model based an analyzing the predictions of the previous run and adapt to the positive result, isn't that supervised learning with back-propagation?

Re: Artificial intelligence pioneer says we need to start over

#52

Post by eanzenberg »

31reasons wrote: post #40
eanzenberg wrote: post #35

Earlier quoted context omitted.

Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.

I think innate intelligence/ animal intelligence is developed by evolutionary algorithm. You don't need to do back prop for it. If the network is wrong, the animal just dies.

What you call "evolutionary algorithm" is just a pre-trained model your DNA created. That took 3.5 billion years to get to this point.

Re: Artificial intelligence pioneer says we need to start over

#53

Post by zodiac »

sillysaurus3 wrote: post #26
iandanforth wrote: post #22

Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…

You need to learn without someone telling you what the answer is. Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. There is: Death, without reproducing. If an organism dies quickly but left children, they were biologically successful. If you didn't, then your evolutionary branch goes away. I'm not sure this can be simul…

Of course you need some objective function and feedback about how well you're doing wrt that. In RL this is true too. The feedback needed for backpropagation might be, for each action you take in life, what your chance of death is right after that action

Re: Artificial intelligence pioneer says we need to start over

#54

Post by eanzenberg »

felippee wrote: post #44
eanzenberg wrote: post #35

Earlier quoted context omitted.

Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.

Except that by the age you reach school you already know amazing wealth of knowledge about the world and how it works, particularly the kind of commonsense knowledge that AI has always had enormous trouble to acquire. See Moravec's paradox.

5 year olds "know amazing wealth of knowledge about the world and how it works"?? Que? How many 5 year olds do you know or interact with?

Re: Artificial intelligence pioneer says we need to start over

#55

Post by zardo »

Animats wrote: post #45

Machine learning may be nearing its ceiling. The history of AI goes in cycles. Someone has a good idea which solves some problems, followed by "strong AI Real Soon Now" enthusiasm, followed by that idea hitting its ceiling. AI has been through search, backtracking, the General Problem Solver, hill-climbing, and expert systems. Each was overhyped at the time, and each hit its ceiling. The big difference this time is t…

>The big difference this time is that the ceiling with machine learning is high enough for large-scale profitable applications. That wasn't the case with the previous rounds.

My company (fortune 500) depends on expert systems, some even date back to the 80's AI boom. Doing what we do, at the scale we do it, would simply not be possible without them.

Lots of companies make a lot of money with expert systems, but they have no incentive to fund AI research since expert systems have hit a clear capability ceiling.

Re: Artificial intelligence pioneer says we need to start over

#56

Post by 52-6F-62 »

ethnic_throw wrote: post #16
52-6F-62 wrote: post #9

Earlier quoted context omitted.

That's the one. I'm actually just finishing it up now, so it's fresh enough in my mind that it jumps out at me in application.

I'd be careful reading it: Kuhn is excellent at building narratives, and what is more narrative friendly than revolution? In reality science is far more messy than he acknowledges. Unfortunately while I think many of Kuhn's observations are interesting, I'm confused what you think scientists might learn from them. After all, the work in question is science, not philosophy of science, and frankly it seems a lot of tim…

Oh well I didn't think I'd applied any subjective notion to my recognition that it sounds like the paradigmatic event that Kohn describes. And having recently gone through the book it was the first thing I noticed.

My admiration for Hinton there lay separate from my remark about Kuhn.

I'm not quite sure where you got that I think scientists must learn from Kuhn's observations. But I will say, that while it's kept me away from my other studies (for the obvious displacement of reading one thing and not the other), I don't see the harm in indulging in such reflection. But my background isn't engineering, mathematics, or physical sciences before my more recent work. It's in humanities and the arts where multidisciplinary studies are encouraged. I suppose I'll probably always carry that with me. If nothing else, perspective is a good thing to have -- and not all thoughts, reflections, or ideas entertained need be engaged.

At any rate, the book was recommended and I try to remain open minded.

Re: Artificial intelligence pioneer says we need to start over

#57

Post by function_seven »

eanzenberg wrote: post #50
iandanforth wrote: post #41

Earlier quoted context omitted.

Can we go with 'incomplete' rather than 'false'? :) One of the most interesting areas of study for me is early childhood motor learning which is a fascinating combination of unsupervised, reinforced, and imitation learning. There are even significant intrinsic knowledge (instinctual/reflexive/morphological) effects. Also I really like to encompass non-human learning when talking about this stuff to remind myself of a…

Please describe what "unsupervised learning" a child may be doing.

Language acquisition. Sure, parents will point at an object and say, "Ball", "Chair", "Food", etc. But they don't do this for the conjunctions and interjections and prepositions. Nor for most of the 10,000 words a child will learn in the first few years of life.

Children learn the rules of grammar long before they realize there are any rules at all. Tenses and plurals, etc.

Re: Artificial intelligence pioneer says we need to start over

#58

Post by throwaway613834 »

zardo wrote: post #55
Animats wrote: post #45

Machine learning may be nearing its ceiling. The history of AI goes in cycles. Someone has a good idea which solves some problems, followed by "strong AI Real Soon Now" enthusiasm, followed by that idea hitting its ceiling. AI has been through search, backtracking, the General Problem Solver, hill-climbing, and expert systems. Each was overhyped at the time, and each hit its ceiling. The big difference this time is t…

>The big difference this time is that the ceiling with machine learning is high enough for large-scale profitable applications. That wasn't the case with the previous rounds. My company (fortune 500) depends on expert systems, some even date back to the 80's AI boom. Doing what we do, at the scale we do it, would simply not be possible without them. Lots of companies make a lot of money with expert systems, but they…

> Lots of companies make a lot of money with expert systems, but they have no incentive to fund AI research since expert systems have hit a clear capability ceiling.

I'm not sure I follow the reasoning here. Is the suggestion that AI cannot do better than expert systems?

Re: Artificial intelligence pioneer says we need to start over

#59

Post by function_seven »

eanzenberg wrote: post #54
felippee wrote: post #44

Earlier quoted context omitted.

Except that by the age you reach school you already know amazing wealth of knowledge about the world and how it works, particularly the kind of commonsense knowledge that AI has always had enormous trouble to acquire. See Moravec's paradox.

5 year olds "know amazing wealth of knowledge about the world and how it works"?? Que? How many 5 year olds do you know or interact with?

You're severely underestimating children (or over-estimating the current state of AI). 5 year-olds have better natural language skills than any AI in existence, better motor and navigation skills than the most advanced DARPA robot, better mood recognition capabilities, etc.

You appear to be thinking of just facts and figures. AI is not that.

Re: Artificial intelligence pioneer says we need to start over

#60

Post by phkahler »

iandanforth wrote: post #22

Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…

>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against.

Is it possible that your conscious self is the oracle that provides feedback to subconscious networks?

Post reply on HN