AI is mostly about curve fitting (2018) • Hacker News Board

Re: AI is mostly about curve fitting (2018)

#41

Post by yk » Sat, Nov 23, 2019, 3:15 PM UTC

galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card.

So, you can argue something like our brains are nothing but curve fitting machine with enough parameters, but then you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness.

There are of course two possible ways around that, either one can argue for p-zombies, that is intelligent but not conscious beings, which then seems to require a super natural explanation for consciousness. Or you can argue that the brain is different, and that this gives rise to consciousness, and to general intelligence, which is the explanation that at least corresponds most closely to my subjective experience (but that is precisely what a soulless machine would write, isn't it?)

Re: AI is mostly about curve fitting (2018)

#42

Post by queensnake » Sat, Nov 23, 2019, 3:16 PM UTC

Out of date. Reinforcement Learning makes truly ‘smart’ actions, courses of action. Calling something ‘curve fitting’ pooh poohs it (especially ‘just’ curve fitting) - calling it ‘distilling’ gives it the respect that it deserves.

Re: AI is mostly about curve fitting (2018)

#43

Post by H8crilA » Sat, Nov 23, 2019, 3:19 PM UTC

It's still not clear to me that natural intelligence is anything more than curve fitting.

Re: AI is mostly about curve fitting (2018)

#44

Post by gus_massa » Sat, Nov 23, 2019, 3:19 PM UTC

crazygringo wrote: post #3

galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

I was say fundamentally no -- and Douglas Hofstadter is probably the foremost spokesperson against intelligence being curve fitting. General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) and drawing cause-and-effect inferences that assist us in achieving goals . Curve fitting is just another name for pattern recognition, mapping to previously defined categories…

> General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) …

There are some algorithms like https://en.wikipedia.org/wiki/K-means_clustering that get a set of data and try to create the categories to better classify them. There are many algorithm and the results don't agree all the time. But this is an open ended task, like the classification of biological species in animals. (Plants are more difficult, and bacterias even more.)

Re: AI is mostly about curve fitting (2018)

#45

Post by [deleted] » Sat, Nov 23, 2019, 3:19 PM UTC

crazygringo wrote: post #3

galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

I was say fundamentally no -- and Douglas Hofstadter is probably the foremost spokesperson against intelligence being curve fitting. General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) and drawing cause-and-effect inferences that assist us in achieving goals . Curve fitting is just another name for pattern recognition, mapping to previously defined categories…

[deleted]

Re: AI is mostly about curve fitting (2018)

#46

Post by account73466 » Sat, Nov 23, 2019, 3:21 PM UTC

ipsa wrote: post #33

"Machine learning" used to be a safe haven. You could flee there to escape the Terminators and brain-on-a-chip graphics. Business PR deliberately killed that. They wanted their ML algorithms to be refered to as AI, so they could fully ride the hype train. AI used to be a tight quirky community. Having the brain as inspiration led to all sorts of anthropomorphizing. This was ok. Researchers understood what was meant w…

Very well said. Also, curve fitting is not a corner case. Most relevant and intelligent things we care about can be solved with "just" curve fitting + extrapolation.

Re: AI is mostly about curve fitting (2018)

#47

Post by scottlocklin » Sat, Nov 23, 2019, 3:21 PM UTC

tomthe wrote: post #16

Well, most of science is also just curve fitting.

Pretty sure Maxwells Equations and the laws of thermodynamics didn't come from curve fitting!

Experimentalists may have verified these ideas using some kind of curve fitting, but thinking in abstractions, aka, "a ball rolled out in the street, maybe a child will follow," is one of the things curve fitting can't do.

Re: AI is mostly about curve fitting (2018)

#48

Post by gus_massa » Sat, Nov 23, 2019, 3:21 PM UTC

wayoutthere wrote: post #27

dr_dshiv wrote: post #15

So, one view of AI is that is is machine learning. Another view is that it consists of automated process. So, if expert systems count as AI, or if production rule systems count as AI, then pretty much any handcrafted if-then statement counts as AI. And, so too might any automated process.. This is has the advantage of recognizing how human intelligence can be automated and aggregated into system processes. And, the d…

I too hold the view that artificial intelligence is not reliant on computers. Sufficiently complex business logic is impossible for a single person -- even the CEO -- to have an end-to-end view (much less have a significant influence on). The emergent behavior of large companies fits a definition of an AI today; human "decision makers" are increasingly just reviewing and approving the output of algorithms. I think th…

I agree. Just note that it is related to https://en.wikipedia.org/wiki/Chinese_room

Re: AI is mostly about curve fitting (2018)

#49

Post by Isinlor » Sat, Nov 23, 2019, 3:24 PM UTC

galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

If someone wants to go as far as to claim that any computation is just curve-fitting then your statement is equivalent to Church–Turing thesis. There are no formal arguments against Church–Turing thesis.

From that perspective intelligence is indeed just a curve fitting.

https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

I really enjoyed the "The Measure of Intelligence" by François Chollet. https://arxiv.org/abs/1911.01547

"We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power."

He argues that we should move towards evaluating "Intelligence as skill-acquisition efficiency".

I agree with him. We should move away from benchmarks that involve training and evaluating algorithms on the same datasets. This is indeed more or less "curve fitting". Instead we should focus on benchmarking how efficient algorithms are at solving tasks involving completely new datasets, preferably even unknown to the developers. For example, language model GPT-2 was trained to predict next word given some previous words. After that training GPT-2 was able to do things that were unrelated like question answering, translating etc. GPT-2 is of course doing that very badly, and requires GB of training data, but it is a step towards skill-acquisition efficiency and away from what everyone sees as curve-fitting.

We should benchmark models so that we select for these that are able to do solve tasks they were not build to solve.

Re: AI is mostly about curve fitting (2018)

#50

Post by streetcat1 » Sat, Nov 23, 2019, 3:27 PM UTC

bitL wrote: post #28

streetcat1 wrote: post #22

So RL is true AI. Alpha GO did make moves inconceivable to the best human mind on the subject. There was no curve to fit into.

The learning from game itself was curve fitting, the Deep in Deep Reinforcement Learning usually means some difficult function is replaced by a deep neural network, approximating optimal values (for moves) trained on gameplay samples, usually in sense of rewards/punishments for reaching certain states; in games they could rank e.g. good/bad moves, winning states, losing states etc.

Right. But the curve itself, was invented by the machine.