Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is mostly about curve fitting (2018)

diginomica.com

Post reply on HN
Print view View original

71–80 of 192 posts

Re: AI is mostly about curve fitting (2018)

#71

Post by nextos »

account73466 wrote: post #46
ipsa wrote: post #33

"Machine learning" used to be a safe haven. You could flee there to escape the Terminators and brain-on-a-chip graphics. Business PR deliberately killed that. They wanted their ML algorithms to be refered to as AI, so they could fully ride the hype train. AI used to be a tight quirky community. Having the brain as inspiration led to all sorts of anthropomorphizing. This was ok. Researchers understood what was meant w…

Very well said. Also, curve fitting is not a corner case. Most relevant and intelligent things we care about can be solved with "just" curve fitting + extrapolation.

I think curve fitting is an important component of future AGI. But it definitely needs causal reasoning baked in, which leads to better models with less data [1,2].

My intuition is that there's a lot of important work to be done using logical representations of models and transforming them back and forth using well understood semantics operators. Deep functions will be part of said models, but the whole model does not necessarily need to be deep. We can already see hints of the field going in this direction in deep generative models [3].

[1] http://web.stanford.edu/class/psych209/Readings/LakeEtAlBBS....

[2] https://probmods.org/

[3] http://pyro.ai/examples/

Re: AI is mostly about curve fitting (2018)

#72

Post by lostmsu »

ryeguy_24 wrote: post #66

It was so refreshing to read the title of this article. This couldn’t be more true and timely. Everyone and their brother is talking about artificial intelligence and it’s frankly annoying at this point. These models are, simply put, just fancy interpolation/ extrapolation approaches.

And we are not?

Re: AI is mostly about curve fitting (2018)

#74

Post by sgt101 »

Isinlor wrote: post #49
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

If someone wants to go as far as to claim that any computation is just curve-fitting then your statement is equivalent to Church–Turing thesis. There are no formal arguments against Church–Turing thesis. From that perspective intelligence is indeed just a curve fitting. https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis I really enjoyed the "The Measure of Intelligence" by François Chollet. https://arxiv.org/…

>There are no formal arguments against Church–Turing thesis.

do you mean "all computation that can be done, can be done using a Turing machine"? Or do you mean "no one has proven Church Turing wrong?"

If it's the second - yes, that's so. But so what?

If it's the first then many people in Quantum Computing community will be quite upset, P=BQP? You have proof?

Re: AI is mostly about curve fitting (2018)

#75

Post by joycian »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

Why is it weird that simple programs have a sliver of consciousness? That seems like a nice practical, non-magic conclusion?

Re: AI is mostly about curve fitting (2018)

#76

Post by missosoup »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

There's no need for a supernatural explanation of consciousness.

We are all p-zombies. Problem solved.

The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in something we can't communicate with and that consciousness is a sliding scale rather than a binary property. Ascribing consciousness only to ourselves is hubris.

If one spends a bit of time observing humans, they will inevitably realise that some humans are more 'conscious' than others also.

tl;dr: we're all p-zombies. The fact that we think that each one of us isn't individually doesn't detract from that.

Just as natural sciences left less and less hiding places for god to exist, ML is leaving less and less hiding places for this borderline magical version of human-unique consciousness to exist. Answering this question in any more detail requires a much more rigorous definition of consciousness which is a big can of worms in itself.

Re: AI is mostly about curve fitting (2018)

#77

Post by lordnacho »

I actually started reading the Book of Why and I recommend it to the HN crowd. Pearl does a really good job of going through the history of causality, including the quite interesting story behind the now known to be wrong (or at least incomplete) claim that "correlation is not causality".

He then places causality on a 3-rung scale. The bottom rung is association in data, which is where he says AI is stuck. Then there's intervention, "what if I do this thing?", and then there's counterfactuals, "what if I had done some other thing?"

He then makes a case for what intelligence actually is, and unsurprisingly it's getting up those three rungs. The method revolves around directed graphs, which have certain unintuitive properties. For instance if A and B can both cause C, knowing that A is unlikely make B more likely, given that C happened. There's a few other stock situations in various graphs that he walks through as well.

In the end the point seems to be that we could have a causal machine if we'd spend some more time on it. It would take data and try out some potential graphs, and some of the graphs would be ruled out by the data. And then some algo would tell you things like whether a randomized trial is necessary or even whether you'd need one (yes, this is another revelation).

I think there's also an argument that this is how people actually think, which makes sense because the graphs are not terribly large and they need to fit in your meat hardware. I haven't finished it but I would guess that you could take it in another direction and say this is why some animals sorta have intelligence in that they learn patterns, but they don't know the higher rungs.

Really interesting ideas, and at least clarifies what we mean by causality.

Re: AI is mostly about curve fitting (2018)

#78

Post by lasagnaphil »

umarniz wrote: post #38

I think intelligence has for some time already been boiled down to curve fitting, even for humans. Our current accepted definition of intelligence in schools is to get a score that is higher than the average to be considered sufficiently intelligent to proceed to the next grade. I feel anything that we develop for AI would fundamentally always be inspired by our own experiences and hence curve fitting is something we…

I agree. In our current neoliberal era, the ones in power have already been replacing various systems in our society with algorithms and computers, and formulating everything into an optimization problem. As a result of this, the reverse has happened: the systems are now shaping humanity into something that could be optimized.

We have been trying to manage governments, public services, and education the same as corporations, creating numerical targets for institutions to optimize for. Education itself was formulated as an optimization problem about how to create more jobs. Public services like healthcare were privatized and became a target for profit optimization. Half of the stock market is controlled by High-Frequency Trading supercomputers, which will do virtually anything to gain an upper-hand in profits. Those methods were all inherited from the management styles of corporations that began in the neoliberal era. As fundamental parts of our society are replaced by those systems, the society now curve-fits the systems rather than the systems curve-fitting the society. We now hyperoptimize ourselves to fit in this neoliberal landscape; our time is told as something to be optimized between work, socializing, exercising, and self-improving, with no space for "actual free time" of our own. We go to college not to learn but to pass exams and get ourselves a good job that can sustain us. And the faults of our systems are now blamed to be individual problems: "You didn't optimize towards the current trends of the job market, it's your fault." And from the view of the corporations, we are just AI agents waiting to be optimized for cash, and we're now becoming one through fitting our bodies and minds to the social media that tries to maximize engagement and ad revenue no matter the real societal cost.

Now, the real problem of AI compared to the algorithms of the past is its data-driven nature: it can only learn from what data you give it for training. We can only accumulate data from the past and never from the future, so the AI systems will just keep repeating the past, no matter what unseen change will come. We will lose the ability to imagine new political, economic alternatives, we will just be feeding ourselves the status quo, and societal advancement will stagnate at the hands of automated systems. The cancellation of the future: this is what I'm ultimately afraid of.

Re: AI is mostly about curve fitting (2018)

#79

Post by [deleted] »

ur-whale wrote: post #6
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

First, no AI that I know of has at its disposal a full blown model of the world it operates in, whereas most human brains do, and even if the model is imperfect, it is capable of producing fairly accurate simulations (what-if scenarios). Second, deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting (that is, within the confines of the model)…

[deleted]

Re: AI is mostly about curve fitting (2018)

#80

Post by hr45eahear5 »

ipsa wrote: post #33

"Machine learning" used to be a safe haven. You could flee there to escape the Terminators and brain-on-a-chip graphics. Business PR deliberately killed that. They wanted their ML algorithms to be refered to as AI, so they could fully ride the hype train. AI used to be a tight quirky community. Having the brain as inspiration led to all sorts of anthropomorphizing. This was ok. Researchers understood what was meant w…

> you do not need to fit a curve to see that

You haven't proven this statement. It's possible within your own brain is nothing more than a rudimentary curve fitting algorithm that allowed you to see this pattern.

Post reply on HN