Live data from Hacker News

AI is mostly about curve fitting (2018)

diginomica.com

51–60 of 192 posts

Re: AI is mostly about curve fitting (2018)

#51

Is it feasible to create AI on top of a bunch of other AI? Current AIs seem more like specific, fairly simple, brain regions. Maybe we need a level up.

Yes, it's done all the time. Suppose you have a model A that predicts some value for input x and you also have model B for the same problem, and they both work, but are not optimal for all cases, so they do the following:

   C(x) = alfa * A(x) + (1-alfa) * B(x)
tada. nil nove sub sole

Re: AI is mostly about curve fitting (2018)

#52
post #33

"Machine learning" used to be a safe haven. You could flee there to escape the Terminators and brain-on-a-chip graphics. Business PR deliberately killed that. They wanted their ML algorithms to be refered to as AI, so they could fully ride the hype train. AI used to be a tight quirky community. Having the brain as inspiration led to all sorts of anthropomorphizing. This was ok. Researchers understood what was meant w…

> The researchers actually busy solving the hard problems of vision, natural language understanding, and common sense, do not have time to write books about how AI is not yet general.

I've come to terms with the hype. There are still researchers doing the hard theoretical work, and they will still be toiling away after the next economic downturn. We can all choose every day whether to find fulfillment through seeking attention from other people, money, or satisfying our curiosity to solve problems.

> Nobody from the research community ever claimed that [AGI], nobody came forward to claim they've solved these decade-old problems. It is people selling books railing against the popular reporting of AI. Boring, self-serving, and predictive, and you do not need to fit a curve to see that.

Hear hear! That said, this is a good article by a respected researcher. Here's what LeCun had to say about it,

> ...In general, I think a lot of people who see the field from the outside criticize the current state of affair without knowing that people in the field actively work on fixing the very aspects they criticize.

> That includes causality, learning from unlabeled data, reasoning, memory, etc. [1]

[1] https://www.facebook.com/yann.lecun/posts/10156387222842143

Re: AI is mostly about curve fitting (2018)

#55

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

I am not accusing you of doing this in a bad way, but it kinda begs the question to argue that all intelligence is curve fitting, because if it is to be looked at that way, then we don't know the characteristics of the hyperspace on which it can be defined as "just" curve fitting, which is an important aspect of being able to carefully say that it is "just" curve fitting. While there almost certainly is such a hyperspace, just by virtue of the fact we're leaving ourselves so many degrees of freedom in our unspecified speculations here that it can't help but cover everything we ever do, we can't be confident that if we did know everything involved that it would be anywhere near the best representation, which is the only standard we can have in terms of whether or not something is "really" curve fitting. It isn't that hard to imagine that while an n-dimensional space can be defined upon which our intelligence is "curve fitting" that there is some better computational paradigm that would both describe it in fewer free parameters (or, in this case, a big-O different number of free parameters, probably) and also be easier to work with computationally, in which case we would have a solid ground to stand on to say that, no, it isn't really just "curve fitting".

Or to bring it down to Earth in another way, consider just the act of writing a program in the modern world. If you work really, really hard, you can define a space in which our act of programming is just "curve fitting"... but it's far from obvious that that is even remotely a sensible way to look at the world. (See "differentiable programming" for the best counterpoint I know to that: https://en.wikipedia.org/wiki/Differentiable_programming but it's a very small niche right now.) When I'm debugging a program there is almost never any utility at all in trying to think about it as a "curve" and trying to get it closer to a the "correct" curve. A Turing-complete-complex space can be described as curves, but those curves are just awfully complicated and I don't see how it would be a help.

My personal suspicion is that while our cognition involves rather less of this "Turing complete" thinking than we'd like to fancy ourselves using, we do irreducibly use elements of it [1], and as long as our best AI models are incapable of representing Turing-complete computations there is simply no chance of them being the answer to true human-scale cognition. (We do have models that can do it, e.g., evolutionary computation, but we lack any sensible idea of how to "update" such models like a neural net. Neural nets themselves in the simplest case aren't Turing complete, and none of the hybrid models seem to get there to me either, though I welcome correction on that point.)

[1]: Evidence: I don't think we could program Turing-complete machines if we were incapable of thinking that way ourselves. We aren't necessarily great at it, our engineering techniques are deeply characterized by the fact we can't really manipulate very many things at once in this manner and we have no choice but to break things up into very small modules and for us to combine them in a way that means that at any given time we have only a very small number of things to keep track of locally, but we are still doing non-trivially more than zero of the Turing-complete style of thinking. It isn't a hard guess from there to think that even if we aren't all that great at a full mathematical manifestation of this style of thinking, we may indeed be doing something somewhere between what our current neural nets do and this full TC-style thinking at a larger scale, and the inability to capture this in our neural nets is a currently-fatal-flaw.

Re: AI is mostly about curve fitting (2018)

#57
Great article. Why does AI hype persist? Simply because we're seeing moderate gains?

Newsflash- We're likely to continue seeing gains from ML tech for decades. Basic techniques may become part of core CS. With all the private siloed data out there, ML will need to be applied uniquely over and over.

In decades to come, if marketing continues as-is, are salesmen going to keep laying claim to the moon (AGI) the whole time? It's hard to believe we can remain on this tilt for so long.

Re: AI is mostly about curve fitting (2018)

#58

Is it feasible to create AI on top of a bunch of other AI? Current AIs seem more like specific, fairly simple, brain regions. Maybe we need a level up.

This is kinda just what libraries like pytorch and tensorflow make easy. Architecting neural networks at a higher level and gluing them together.

I guess what I mean is at a much larger scale. Like 1000s of networks or maybe more, like I presume you get in the brain. Is this already what’s happening currently?

Re: AI is mostly about curve fitting (2018)

#59
> Our machines are still incapable of independently coming up with a thought or hypothesis, testing it against others and accepting or rejecting its validity based on reasoning and experimentation, i.e. following the core principles of the scientific method.

https://www.scientificamerican.com/article/robots-adam-and-e...

A prime example of this is "Adam," an autonomous mini laboratory that uses computers, robotics and lab equipment to conduct scientific experiments, automatically generate hypotheses to explain the resulting data, test these hypotheses, and then interpret the results.

Re: AI is mostly about curve fitting (2018)

#60

I'm pretty sure that I'm just a curve trying to fit what it means to survive as a human.

The difference is that you have the metacognition. You can think about your fitting. It’s possible to say that such a process is just another set of parameters, but they impact retro actively. At that point the metaphor of curve fitting might break down.
Post reply on HN