Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is mostly about curve fitting (2018)

diginomica.com

Post reply on HN
Print view View original

81–90 of 192 posts

Re: AI is mostly about curve fitting (2018)

#81

Post by youdontknowtho »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

My exposure to these subjects has been limited, but the thing that I liked about Dan Dennets rebuttal to Chalmers is that it does away with the special-ness of consciousness.

It is more likely that consciousness is just a property of brains inside bodies. The interesting question to me is the level of complexity of brain and body required to produce something like what we experience and what is it like in other arrangements of brains and bodies.

I also don't know that intelligence is the great thing that we think it is. It's an adaptation. It's an adaptation that lots of other organisms survive just fine without.

Re: AI is mostly about curve fitting (2018)

#82

Post by youdontknowtho »

missosoup wrote: post #76
yk wrote: post #41

Earlier quoted context omitted.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

There's no need for a supernatural explanation of consciousness. We are all p-zombies. Problem solved. The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in someth…

I couldn't agree more. It also just seems like a continuation of the old story in science where the things that we think are special or magical just aren't.

Re: AI is mostly about curve fitting (2018)

#83

Post by scottlegrand2 »

When I was working on recommenders at Amazon. I didn't really perceive this as modeling the mind of some sort of Oracle.

Instead I saw it as a change of basis from the space of previous purchases to the space of recent purchases. That approach led to an entire AI framework and it still makes Amazon quite a bit of money annually apparently if Jeff Wilke's 2019 Re:MARS speech is to be believed.

I'd love to work on something more ambitious like AlphaGo or AlphaFold but those require tremendous resources and I'm really focused on bang for buck. But even then I'd see it as the marriage of classical search with modeling the probability of victory.

If someone says that the AGI is almost upon us I pretty much bozo bit them no matter how prestigious or fancy they may be otherwise.

Re: AI is mostly about curve fitting (2018)

#84

Post by wwarner »

Some really insightful discussion of the possibilities and limits of ML can be found on Les Fridman's AI podcast. Especially good were interviews with Yann LeCunn, Jeff Hawkins, Elon Musk and Francois Chollet. One memorable quote from LeCun "There is no intelligence without learning." Here's a link to the LeCun interview, though many aothers are excellent. https://lexfridman.com/yann-lecun/

Re: AI is mostly about curve fitting (2018)

#85

Post by degyves »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

That depends of which precise definition of consciouness you're talking about, which is a flame wars research field. But it is a consensus that consciouness is not equal to inteligence.

Re: AI is mostly about curve fitting (2018)

#86

Post by scottlegrand2 »

umarniz wrote: post #38

I think intelligence has for some time already been boiled down to curve fitting, even for humans. Our current accepted definition of intelligence in schools is to get a score that is higher than the average to be considered sufficiently intelligent to proceed to the next grade. I feel anything that we develop for AI would fundamentally always be inspired by our own experiences and hence curve fitting is something we…

We certainly tend to fall into scripts and follow those scripts for many things in life. GPT-2 demonstrated how much online prose is glorified mad libbing IMO and yet it has no sense of a consistent long-term context nor do its articles make any sort of significant point. Reading its output is impressive but it reminds me of what I hear from someone with dementia.

that said I think if you made its context recurrent in some way between responses and queries you could probably build a really interesting chatbot that could ramble about just about anything and nothing but do so in a sufficiently coherent way to scare up venture capital. Just a guess but it is my guess.

Re: AI is mostly about curve fitting (2018)

#87

Post by manifestsilence »

Veedrac wrote: post #65

The article is less awful than the title. In short, the thesis is that ML seems only able to learn associations, rather than stronger, causal models. It should be fairly obvious that ‘curve fitting’ is a misleading category—these models are clearly learning highly meaningful latent spaces that no prior approaches ever did. But I would agree that the actual high-level ability to make causal inferences seems to be lack…

This is interesting, as this appears to be similar to the difference between Daniel Kahneman's System1 and System2 modes of human thought. ML is perhaps beginning to approach our subconscious, associative intelligence. This shows up in its excellence for things like image processing, which we do instantly and automatically. Perhaps growing the tech equivalent of a prefrontal cortex is what's going to be hard, or require a different approach.

Re: AI is mostly about curve fitting (2018)

#88

Post by red75prime »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

> hen you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness.

Why not? Those things have zero-consciousness that is conscious only of itself and which correctly reflects their lack of self-model.

Re: AI is mostly about curve fitting (2018)

#89

Post by sgt101 »

jonplackett wrote: post #58
Lambdanaut wrote: post #39

Earlier quoted context omitted.

This is kinda just what libraries like pytorch and tensorflow make easy. Architecting neural networks at a higher level and gluing them together.

I guess what I mean is at a much larger scale. Like 1000s of networks or maybe more, like I presume you get in the brain. Is this already what’s happening currently?

Well - it's what Minsky advocated... https://en.wikipedia.org/wiki/Society_of_Mind

But, I think that its a bit "hit and hope" - as an idea it doesn't really address any questions unless/until it is realised and works.

Re: AI is mostly about curve fitting (2018)

#90

Post by lordnacho »

umarniz wrote: post #38

I think intelligence has for some time already been boiled down to curve fitting, even for humans. Our current accepted definition of intelligence in schools is to get a score that is higher than the average to be considered sufficiently intelligent to proceed to the next grade. I feel anything that we develop for AI would fundamentally always be inspired by our own experiences and hence curve fitting is something we…

> I think intelligence has for some time already been boiled down to curve fitting, even for humans.

No, the book actually makes an important point about this.

For instance, let's take a counterfactual. How do you know what would have happened if Barcelona had played Lionel Messi in goal over the latest season?

They've never done this, there are no data points for you to fit. The situation almost never arises that an outfield player plays in goal, and when they do it is always in a situation where someone's been sent off or injured, which is also rare.

And yet you and I and everyone else who can think knows this would result in Barcelona having a much worse season.

Just to be clear, it's not only because you'd be taking a top player out of offense where he is worth a lot, which you can surely fit some curves to show.

We can all guess that he'll be a worse keeper than Ter Stegen, but what curve would you fit that shows this? There's no data about Messi in goal.

Pearl does give a way to work it out via counterfactual analysis though.

Post reply on HN