Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is mostly about curve fitting (2018)

diginomica.com

Post reply on HN
Print view View original

141–150 of 192 posts

Re: AI is mostly about curve fitting (2018)

#141

Post by danans »

yk wrote: post #136
red75prime wrote: post #88

Earlier quoted context omitted.

> hen you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness. Why not? Those things have zero-consciousness that is conscious only of itself and which correctly reflects their lack of self-model.

The argument is, if our brains are just a curve fitting machine, then we can dial in the complexity of the computation. Start with a single layer parameter, then two parameter, and so on until we are at the complexity of the brain. By that procedure, we can ask after each parameter, if the machine is now conscious, and I strongly doubt that there is a good answer.

The way we ascribe consciousness to entities other than ourselves is based on similarity to ourselves.

Obviously other living humans have the highest similarity, so they are automatically deemed conscious. Next are other primates, followed by other domesticated mammals, and other animals.

Furthest from the status of conscious are creatures we see as automata like dung beetles rolling their food, or jellyfish ... jellyfishing.

Presumably we'd apply a similar process to hypothetical AGIs.

Re: AI is mostly about curve fitting (2018)

#142

Post by bermanoid »

throwaway713 wrote: post #95
missosoup wrote: post #76

Earlier quoted context omitted.

There's no need for a supernatural explanation of consciousness. We are all p-zombies. Problem solved. The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in someth…

You make a strong claim without anything to back it up. I don’t understand why it’s become trendy to deny that consciousness exists; to me it’s isomorphic to saying “We don’t actually exist at all. Prove me wrong.” It’s a vacuous statement, meant to sound evocative, but difficult to respond to in any meaningful way. If I made a list of everything in order of how certain I am that the item on the list exists, consciou…

Claiming that some mysterious and hard to define property that we can't measure even in principle "exists" in some meaningful way strikes me as the stronger claim than the skeptical take does.

Why do you think the burden of proof should be inverted? The mere fact that most humans intuitively feel "something" doesn't count for much of anything, especially once you stipulate that p-zombies would vote the same way.

Re: AI is mostly about curve fitting (2018)

#143

Post by quikoa »

robotresearcher wrote: post #122
yters wrote: post #96

What if human intelligence is not computable? Why does no researcher address the fundamental assumption of the field?

This has been addressed by many authors since modern computation was formulated, for example by Turing himself. We learn about and discuss Newell and Simon's Physical Symbol System hypothesis in undergrad classes, ie. explicitly stating the underlying assumption. Once in a while someone will assert that computation is not sufficient, eg. Penrose, and generate discussion. In practice you can work on AI-the-engineering…

It could be possible that it's computable from a theoretic perspective but not a practical one. After all our biological brains are quite different from our electrical computer chips.

Re: AI is mostly about curve fitting (2018)

#144

Post by Symmetry »

manifestsilence wrote: post #87
Veedrac wrote: post #65

The article is less awful than the title. In short, the thesis is that ML seems only able to learn associations, rather than stronger, causal models. It should be fairly obvious that ‘curve fitting’ is a misleading category—these models are clearly learning highly meaningful latent spaces that no prior approaches ever did. But I would agree that the actual high-level ability to make causal inferences seems to be lack…

This is interesting, as this appears to be similar to the difference between Daniel Kahneman's System1 and System2 modes of human thought. ML is perhaps beginning to approach our subconscious, associative intelligence. This shows up in its excellence for things like image processing, which we do instantly and automatically. Perhaps growing the tech equivalent of a prefrontal cortex is what's going to be hard, or requ…

Yeah, that seems to be part of the reason that AlphaZero, which has its System2-like alpha-beta search seems so much smarter than AlphaStar which plays like a brilliant somnambulist.

Maybe if we had some way of abstracting out the things a machine learning system implicitly learns so we could deal with them in a more classical AI-like way?

Re: AI is mostly about curve fitting (2018)

#145

Post by zby »

galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

What our brains do is curve fitting plus experiments. Causality is learned through experiments - and as children we do a lot of experiments (with moving our limbs etc). With just observations you can have only correlations. For example you can correlate smoke with fire, but only through experiment you can learn that it is fire that causes smoke not the other way around.

Re: AI is mostly about curve fitting (2018)

#146

Post by eli_gottlieb »

EsssM7QVMehFPAs wrote: post #100
nextos wrote: post #71

Earlier quoted context omitted.

I think curve fitting is an important component of future AGI. But it definitely needs causal reasoning baked in, which leads to better models with less data [1,2]. My intuition is that there's a lot of important work to be done using logical representations of models and transforming them back and forth using well understood semantics operators. Deep functions will be part of said models, but the whole model does no…

Casual reasoning is one thing that is lacking. But what about creativity? What about drive and desire? What about belief and the will to fail on the road to success? What about collective intelligence and the need to peer up in efforts? What about emotional intelligence? I personally do not believe in AGI since I also do not believe in psychology, sociology or neurobiology being anywhere near understanding the holist…

> Causal reasoning is one thing that is lacking. But what about creativity? What about drive and desire? What about belief and the will to fail on the road to success? What about collective intelligence and the need to peer up in efforts? What about emotional intelligence?

Hi, I work at one of the intersections of machine learning with certain schools of thought in neuroscience. The following is based entirely on my own understanding, but is at least based on an understanding.

Your list here really only has three problems in it: causal reasoning, theory of mind, and "emotional intelligence". Emotional intelligence works in the service of "drive and desire", considered broadly. Creativity likewise works for the emotions. To be creative, you need aesthetic criteria.

Most of that, we're still really working on putting into mathematical and computational terms.

Re: AI is mostly about curve fitting (2018)

#147

Post by darkmighty »

ur-whale wrote: post #6
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

First, no AI that I know of has at its disposal a full blown model of the world it operates in, whereas most human brains do, and even if the model is imperfect, it is capable of producing fairly accurate simulations (what-if scenarios). Second, deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting (that is, within the confines of the model)…

> no AI that I know of has at its disposal a full blown model of the world it operates in

This is a field called Model-based Reinforcement learning, and it's quite advanced already -- there are indeed models that have an internal state reflecting the world state.

A good recent example:

https://papers.nips.cc/paper/7512-recurrent-world-models-fac...

> deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting

This is also addressed by recent models, somewhat. Once you have an abstract world model, searching for a high reward can be just a matter of running markovian simulation on it using high reward heuristics (given by a network of course), like AG does. This line is also very active right now, one example is the recent MuZero.

https://arxiv.org/abs/1911.08265

Inference at its core really isn't much more than an artful curve fitting (or an artful model search if you like), and it's one of the building blocks of intelligence.

Re: AI is mostly about curve fitting (2018)

#148

Post by tgv »

cs702 wrote: post #103

This article conflates two separate, very different issues into one: * Issue #1: There is a tremendous amount of hype, noise, and snake oil surrounding the moniker "AI." Pretty much everyone agrees with this statement. (And anyone who doesn't agree with it... is probably selling snake oil.) * Issue #2: Is intelligence just a form of "curve fitting," i.e., is it just finding solutions to very complicated, high-dimensi…

> Others believe the answer to #2 is yes.

Others believe the success of our program to teaching pigs to fly will greatly improve as we build taller towers.

Re: AI is mostly about curve fitting (2018)

#149

Post by visarga »

yk wrote: post #41
galangalalgol wrote: post #2

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you ca…

> it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card.

I don't like very much this reverent way of thinking about consciousness, as if it is from another world, or a different essence.

I believe consciousness is the ability of the agent to adapt to the environment in order to protect itself and maximise rewards. It's not just in the matrix multiplication, but in the embodiment, the environment-agent loop. Consciousness is not something that transcends the world and matrices, it's just a power to adapt and survive.

And it feels like something because that feeling has a survival utility, so the agent has a whole neural network to model future possible rewards and actions, which impacts behaviour and outcomes.

Re: AI is mostly about curve fitting (2018)

#150

Post by davidhowlett »

Symmetry wrote: post #144
manifestsilence wrote: post #87

Earlier quoted context omitted.

This is interesting, as this appears to be similar to the difference between Daniel Kahneman's System1 and System2 modes of human thought. ML is perhaps beginning to approach our subconscious, associative intelligence. This shows up in its excellence for things like image processing, which we do instantly and automatically. Perhaps growing the tech equivalent of a prefrontal cortex is what's going to be hard, or requ…

Yeah, that seems to be part of the reason that AlphaZero, which has its System2-like alpha-beta search seems so much smarter than AlphaStar which plays like a brilliant somnambulist. Maybe if we had some way of abstracting out the things a machine learning system implicitly learns so we could deal with them in a more classical AI-like way?

Instead of an alpha-beta search, AlphaZero uses a general-purpose Monte Carlo tree search (MCTS) algorithm. Source: https://science.sciencemag.org/content/362/6419/1140.full

I agree with your comment about System 2 like reasoning not being common right now. I am not an expert in the field but the closest thing I have seen to learned planning is: https://arxiv.org/pdf/1911.08265.pdf

Post reply on HN