Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Karl Friston: a neuroscientist who might hold the key to true AI

wired.com

Post reply on HN
Print view View original

41–50 of 111 posts

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#41

Post by atschantz »

snrji wrote: post #10

By no means will I be ever able to grasp Friston's theory, but the free energy minimisation vaguely reminds me of Curiosity-driven reinforcement learning. Can anyone with more understanding than me confirm or deny this apparent resemblance?

There is a large overlap, for instance, the popular VIME exploration algorithm [1] uses part of the free energy objective function.

However, free energy isn't a theory of curiosity per se, its posed as description of self-organisation. It just so happens that you can express the free energy functional in terms of epistemic (curious) and instrumental (reward) components.

[1] https://arxiv.org/abs/1605.09674

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#42

Post by atschantz »

mafribe wrote: post #6

Clickbait article. It is noteworthy that Friston has, as of November 2018, neither (1) formalised free energy minimisation (FIM) with sufficient precision that it goes beyond a vague research heuristic, that can (and is) adapted in ad-hoc ways; nor (2) come up with sufficient empirical evidence for his claim that FIM is how human or animal brains works -- despite the recent revolution in our ability to measure live n…

Well, a significant portion of empirical neuroscience works under the assumption that parts of the brain operate according to a predictive coding scheme, and there are countless studies that support this notion.

As predictive coding is a form of free energy minimization (under Gaussian assumptions), this implicitly provides empirical evidence.

As for the request to test the idea on live neurons, "In vitro neural networks minimise variational free energy" [1]

https://www.biorxiv.org/content/early/2018/05/16/323550

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#43

Post by atschantz »

sydd wrote: post #19
jamii wrote: post #7

Notes from the last time I tried to understand this - https://www.lesswrong.com/posts/wpZJvgQ4HvJE2bysy/god-help-u...

Thanks, it was a great read! From what I get this whole thing is more like an abstract ruleset describing how decision making in the brain works, rather than a brain model. Or am I wrong, is there anyone who built a network model based on this theory?

In terms of the free energy 'principle', it makes no predictions about how free energy minimized. But there have been multiple process theories suggested, most notably predictive coding (which is a dominant paradigm in neuroscience) [1] and variational message passing [2].

[1] https://en.wikipedia.org/wiki/Predictive_coding [2] http://www.jmlr.org/papers/volume6/winn05a/winn05a.pdf

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#44

Post by atschantz »

conjectures wrote: post #29

Anyone have a reference relating the free energy minimisation principle / active inference to reinforcement learning type environments?

The particular study cited in the article is [1], however for a more general review of the links to reinforcement learning [2].

[1] https://www.biologicalpsychiatrycnni.org/article/S2451-9022(... [2] https://journals.plos.org/plosone/article?id=10.1371/journal...

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#45

Post by SubiculumCode »

temperfidelis2x wrote: post #14
kken wrote: post #4

The article is too long for the idea it tries to convey. I like to read to broaden my mind, not for readings sake.

The article tells a story; it's not meant for people trying to grasp technical details. Furthermore, it can be argued that reading (well-written texts like this) for readings sake also broadens your mind.

At least provide a summary. I mean, "After completing his medical studies, Friston moved to Oxford and spent two years as a resident trainee at a Victorian-era hospital called Littlemore. Founded under the 1845 Lunacy Act, Littlemore had originally been instituted to help transfer all “pauper lunatics” from workhouses to hospitals. By the mid-1980s, when Friston arrived, it was one of the last of the old asylums on the outskirts of England’s cities."

is a story.

But as a neuroscientist with an interest in machine learning, I want to know the idea, not the history of Littlemore, attended by this scientist whose tools and methods I have used(Friston motion parameters, I am looking at you).

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#46

Post by atschantz »

It's worth noting that 'free energy' is just the 'evidence lower bound' that is optimized by a large portion of today's machine learning algorithms (i.e. variational auto-encoders).

It's also worth noting that 'predictive coding' - a dominant paradigm in neuroscience - is a form of free energy minimization.

Moreover, free energy minimization (as predictive coding) approximates the backpropagation algorithm [1], but in a biologically plausible fashion. In fact, most biologically plausible deep learning approaches use some form of prediction error signal, and are therefore functionally akin to predictive coding.

Which is all just to say that the notion of free energy minimization is somewhat commonplace in both neuroscience and machine learning.

[1] https://www.ncbi.nlm.nih.gov/pubmed/28333583

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#47

Post by paraschopra »

For anyone who is interested in a tutorial and actual implementation of active inference (an idea based on Free Energy Principle), here's one in Python https://kaiu.me/2017/07/11/introducing-the-deep-active-infer...

I have been trying to understand FEP, and so far my understanding is that essentially the agent tries to learn the generative model that most closely explains observations and then tries to act in a way that are more likely to cause the environment to generate its preferred observations (say pH and temperature in the right range).

The problem with this approach is in scalability of inference and candidate model generation. By the time you provide model for the agent, you as a designer have coded much of your knowledge already and hence constrain the agent. True AI will build model from the scratch, and not just learn model complexity.

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#48

Post by eli_gottlieb »

paraschopra wrote: post #47

For anyone who is interested in a tutorial and actual implementation of active inference (an idea based on Free Energy Principle), here's one in Python https://kaiu.me/2017/07/11/introducing-the-deep-active-infer... I have been trying to understand FEP, and so far my understanding is that essentially the agent tries to learn the generative model that most closely explains observations and then tries to act in a way t…

>True AI will build model from the scratch, and not just learn model complexity.

There's no such thing as truly learning "from scratch" -- the No Free Lunch Theorem holds no matter what. What you can do is find a sufficiently large (ex: Turing-complete) hypothesis class, and make simplifying assumptions to allow it to be feasibly learnable (such as regularization or priors).

Re: Karl Friston: a neuroscientist who might hold the key to true AI

#50

Post by paraschopra »

snrji wrote: post #10

By no means will I be ever able to grasp Friston's theory, but the free energy minimisation vaguely reminds me of Curiosity-driven reinforcement learning. Can anyone with more understanding than me confirm or deny this apparent resemblance?

There are similarities. The difference in two approaches are: - FEP is Bayesian in nature, while there's usually no notion of uncertainties in curiosity driven RL - In FEP, there's no explicit weighting of explore/exploit tradeoff. It automatically emerges from equations - FEP, since it's Bayesian, allows for more complex reasoning (like counterfactuals) - Curiosity driven RL is scalable while FEP is not feasible for anything other than simple models
Post reply on HN