Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tutorial on the free-energy framework for modelling perception and learning

tmorville.github.io

Post reply on HN
Print view View original

11–20 of 22 posts

Re: A tutorial on the free-energy framework for modelling perception and learning

#11

Post by yorwba »

tomjakubowski wrote: post #10

> The non-linear function that relates size v to photosensory input u is assumed to be g(v)=v^2 I am having a hard time understanding this sentence - how does g(v) = v^2 relate the size v and the input u if the expression mentions only v? Is it meant to be v = g(u) = u^2? Is it u = g(v) = v^2?

"We assume that this signal [i.e. u] is normally distributed with mean g(v) and variance Σ_v."

That means that there's no deterministic function that fixes u for a given v, but only a distribution of possible values. (It's closer to u = v^2 than the opposite, though.) The precise relationship is expressed symbolically in the likelihood function given in the next part.

Re: A tutorial on the free-energy framework for modelling perception and learning

#12

Post by tomjakubowski »

yorwba wrote: post #11
tomjakubowski wrote: post #10

> The non-linear function that relates size v to photosensory input u is assumed to be g(v)=v^2 I am having a hard time understanding this sentence - how does g(v) = v^2 relate the size v and the input u if the expression mentions only v? Is it meant to be v = g(u) = u^2? Is it u = g(v) = v^2?

"We assume that this signal [i.e. u ] is normally distributed with mean g(v) and variance Σ_v ." That means that there's no deterministic function that fixes u for a given v , but only a distribution of possible values. (It's closer to u = v^2 than the opposite, though.) The precise relationship is expressed symbolically in the likelihood function given in the next part.

Thank you!

Re: A tutorial on the free-energy framework for modelling perception and learning

#13

Post by eli_gottlieb »

Disclaimer: I am not the author of the page. I've read the tutorial paper they refer to, and work with related material. In our lab, we've had a very hard time finding accessible material on predictive coding that's actually amenable to equations and code, and so this is something of a godsend.

Re: A tutorial on the free-energy framework for modelling perception and learning

#14

Post by [deleted] »

boltzmannbrain wrote: post #5

For the probabilistic graphical model and belief prop perspective, check out Friston et al. (2017) "The graphical brain: Belief propagation and active inference": https://www.mitpressjournals.org/doi/pdf/10.1162/netn_a_0001... For the neural corollaries of predictive coding, check out Shipp (2016) "Neural Elements for Predictive Coding": https://www.frontiersin.org/articles/10.3389/fpsyg.2016.0179... For a state-of-a…

[deleted]

Re: A tutorial on the free-energy framework for modelling perception and learning

#16

Post by rm2040 »

boltzmannbrain wrote: post #5

For the probabilistic graphical model and belief prop perspective, check out Friston et al. (2017) "The graphical brain: Belief propagation and active inference": https://www.mitpressjournals.org/doi/pdf/10.1162/netn_a_0001... For the neural corollaries of predictive coding, check out Shipp (2016) "Neural Elements for Predictive Coding": https://www.frontiersin.org/articles/10.3389/fpsyg.2016.0179... For a state-of-a…

How does RCN model uncertainty?

Re: A tutorial on the free-energy framework for modelling perception and learning

#17

Post by rm2040 »

elcritch wrote: post #3

Anybody familiar with this method? It looks very intriguing as it looks to be combinging Bayesian posteriors with the neural activation function. Haven’t had time to dig into myself, and it would be nice to know an outside perspective on it.

A few years back I spent some time reading and following the equations in Friston's papers, maybe understanding like 90% of it. you have to know dynamical systems, differential equations, multivariate matrix stuff, etc. Basically stuff physicists are good at. Seemed that the theory wasn't detailed enough to inspire the next deep NN revoluation, basically the form of the generative model Friston used is very general. My impression was that the coded matlab examples reqired you specify known quantities, like velocity or position. But that requires a human to input, not like a NN where you can just point it at some data and it learns. Would love to be shown otherwise...

Re: A tutorial on the free-energy framework for modelling perception and learning

#18

Post by boltzmannbrain »

rm2040 wrote: post #16
boltzmannbrain wrote: post #5

For the probabilistic graphical model and belief prop perspective, check out Friston et al. (2017) "The graphical brain: Belief propagation and active inference": https://www.mitpressjournals.org/doi/pdf/10.1162/netn_a_0001... For the neural corollaries of predictive coding, check out Shipp (2016) "Neural Elements for Predictive Coding": https://www.frontiersin.org/articles/10.3389/fpsyg.2016.0179... For a state-of-a…

How does RCN model uncertainty?

Max-product BP inference propagates local uncertainties in the model to arrive at a globally coherent solution. An example of where this is particularly useful is resolving border-ownership amongst neurons representing a common contour.

Re: A tutorial on the free-energy framework for modelling perception and learning

#20

Post by eli_gottlieb »

rm2040 wrote: post #17
elcritch wrote: post #3

Anybody familiar with this method? It looks very intriguing as it looks to be combinging Bayesian posteriors with the neural activation function. Haven’t had time to dig into myself, and it would be nice to know an outside perspective on it.

A few years back I spent some time reading and following the equations in Friston's papers, maybe understanding like 90% of it. you have to know dynamical systems, differential equations, multivariate matrix stuff, etc. Basically stuff physicists are good at. Seemed that the theory wasn't detailed enough to inspire the next deep NN revoluation, basically the form of the generative model Friston used is very general.…

I'd be interested to hear any advice you might have on understanding the papers. Particularly, how do you understand the "priors" that specify goals/preferences/set-points in active inference?
Post reply on HN