Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Isochronous Curves

en.wikipedia.org

Post reply on HN
Print view View original

31–40 of 65 posts

Re: Isochronous Curves

#31

Post by ur-whale »

mlthoughts2018 wrote: post #24
ur-whale wrote: post #2

Variation calculus is a great lens through which to look at many physics and CS problems. It is IMO under-taught and under-utilized, especially in the machine learning discipline. The framework is basically minimization in functional space (as opposed to R^n or a subset thereof, the more common case). The (to me) surprising thing is that finding an extremum in functional space (i.e. infinite-dimensional space) can be…

I studied computer vision in grad school when these were some of the state of the art methods, like the Mumford-Shah segmentation model and various inpainting models that set up the problem basically as a calculus of variations problem over the appropriate Sobolev space of image intensity functions and/or discontinuity-permitting boundary functions. I’ve worked as a practitioner in computer vision as well and I think…

>In practice, the variational methods are not competitive at all with deep learning models

I really don't see why these two things need be pitted against each others.

What you're describing (Euler-lagrange applied to computer vision problems) is a direct application of the calculus of variation to the image itself.

It's rather naive because it uses an extremely weak representation of the content of the image (i.e. a bunch of pixels which essentially means working with a piecewise-constant basis of the functional space), and as you noted, it is easily beaten by modern techniques.

However: Deep neural nets are nothing but another, much more powerful "basis" of functional space (I put "basis" in quote because they're not linear. OTOH, they're generative).

A deep net is a finite set of real numbers that maps to a point in functional space, and any arbitrary continuous maps from R^n to R^p can be approximated by deep nets given enough weights and neurons.

So, if deep nets are dense in functional space, and "training" a deep net is nothing but solving a minimization problem in functional space, then I don't see why the "calculus of variation" ==> "PDE" via Euler-Lagrange route can't bee applied to training deep nets instead of backprop-based minimizers.

Now, if you're claiming that backprop works better than solving the PDE induced by Euler-Lagrange on the functional minimization problem of training a deep-net, I'm very happy to believe you, but I've yet to see literature demonstrating this.

Re: Isochronous Curves

#32

Post by escapologybb »

anonu wrote: post #30
escapologybb wrote: post #11

Earlier quoted context omitted.

I think we three are in the same Google Bubble as I've just finished watching that glorious Open University video. If the algorithm wills it, I think it means we have to be best friends! Seriously though, I would be weirdly fascinated to see what else you guys subscribe to and if there are any correlations to my subscriptions. I find this channelling people into little boxes fascinating, and those curves are also ver…

Engineer guy, slow mo guys, motherboard vice, primitive technology, smarter everyday

We have a winner! I'm subscribed to all of those.

Re: Isochronous Curves

#33

Post by ISL »

quietbritishjim wrote: post #5
lifeisstillgood wrote: post #4

Is this off the back of a video of a 1970s Open University style presenter showing the properties of isochromus curves - i loved watching it and realised that these are national treasures of programs - and also was quite stunned by the assumption one would have a round empty tobacco tin lying around to do the experiment with - times do chnage :-)

I suspect it is... It turned up in my YouTube suggestions yesterday. Here's a link: https://youtu.be/eBc827pwKf0

Suggested for me this morning, and again just now.

Re: Isochronous Curves

#34

Post by mlthoughts2018 »

ur-whale wrote: post #31
mlthoughts2018 wrote: post #24

Earlier quoted context omitted.

I studied computer vision in grad school when these were some of the state of the art methods, like the Mumford-Shah segmentation model and various inpainting models that set up the problem basically as a calculus of variations problem over the appropriate Sobolev space of image intensity functions and/or discontinuity-permitting boundary functions. I’ve worked as a practitioner in computer vision as well and I think…

>In practice, the variational methods are not competitive at all with deep learning models I really don't see why these two things need be pitted against each others. What you're describing (Euler-lagrange applied to computer vision problems) is a direct application of the calculus of variation to the image itself. It's rather naive because it uses an extremely weak representation of the content of the image (i.e. a…

The variational representations used in image processing are much more complicated than you suggest. In many cases, they are developed with semantic models that inform grammars of what the image can be composed of, and directly model things like occlusion, cusps, edges, blur, etc., when postulating a joint functional space that can describe the “energy” of an image, which then minimizing gives not only pixel values but also parametric curves for boundaries and sometimes also labels.

It’s essentially a way to encode semantics as prior distributions over possible images, then variational methods “optimize” for the posterior parameters given some data.

I’m trying to say that that whole exercise listed above, conceptually, is basically a failed idea compared with solving the same problems using deep learning to automatically discover what that manifold looks like in parameter space and transform it directly to the output space of practical interest (segmentation boundaries, etc.).

Re: Isochronous Curves

#35

Post by sumnulu »

quietbritishjim wrote: post #5
lifeisstillgood wrote: post #4

Is this off the back of a video of a 1970s Open University style presenter showing the properties of isochromus curves - i loved watching it and realised that these are national treasures of programs - and also was quite stunned by the assumption one would have a round empty tobacco tin lying around to do the experiment with - times do chnage :-)

I suspect it is... It turned up in my YouTube suggestions yesterday. Here's a link: https://youtu.be/eBc827pwKf0

Same here 6 hours ago I watched it.

Re: Isochronous Curves

#36

Post by lerax »

quietbritishjim wrote: post #5
lifeisstillgood wrote: post #4

Is this off the back of a video of a 1970s Open University style presenter showing the properties of isochromus curves - i loved watching it and realised that these are national treasures of programs - and also was quite stunned by the assumption one would have a round empty tobacco tin lying around to do the experiment with - times do chnage :-)

I suspect it is... It turned up in my YouTube suggestions yesterday. Here's a link: https://youtu.be/eBc827pwKf0

I'll add wrote a similar comment but I suspect that someone already saw this video and comment here.

But I am a little surprised by the amount of people that get that from the recommender system too.

Re: Isochronous Curves

#37

Post by ur-whale »

mlthoughts2018 wrote: post #34
ur-whale wrote: post #31

Earlier quoted context omitted.

>In practice, the variational methods are not competitive at all with deep learning models I really don't see why these two things need be pitted against each others. What you're describing (Euler-lagrange applied to computer vision problems) is a direct application of the calculus of variation to the image itself. It's rather naive because it uses an extremely weak representation of the content of the image (i.e. a…

The variational representations used in image processing are much more complicated than you suggest. In many cases, they are developed with semantic models that inform grammars of what the image can be composed of, and directly model things like occlusion, cusps, edges, blur, etc., when postulating a joint functional space that can describe the “energy” of an image, which then minimizing gives not only pixel values b…

>The variational representations used in image processing are much more complicated than you suggest

Fair enough, but my main point stands: has anyone tried to apply Euler-Lagrange and a PDE solver to the problem of training a deep net or is it obvious that backprop-based methods will work better?

Re: Isochronous Curves

#38

Post by bordercases »

quietbritishjim wrote: post #5
lifeisstillgood wrote: post #4

Is this off the back of a video of a 1970s Open University style presenter showing the properties of isochromus curves - i loved watching it and realised that these are national treasures of programs - and also was quite stunned by the assumption one would have a round empty tobacco tin lying around to do the experiment with - times do chnage :-)

I suspect it is... It turned up in my YouTube suggestions yesterday. Here's a link: https://youtu.be/eBc827pwKf0

Ditto

Re: Isochronous Curves

#39

Post by montalbano »

ur-whale wrote: post #2

Variation calculus is a great lens through which to look at many physics and CS problems. It is IMO under-taught and under-utilized, especially in the machine learning discipline. The framework is basically minimization in functional space (as opposed to R^n or a subset thereof, the more common case). The (to me) surprising thing is that finding an extremum in functional space (i.e. infinite-dimensional space) can be…

Feynman on Calculus of variations:

“When I was in high school, my physics teacher—whose name was Mr. Bader—called me down one day after physics class and said, ‘You look bored; I want to tell you something interesting.’ Then he told me something which I found absolutely fascinating, and have, since then, always found fascinating. Every time the subject comes up, I work on it. In fact, when I began to prepare this lecture I found myself making more analyses on the thing. Instead of worrying about the lecture, I got involved in a new problem. The subject is this—the principle of least action."

The rest at: http://www.feynmanlectures.caltech.edu/II_19.html

Re: Isochronous Curves

#40

Post by ak39 »

lerax wrote: post #36
quietbritishjim wrote: post #5

Earlier quoted context omitted.

I suspect it is... It turned up in my YouTube suggestions yesterday. Here's a link: https://youtu.be/eBc827pwKf0

I'll add wrote a similar comment but I suspect that someone already saw this video and comment here. But I am a little surprised by the amount of people that get that from the recommender system too.

I got this video recommended to the point of nuisance. Ended up subscribing after watching the video though.
Post reply on HN