Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linear logic and deep learning [pdf]

therisingsea.org

Post reply on HN
Print view View original

11–20 of 49 posts

Re: Linear logic and deep learning [pdf]

#11

Post by yaseer »

yaseer wrote: post #10
dmurfet wrote: post #9

Author here. The theoretical background can be found in: https://arxiv.org/abs/1407.2650 https://arxiv.org/abs/1701.01285 http://therisingsea.org/notes/MScThesisJamesClift.pdf As neel_k notes, a good way to understand this picture is in terms of differential linear logic (a refinement of simply-typed differential lambda calculus). I did not provide references in the talk as unfortunately I did not understand the subj…

Fantastic stuff! Does your work still keep the non-determinism of Ehrhard and Regnier? This was the part that 'bothered' me, but it provided evidence of a link with process calculi. I've also thought Ehrhard and Regnier's work was groundbreaking. It potentially opens up not just insights into differentiable programming, but a theory of concurrent computation and an algebraic theory of computation. My expectation is t…

P.S - Silly question, I can see it does. I responded before reading.

Re: Linear logic and deep learning [pdf]

#12

Post by dmurfet »

yaseer wrote: post #10
dmurfet wrote: post #9

Author here. The theoretical background can be found in: https://arxiv.org/abs/1407.2650 https://arxiv.org/abs/1701.01285 http://therisingsea.org/notes/MScThesisJamesClift.pdf As neel_k notes, a good way to understand this picture is in terms of differential linear logic (a refinement of simply-typed differential lambda calculus). I did not provide references in the talk as unfortunately I did not understand the subj…

Fantastic stuff! Does your work still keep the non-determinism of Ehrhard and Regnier? This was the part that 'bothered' me, but it provided evidence of a link with process calculi. I've also thought Ehrhard and Regnier's work was groundbreaking. It potentially opens up not just insights into differentiable programming, but a theory of concurrent computation and an algebraic theory of computation. My expectation is t…

I think you are right, the meaning of the sums that appear in syntactic derivatives is quite subtle, and I don’t claim to have an authoritative answer as to their deep meaning.

Semantically, however, I think things are clearer. The denotation of the syntactic derivative of a proof is a limit, and the terms in the limit have an interpretation as probabilistic processes in a (close to) standard sense. So while the sum-over-linear-substitutions doesn’t have a clear probabilistic interpretation (at least that I can see) it is the limit of computational processes where you run the probability of making a substitution to zero.

I don’t know anything about process calculi, unfortunately!

Re: Linear logic and deep learning [pdf]

#13

Post by regularfry »

tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

Often the trick is not to make the new thing, but to make the old thing comprehensible so someone else can pick it up and run with it.

Re: Linear logic and deep learning [pdf]

#14

Post by regularfry »

thomasahle wrote: post #5
YeGoblynQueenne wrote: post #2

>> So I’d like to begin by summarising some of the recent history in the field of artificial intelligence, or machine learning as its now called. To be more precise, the field is still known as AI, but people outside the field only know (and care) about machine learning, presumably because that's what Googe, Facebook, et al are recruiting for. This is a bit of a sad situation, really. AI, in its drive to solve major…

> Machine learning is just one such technique which has gained popularity outside AI. I'd argue the machine learning label can be applied to any AI system that's data-driven in some sense (even self-generating the data using reinforcement learning). Wikipedia lists the following approaches to Machine Learning. Surely you wouldn't call them all 'one technique'?: Decision tree learning, Association rule learning, Artif…

AI used to cover Expert Systems, which have nothing we'd recognise as a "learning" phase at all. Hell, it used to cover parsers.

Re: Linear logic and deep learning [pdf]

#15

Post by tw1010 »

regularfry wrote: post #13
tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

Often the trick is not to make the new thing, but to make the old thing comprehensible so someone else can pick it up and run with it.

I agree. The real low hanging fruit is not new research, but to make the easy, basic first or second year graduate math understandable in the context of ML.

Re: Linear logic and deep learning [pdf]

#16

Post by YeGoblynQueenne »

thomasahle wrote: post #5
YeGoblynQueenne wrote: post #2

>> So I’d like to begin by summarising some of the recent history in the field of artificial intelligence, or machine learning as its now called. To be more precise, the field is still known as AI, but people outside the field only know (and care) about machine learning, presumably because that's what Googe, Facebook, et al are recruiting for. This is a bit of a sad situation, really. AI, in its drive to solve major…

> Machine learning is just one such technique which has gained popularity outside AI. I'd argue the machine learning label can be applied to any AI system that's data-driven in some sense (even self-generating the data using reinforcement learning). Wikipedia lists the following approaches to Machine Learning. Surely you wouldn't call them all 'one technique'?: Decision tree learning, Association rule learning, Artif…

Let's say that it's a class of techniques that learn models (or progams, in the case of ILP) from data.

My main concern is not about defining what machine learning is, however. Rather, I'm worried about the definition of AI shrinking to "it's what we now call machine learning". Which is at the very least unhistorical.

Re: Linear logic and deep learning [pdf]

#17

Post by lou1306 »

tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

On a similar note, taking a look at a mathematician's coursebooks made me realize how puny our mathematical tools are. Most Computer Science topics only rely on (relatively) basic concepts from logic, algebra, calculus, and/or probability. The hardest course I ever attended was a quantum computing one, and even that does not require much, apart from a generalization of probability theory to complex numbers.

I bet there are lots of brilliant results that a smart mind could apply to our field.

Re: Linear logic and deep learning [pdf]

#18

Post by senatorobama »

tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

What are some good results coming out from applied mathematics?

Re: Linear logic and deep learning [pdf]

#19

Post by alexilliamson »

regularfry wrote: post #13
tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

Often the trick is not to make the new thing, but to make the old thing comprehensible so someone else can pick it up and run with it.

Or applying the old thing in a new way to a different audience, like Einstein using Lorentzian manifolds.
Post reply on HN