Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linear logic and deep learning [pdf]

therisingsea.org

Post reply on HN
Print view View original

21–30 of 49 posts

Re: Linear logic and deep learning [pdf]

#21

Post by kruhft »

kruhft wrote: post #20

"Lively Linear Lisp--'Look Ma, No Garbage!'". ACM Sigplan Notices 27, 8 (August 1992),89-98. How to implement a 'linear' language efficiently through 'behind the scenes' hash consing. http://home.pipeline.com/~hbaker1/LinearLisp.html

From the abstract:

Linear logic has been proposed as one solution to the problem of garbage collection and providing efficient "update-in-place" capabilities within a more functional language. Linear logic conserves accessibility, and hence provides a mechanical metaphor which is more appropriate for a distributed-memory parallel processor in which copying is explicit. However, linear logic's lack of sharing may introduce significant inefficiencies of its own.

We show an efficient implementation of linear logic called Linear Lisp that runs within a constant factor of non-linear logic. This Linear Lisp allows RPLACX operations, and manages storage as safely as a non-linear Lisp, but does not need a garbage collector. Since it offers assignments but no sharing, it occupies a twilight zone between functional languages and imperative languages. Our Linear Lisp Machine offers many of the same capabilities as combinator/graph reduction machines, but without their copying and garbage collection problems.

Re: Linear logic and deep learning [pdf]

#22

Post by LunaSea »

tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

So according to you, why hasn't it happened yet? Not trying to flame, I'm genuinely curious.

I would think that the economic incentive to do exactly this would be pretty high.

Re: Linear logic and deep learning [pdf]

#23

Post by qcoh »

dmurfet wrote: post #9

Author here. The theoretical background can be found in: https://arxiv.org/abs/1407.2650 https://arxiv.org/abs/1701.01285 http://therisingsea.org/notes/MScThesisJamesClift.pdf As neel_k notes, a good way to understand this picture is in terms of differential linear logic (a refinement of simply-typed differential lambda calculus). I did not provide references in the talk as unfortunately I did not understand the subj…

Sorry for the unrelated comment but I wanted to thank you for your nice lecture notes. I found them clear and very helpful during my studies.

Re: Linear logic and deep learning [pdf]

#25

Post by osense »

I don't understand why the type of bint was chosen, and what impact, exactly, does it have on the usability of the network.

Or, asked differently: Given some task that you want to train your network for, how do you decide which type you should choose?

Re: Linear logic and deep learning [pdf]

#26

Post by danharaj »

osense wrote: post #25

I don't understand why the type of bint was chosen, and what impact, exactly, does it have on the usability of the network. Or, asked differently: Given some task that you want to train your network for, how do you decide which type you should choose?

The type bint is an encoding of binary sequences, so it's simple enough as a working example but it's nontrivial enough that you could construe any sort of learning algorithm as operating on it.

The type you choose is part of the model you're training so your question is as difficult as "what kind of network topology should I use?".

Re: Linear logic and deep learning [pdf]

#27

Post by reader5000 »

lou1306 wrote: post #17
tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

On a similar note, taking a look at a mathematician's coursebooks made me realize how puny our mathematical tools are. Most Computer Science topics only rely on (relatively) basic concepts from logic, algebra, calculus, and/or probability. The hardest course I ever attended was a quantum computing one, and even that does not require much, apart from a generalization of probability theory to complex numbers. I bet the…

I think the human brain is capable of only grasping so much abstraction. I'm sure every genius working in the deepest areas of advanced mathematics has heard of deep learning. If there was some amazing concept from their field directly applicable to deep learning we would have heard about it by now.

Re: Linear logic and deep learning [pdf]

#28

Post by danharaj »

neel_k wrote: post #4
noelwelsh wrote: post #3

I haven't had time to get into the details of this, but I'm very interested in differentiable programming that doesn't repeat the mistakes of imperative programming. I'd like to understand how the model in this work differs from, e.g. http://www.bcl.hamilton.ie/~barak/papers/sound-efficient-ad2...

At a super high level, the central problem is higher-order functions. If you have a programming language, you can think of differentiation as being a higher-order function that takes a function a and returns a new function as a result. If your language has support for higher-order functions, then (a) you have to extend the definition of the derivative to cope with differentiation at higher type, and (b) ideally you'd…

Re GoI and defunctionalization, is "On the relation of interaction semantics and defunctionalization" by Schopp a good reference? I've just started reading the literature regarding applications of GoI to pl theory and would love to add this line of research to my queue.

Re: Linear logic and deep learning [pdf]

#29

Post by davesque »

tw1010 wrote: post #8

To a trained mathematician, deep learning is so so far away from the cutting edge. If there's anything that's going to make a massive, revolutionary not evolutionary, change in the deep learning landscape, it's not going to come from engineers walking around on the surface of what's already there, it'll come from pure mathematicians connecting it to the insights ripe for the picking found deep deep down in the theore…

I guess that's why pure mathematicians have been at the forefront of all the major developments in neural networks and deep learning? This is the classic applied vs. theoretical debate. Let's not pretend like there's ever a clear winner.

Re: Linear logic and deep learning [pdf]

#30

Post by breatheoften »

YeGoblynQueenne wrote: post #16
thomasahle wrote: post #5

Earlier quoted context omitted.

> Machine learning is just one such technique which has gained popularity outside AI. I'd argue the machine learning label can be applied to any AI system that's data-driven in some sense (even self-generating the data using reinforcement learning). Wikipedia lists the following approaches to Machine Learning. Surely you wouldn't call them all 'one technique'?: Decision tree learning, Association rule learning, Artif…

Let's say that it's a class of techniques that learn models (or progams, in the case of ILP) from data. My main concern is not about defining what machine learning is, however. Rather, I'm worried about the definition of AI shrinking to "it's what we now call machine learning". Which is at the very least unhistorical.

I thought it was common for people to think of AI as largely disjoint from machine learning ... AI covering all the expert/rule based systems, grammars, and search algorithms with ML being pretty strictly referring to techniques for building predictive models from datasets ...
Post reply on HN