Live data from Hacker News

Deep learning experiments in OCaml

blog.janestreet.com

91–99 of 99 posts

Re: Deep learning experiments in OCaml

#91

Earlier quoted context omitted.

> “Say someone wants to implement a deep learning model with complex valued activations or quaternion valued activations. What then?” This sounds like premature abstraction to me...

There is work on rotationally invariant networks, e.g. for identifying galaxies, or cells under a microscope. For example: https://arxiv.org/abs/1612.04642 https://arxiv.org/abs/1805.12301 I haven't looked closely enough to be sure if they literally had complex activations, but this seems like an obvious use. Maybe they would have, if only tensorflow made it easy.

Why would you ever want to represent activations directly as complex numbers in that case? I can’t think of any good reason, compared with putting them in rotation matrix form or some other equivalent form that actually maps to the domain modeling problem.

Even when working in signal processing problems that require complex arithmetic, the underlying representations are just based on tuples of doubles and operator conventions, and you always need to map to real spaces (real part, imaginary part, angle, or magnitude) for any type of analytical representation that can be human readable.

In all these cases, the idea that what we should optimize for is overhead-free easy expression of cutesy math domain verbiage is a bad idea.

Writing libraries that expose an API that matches the user’s domain mental model is a great thing. But enforcing a particular abstraction and extensibility hierarchy so those things can be “autogenerated” just by parameterizing over a new type turns out to be actually much worse than just writing that type separately, with helper functions and converters, and customizing its API to be efficient from a domain mental model perspective.

A better way, for example, might be to use mixin patterns or decorators and other metaprogramming, while writing a custom data type and its associated methods.

Re: Deep learning experiments in OCaml

#92

Earlier quoted context omitted.

There is work on rotationally invariant networks, e.g. for identifying galaxies, or cells under a microscope. For example: https://arxiv.org/abs/1612.04642 https://arxiv.org/abs/1805.12301 I haven't looked closely enough to be sure if they literally had complex activations, but this seems like an obvious use. Maybe they would have, if only tensorflow made it easy.

Why would you ever want to represent activations directly as complex numbers in that case? I can’t think of any good reason, compared with putting them in rotation matrix form or some other equivalent form that actually maps to the domain modeling problem. Even when working in signal processing problems that require complex arithmetic, the underlying representations are just based on tuples of doubles and operator co…

Something like this? https://github.com/andyferris/Traitor.jl

Re: Deep learning experiments in OCaml

#93

Earlier quoted context omitted.

How is it premature abstraction if it takes zero extra lines of code to support it and have it optimized? That's kind of the beauty of Julia.

It didn’t require zero lines of code. It required a huge amount of backend code to set up the abstraction and make lots of built-in types that adhere to the abstraction. And in cases when the abstraction fails to offer the exact type of extensibility needed (which is most of the time unless you’re authoring yet another highly abstracted library that can tie its use cases to that underlying abstraction, which is never…

Right here we have a classic case of someone on the internet anonymously saying something is impossible to do while there are many many examples of exactly this working. I recommend readers of these posts to ignore the FUD and do some Google searches to look through some Julia code repositories to see it in action. There are some great tutorials and fact-based discussions out there that can lead you to some useful examples with tricks you can employ in your own code.

Re: Deep learning experiments in OCaml

#94

Earlier quoted context omitted.

It's for differential equations. Getting uncertainty without parameter sampling saves a lot of computational time and really opens up the problems that can be solved.

But why does “getting uncertainty without parameter sampling” have anything to do with parametric numeric types? The latter is just a possible manner of implementation (that I’d argue is too cutesy), but there are many other ways to design a system like that, for example like fused types in Cython combined with numba class jitting. I still see no reason to believe that something that parametrizes differential equatio…

The Julia solution didn't require anyone to actually think about making it work, and it works well (we found out from a Discourse post that it works, the developers didn't even know :)), and now we are using it in our research codes because it is a great way to speedup what was traditionally done via parameter sampling.

Until I see someone else take an existing ODE solver like LSODA and convert it into something that can output uncertainties without having to do parameter sampling, I won't think other ecosystems are very close to what we have already done. Places like SciPy are still calling out to Fortran routines from ODEPACK for this, so making it work with Numba class jitting is a long way away. Show how easy it is to code it by showing code. Ours is already done: the ball is in your court.

Re: Deep learning experiments in OCaml

#95

Earlier quoted context omitted.

It didn’t require zero lines of code. It required a huge amount of backend code to set up the abstraction and make lots of built-in types that adhere to the abstraction. And in cases when the abstraction fails to offer the exact type of extensibility needed (which is most of the time unless you’re authoring yet another highly abstracted library that can tie its use cases to that underlying abstraction, which is never…

Right here we have a classic case of someone on the internet anonymously saying something is impossible to do while there are many many examples of exactly this working. I recommend readers of these posts to ignore the FUD and do some Google searches to look through some Julia code repositories to see it in action. There are some great tutorials and fact-based discussions out there that can lead you to some useful ex…

It looks like you are just posting knee-jerk defensive posts about julia I guess. Whatever this is, it’s clearly not related to my earlier comments in the thread.

What are you talking about? Where did I say any of this was not possible? It’s obviously possible.

It just turns out to be bad when you do it. It causes problems that the company line memo about zero overhead never is upfront about.

Re: Deep learning experiments in OCaml

#96

Earlier quoted context omitted.

But why does “getting uncertainty without parameter sampling” have anything to do with parametric numeric types? The latter is just a possible manner of implementation (that I’d argue is too cutesy), but there are many other ways to design a system like that, for example like fused types in Cython combined with numba class jitting. I still see no reason to believe that something that parametrizes differential equatio…

The Julia solution didn't require anyone to actually think about making it work, and it works well (we found out from a Discourse post that it works, the developers didn't even know :)), and now we are using it in our research codes because it is a great way to speedup what was traditionally done via parameter sampling. Until I see someone else take an existing ODE solver like LSODA and convert it into something that…

I just looked up your paper with Nie and indeed you’re just using a particular set of patterns with multiple dispatch and metaprogramming. Nothing is fundamentally different than other ways of implementing the same thing that don’t rely on parametric abstraction.

You seem not to know about numba and Cython given that you responded with a comment about scipy using FORTRAN, which is not relevant. You can do the exact same multiple dispatch patterns with Cython fused types, and with several dispatch techniques in numba.

Look, I’m glad people like your library. It doesn’t change the larger points about this type of design pattern being premature abstraction.

Re: Deep learning experiments in OCaml

#97
post #92

Earlier quoted context omitted.

Why would you ever want to represent activations directly as complex numbers in that case? I can’t think of any good reason, compared with putting them in rotation matrix form or some other equivalent form that actually maps to the domain modeling problem. Even when working in signal processing problems that require complex arithmetic, the underlying representations are just based on tuples of doubles and operator co…

Something like this? https://github.com/andyferris/Traitor.jl

That is definitely a cool direction to take it!

Re: Deep learning experiments in OCaml

#98

Earlier quoted context omitted.

>Organizing things with type class patterns and algebraic data types costs you by making you write more code and more boilerplate And tests are writing themselves for you?

I don’t understand why you believe that question is rhetorically interesting or connected to anything that has been discussed. You’d have to write virtually all the same tests in this type of use case whether you are using the static typing approach or not. The tests won’t explicitly check types in the dynamic typing case, but will verify type safety for fixture settings and data indirectly, as a byproduct of all the…

>You’d have to write virtually all the same tests in this type of use case whether you are using the static typing approach or not.

No, I don't need to write tests, it I could prove something with types. Here is an example of quicksort, where all invariants and properties are ensured with types, so this code does not need any tests at all.

https://github.com/FStarLang/FStar/blob/master/examples/algo...

>integration tests

You could reason about your program's correctness on any level with types.

Re: Deep learning experiments in OCaml

#99

Earlier quoted context omitted.

The Julia solution didn't require anyone to actually think about making it work, and it works well (we found out from a Discourse post that it works, the developers didn't even know :)), and now we are using it in our research codes because it is a great way to speedup what was traditionally done via parameter sampling. Until I see someone else take an existing ODE solver like LSODA and convert it into something that…

I just looked up your paper with Nie and indeed you’re just using a particular set of patterns with multiple dispatch and metaprogramming. Nothing is fundamentally different than other ways of implementing the same thing that don’t rely on parametric abstraction. You seem not to know about numba and Cython given that you responded with a comment about scipy using FORTRAN, which is not relevant. You can do the exact s…

[deleted]
Post reply on HN