Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Owl – An OCaml Numerical Library

github.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: Owl – An OCaml Numerical Library

#21

Post by escapetech »

tbenst wrote: post #6

I looked at OCaml a few years back for scientific computing, but was turned off by the global lock GC for multithreading (similar to python). Anyone know if this has been changed yet? Edit: doesn't look like this has changed, but there's an effort at https://github.com/ocamllabs/ocaml-multicore that is slowly making progress

Also problematic for numeric code in Ocaml is the lack of native interoperability with C data types and native arrays and instead having to use time-consuming abstractions such as BigArray or marshalling via FFI. Unfortunately, solutions like as CTypes only seem to compound the problem by not addressing a particular issue with a runtime, but finding clever ways around the limitation.

Re: Owl – An OCaml Numerical Library

#22

Post by vog »

etatoby wrote: post #17
Sean1708 wrote: post #14

Earlier quoted context omitted.

Slightly off topic sorry, but how Algorithm Differentiation usually implemented? Do you just apply the chain rule to an AST, or is there something more complex you have to do?

Notice that AD is a different technique than symbolic differentiation, which is what we do with pen and paper to compute a derivative. Wikipedia explains it properly, if a bit tersely: https://en.wikipedia.org/wiki/Automatic_differentiation In the case of Owl, it would appear that you have to write your functions using a set of overloaded operators contained in module Maths here: https://github.com/ryanrhymes/owl/blo…

> ... with their own specific algebra

Nevertheless, for simple computations this specific algebra indeed boils down to "applying the chain rule to the AST" (as well as the diff rules for all primitive functions). In forward mode, the only difference to symbolic differentiation is that common terms are reused (items in the AST, treating the AST as DAG), so the expression terms do not explode in size, and the computation is very efficient.

In reverse mode (and all other, more complicated modes), the chain rule still plays a central role, but is applied in a different way. Here, saying the "chain rule is applied to the AST" might still be technically correct, but would be very misleading, as the whole control flow is different, not to mention memory usage.

Re: Owl – An OCaml Numerical Library

#23

Post by Chris2048 »

vog wrote: post #10

Wow, this even supports Algorithmic Differentiation! https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic... (although "only" forward and reverse mode)

Is it the same as automatic differentiation?

Is this native support (as opposed to there being a library for it)?

Re: Owl – An OCaml Numerical Library

#24

Post by vog »

Chris2048 wrote: post #23
vog wrote: post #10

Wow, this even supports Algorithmic Differentiation! https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic... (although "only" forward and reverse mode)

Is it the same as automatic differentiation? Is this native support (as opposed to there being a library for it)?

> Is it the same as automatic differentiation?

Yes, "algorithmic differentiation" is the modern term for what was formerly called "automatic differentiation".

> Is this native support (as opposed to there being a library for it)?

I didn't look into the details, but it seems to be essentially the "operator overloading" approach, not the "source transformation" approach. However, given the optimizing compiler and OCaml's very good module type system, the result might be the same, at least for the forward mode.

Re: Owl – An OCaml Numerical Library

#25

Post by etatoby »

vog wrote: post #22
etatoby wrote: post #17

Earlier quoted context omitted.

Notice that AD is a different technique than symbolic differentiation, which is what we do with pen and paper to compute a derivative. Wikipedia explains it properly, if a bit tersely: https://en.wikipedia.org/wiki/Automatic_differentiation In the case of Owl, it would appear that you have to write your functions using a set of overloaded operators contained in module Maths here: https://github.com/ryanrhymes/owl/blo…

> ... with their own specific algebra Nevertheless, for simple computations this specific algebra indeed boils down to "applying the chain rule to the AST" (as well as the diff rules for all primitive functions). In forward mode, the only difference to symbolic differentiation is that common terms are reused (items in the AST, treating the AST as DAG), so the expression terms do not explode in size, and the computati…

I would add that (forward) Algorithmic Differentiation is also very well-suited for differentiating algorithms (lol, you wouldn't say!) and not just pure mathematical formulas.

For example, suppose you have an algorithm with loops, conditionals, recursion, sub-functions calls, even using global variables and side effects: if you use the overloaded operators for all numerical paths in your algorithm, AD can differentiate it in a natural way, just by executing it on the dual number system (AFAIK)

Re: Owl – An OCaml Numerical Library

#26

Post by etatoby »

fithisux wrote: post #13

Any hope for windows users? Hopefully OPAM works under windows OOTB.

Have you tried the recent Linux subsystem for Windows, or Ubuntu for Windows? I have had a lot of success running various development tools in it. You can use pre-compiled Ubuntu binaries or compile your own stack, and it all runs at native speed, since there is no virtualization or emulation of any kind.

https://news.ycombinator.com/item?id=11390545 https://news.ycombinator.com/item?id=14741714

Re: Owl – An OCaml Numerical Library

#28

Post by srean »

avsm wrote: post #16
srean wrote: post #15

Earlier quoted context omitted.

> I think multicore capabilities will be here soon. Just a word of caution to set expectations. The "will be here soon" status has been there for a while, to the extent that it now evokes memories of GNU Hurd, Duke Nukem Forever. It will be ready whenever it will be ready, its in good hands, but don't hold your breath, try to work around it for now.

Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September. There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionew…

> so it's not quite fair to compare it to Duke Nukem Forever :-)

Apologies if it came out harsh. Oodles of respect for all the work you guys are doing. Its a lot of work and that's why I said things are in good hands.

Re: Owl – An OCaml Numerical Library

#29

Post by kornish »

etatoby wrote: post #20

This is going to be extremely useful! I just recently settled on OCaml as my language of choice for most projects, after working with dozens of others, from C to Haskell. I was actually looking for a good numerical library, supporting complex numbers, matrix operations, and multi-dimensional arrays. I'm looking forward to using and contributing to this one!

What are some of the factors that contribute to OCaml being your language of choice / do you feel limited by the poor multicore parallelism story?

Re: Owl – An OCaml Numerical Library

#30

Post by throwaway7645 »

kornish wrote: post #29
etatoby wrote: post #20

This is going to be extremely useful! I just recently settled on OCaml as my language of choice for most projects, after working with dozens of others, from C to Haskell. I was actually looking for a good numerical library, supporting complex numbers, matrix operations, and multi-dimensional arrays. I'm looking forward to using and contributing to this one!

What are some of the factors that contribute to OCaml being your language of choice / do you feel limited by the poor multicore parallelism story?

Multicore is always just around the corner!
Post reply on HN