Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia library for fast machine learning

turing.ml

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Julia library for fast machine learning

#31

Post by mkborregaard »

Buttons840 wrote: post #16
s1t5 wrote: post #13

Looking at the examples, I still struggle with code that imports multiple libraries at the top and then uses naked function names without telling me where those functions come from.

In Julia you can `@which naked_function`. That might help. With Julia's function overloading a function might come from multiple packages.

Even though you can do `@which naked_function` it's a fair position to have that you prefer to import modules and be explicit about where they come from. I tend to prefer explicitness like that in package code, but for data analytical scripts (like the examples here) it would be superfluous in my opinion.

It is nice that Julia leaves this style decision to the user. I personally find the constant prepending of modules to be one of the clumsiest aspects of Python for data analysis.

Re: Julia library for fast machine learning

#32

Post by mark_l_watson »

Turing.jl is great as is Flux.jl (which I have used more).

I retired a year ago and was looking to settle down and just use one programming language - easier on an old guy like myself, who has always been fascinated with many programming languages. I ended up picking Common Lisp for my retirement projects, mostly because of almost 4 decades experience with CL.

For a Lisp programmer, Julia feels like a very comfortable language. The interactive repl is very nice. I experimented with Flux a lot, and wrote snippets for web scraping, text processing, etc. It also worked really well for these non-numeric use cases: a very good general programming language, not just for numeric processing.

Julia has great Python interop so just in case you need a Python library, you have it available.

Re: Julia library for fast machine learning

#33

Post by s1t5 »

StefanKarpinski wrote: post #29
s1t5 wrote: post #26

Earlier quoted context omitted.

I still don't get why multiple dispatch was chosen over having seperate functions with typed arguments. It just seems to add complexity with limited benefit.

This video is my best attempt to explain it (not a rickroll, I swear): https://youtu.be/kc9HwsxE1OY

Nice talk, you're a really good speaker. Thanks.

Re: Julia library for fast machine learning

#34

Post by cuban_MMA »

ChrisRackauckas wrote: post #8
abeppu wrote: post #2

Can someone informed give some suggestions as to compare/contrast to other tools at the intersection of probabilistic programming and deep learning? What are relative strengths and weaknesses vs edward or pyro?

Turing.jl is in an interesting spot because it is essentially a DSL-free probabilistic programming language. While it technically has a DSL of sorts given by the `@model` macro, anything that is AD-compatible can be used in this macro and since Julia's AD tools work on things written in the Julia language, this means that you can just throw code from other Julia packages into Turing and just expect AD-compatible thin…

Hi Chris, as a heads up, Stan actually has had three built-in methods for a while now. There is a non-stiff Adams-Moulton solver introduced in 2018. It unfortunately was only just exposed in the Stan 2.23 documentation: https://mc-stan.org/docs/2_23/stan-users-guide/ode-solver-ch.... Certainly, the devs have been talking about adding more solvers for a while, including SDE and DDE solvers, and your DifferentialEquations.jl ecosystem is an excellent model; it is an area that we know Stan has been lacking in. I think Steve Gronder will be trying to work with you regarding benchmarking.

Re: Julia library for fast machine learning

#35

Post by ChrisRackauckas »

cuban_MMA wrote: post #34
ChrisRackauckas wrote: post #8

Earlier quoted context omitted.

Turing.jl is in an interesting spot because it is essentially a DSL-free probabilistic programming language. While it technically has a DSL of sorts given by the `@model` macro, anything that is AD-compatible can be used in this macro and since Julia's AD tools work on things written in the Julia language, this means that you can just throw code from other Julia packages into Turing and just expect AD-compatible thin…

Hi Chris, as a heads up, Stan actually has had three built-in methods for a while now. There is a non-stiff Adams-Moulton solver introduced in 2018. It unfortunately was only just exposed in the Stan 2.23 documentation: https://mc-stan.org/docs/2_23/stan-users-guide/ode-solver-ch... . Certainly, the devs have been talking about adding more solvers for a while, including SDE and DDE solvers, and your DifferentialEquat…

Awesome. If we can get FFI with Stan I'd like to connect DifferentialEquations.jl to it and poke at it with various problems to see how well it does on a few things. We can provide custom gradients if there's an interface for it, but I couldn't figure out how to do it without modifying the Stan source itself.

Re: Julia library for fast machine learning

#36

Post by thisrod »

systemvoltage wrote: post #9
snicker7 wrote: post #7

The documentation / project page is very well-done, something unfortunately rare in the Julia ecosystem.

To add to that, even if Julia had excellent documentation literally everywhere and on every library, I wish there were better stack trace and meaningful error messages. Even if Julia performed 10x worse, this overlooked aspect of Julia would make up for it. It is rather unbelievable how much time I need to spend to figure out what's wrong with a particular piece of Julia code. Founders of Julia - please focus on erro…

I don't think that there's a silver bullet for this. A compiler can only tell you where it caught an error: telling you where the error occurred is equivalent to the hard AI problem.

On the other hand, we the users can do a lot to help. After you've worked through those 30 stack frames to find the one that went wrong, you could send a PR to detect that. In many cases, I suspect it would help to just catch the error then re-throw it from the method that went wrong.

Re: Julia library for fast machine learning

#37

Post by ivanyashchuk »

ChrisRackauckas wrote: post #35
cuban_MMA wrote: post #34

Earlier quoted context omitted.

Hi Chris, as a heads up, Stan actually has had three built-in methods for a while now. There is a non-stiff Adams-Moulton solver introduced in 2018. It unfortunately was only just exposed in the Stan 2.23 documentation: https://mc-stan.org/docs/2_23/stan-users-guide/ode-solver-ch... . Certainly, the devs have been talking about adding more solvers for a while, including SDE and DDE solvers, and your DifferentialEquat…

Awesome. If we can get FFI with Stan I'd like to connect DifferentialEquations.jl to it and poke at it with various problems to see how well it does on a few things. We can provide custom gradients if there's an interface for it, but I couldn't figure out how to do it without modifying the Stan source itself.

In order to connect Stan with DifferentialEquations.jl the steps would be:

1. Create "diffeqcpp", a C++ interface to DifferentialEquations.jl (that would be similar to diffeqpy, diffeqr) possibly using CxxWrap.jl

2. Make it possible to evaluate vector-Jacobian products (VJP) with "diffeqcpp". Probably that would require ODE RHS to be coded as a string of Julia code, to make Julia AD libraries compatible with it.

At this point, it should be possible to call Julia solvers from C++ and evaluate the derivatives.

In Stan, there is stan::math::adj_jac_apply that makes it possible to define custom functions with custom VJP without having to deal with Stan autodiff types, it works for example with Eigen::Matrix. https://discourse.mc-stan.org/t/adj-jac-apply/5163

3. Make a class (let's call it JuliaODESolver) that implements two methods:

    operator() // calls Julia solver for the given input 

    multiply_adjoint_jacobian() // evaluates VJP for the given vector
4. In .stan file add a custom function in "functions {}" block, and write a header file that implements that custom function. That would probably be one line

    return stan::math::adj_jac_apply(ode_solver_inputs);
More info on using external C++ code is in Section 4.5 CmdStan Manual.

5. Modify cmdstan/main.cpp to initialize and finalize Julia context to be able to call Julia functions. This is probably the only place where Stan source itself needs to be modified.

I don't know what would be needed to make forward mode, and higher-order derivatives to work.

I think it would be much better for a fair benchmarking if there was a convenient and documented interface to Stan algorithms to use with user-provided log-density function, similar to DynamicHMC.jl and AdvancedHMC.jl libraries. It would be then easy to call it from Julia/Python/R/C++ or anything else.

Re: Julia library for fast machine learning

#38

Post by patagurbon »

ViralBShah wrote: post #23
systemvoltage wrote: post #9

Earlier quoted context omitted.

To add to that, even if Julia had excellent documentation literally everywhere and on every library, I wish there were better stack trace and meaningful error messages. Even if Julia performed 10x worse, this overlooked aspect of Julia would make up for it. It is rather unbelievable how much time I need to spend to figure out what's wrong with a particular piece of Julia code. Founders of Julia - please focus on erro…

As walnuss says below, these things are already being worked on. If you tried Julia a few months or couple of years ago, you'll find that stacktraces have already got quite a lot better. One of the things that the Julia community could greatly benefit from is more compiler contributors. Languages like Rust naturally attract compiler folks, and those like Go have the backing of Google. Julia has more complex compiler…

As a user of Julia for 2-3 years, is there any guidance for someone looking to help with the compiler?

Re: Julia library for fast machine learning

#40

Post by darsnack »

nerdponx wrote: post #4
0d9eooo wrote: post #3

Earlier quoted context omitted.

I've only used each of those a little. Turing is a bit like Stan, JAGS, or BUGS, it's [closer to] a general probabilistic programming system with a Bayesian emphasis, although maybe less Bayesian in emphasis than those other DSLs. PyMC would be another comparison. Edward is (or when I used it) coming more from a very general latent variable modeling framework, encompassing hidden variable models, and is less focused…

I've been excited by Turing, just to see a probabilistic programming framework like that in Julia. I think the expressiveness of Julia and it being native to that framework will be helpful. I hope so too. But hasn't Julia's TF/Torch equivalent, Flux, had performance problems? That was the rumor I heard anyway, I haven't had the chance to use it myself.

Most of that is not fundamental to Julia or Flux itself. It’s the difference between a monolithic package like TF and source-to-source AD in Julia. The former allows the designers to use their own data structures and external libraries to do optimizations. Source-to-source relies on the underlying IR used by Julia, making optimizations challenging without some compiler assistance. But all of that is in the pipeline with stopgap solutions on the way.

As with most things in Julia, the code developers don’t just want to hack changes that work, but make changes that are flexible, extensible, and can solve many problems at once. So, Flux isn’t ready for prime time yet, but it is definitely worth keeping your eye on it.

Post reply on HN