Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia library for fast machine learning

turing.ml

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Julia library for fast machine learning

#21

Post by wallnuss »

systemvoltage wrote: post #9
snicker7 wrote: post #7

The documentation / project page is very well-done, something unfortunately rare in the Julia ecosystem.

To add to that, even if Julia had excellent documentation literally everywhere and on every library, I wish there were better stack trace and meaningful error messages. Even if Julia performed 10x worse, this overlooked aspect of Julia would make up for it. It is rather unbelievable how much time I need to spend to figure out what's wrong with a particular piece of Julia code. Founders of Julia - please focus on erro…

It's being worked on. I am rather excited for the upcoming work that makes the parser replaceable and allows us to actually give good syntax errors! There is some discussion about making error printing more configurable so that one can skip stack-frames that are unlikely to be the cause (albeit that's a double-edged sword).

Re: Julia library for fast machine learning

#22

Post by FabHK »

photon-torpedo wrote: post #18
Buttons840 wrote: post #16

Earlier quoted context omitted.

In Julia you can `@which naked_function`. That might help. With Julia's function overloading a function might come from multiple packages.

Or rather, a function from one package can be extended with methods by other packages.

And to add:

Which of the many method of that function is called depends on the type of all the arguments (not only the first argument, as in single dispatch languages like C++). In other words, the implementation actually executed (and thus the source package) might vary depending on the argument type. If I understand Julia correctly.

Re: Julia library for fast machine learning

#23

Post by ViralBShah »

systemvoltage wrote: post #9
snicker7 wrote: post #7

The documentation / project page is very well-done, something unfortunately rare in the Julia ecosystem.

To add to that, even if Julia had excellent documentation literally everywhere and on every library, I wish there were better stack trace and meaningful error messages. Even if Julia performed 10x worse, this overlooked aspect of Julia would make up for it. It is rather unbelievable how much time I need to spend to figure out what's wrong with a particular piece of Julia code. Founders of Julia - please focus on erro…

As walnuss says below, these things are already being worked on. If you tried Julia a few months or couple of years ago, you'll find that stacktraces have already got quite a lot better.

One of the things that the Julia community could greatly benefit from is more compiler contributors. Languages like Rust naturally attract compiler folks, and those like Go have the backing of Google.

Julia has more complex compiler due to our dynamic type system, which the users absolutely love. But it also puts a lot of strain on the compiler team, which is quite small. So if any folks with experience in compiler technology on HN are looking for interesting projects to contribute to, please do look into Julia.

We've done better over the years increasing our bus number on the compiler codebase overall. More contributions in all areas of the toolchain would be great to have!

Re: Julia library for fast machine learning

#24

Post by krastanov »

ChrisRackauckas wrote: post #6
dklend122 wrote: post #5

Earlier quoted context omitted.

It currently has problems with some classes of models on the GPU, but this is just due to memory management and is going to fixed soon. Julia is natively compiled, so doesn't use a tape, tracing, or for memory management simple ref counting (in this case not as good). This is slated to be fixed in the short term with an abstract tracing framework which eliminates memory allocations. Given Julia's type information and…

Note you can use this on Zygote to preallocate stuff: https://github.com/oxinabox/AutoPreallocation.jl . It doesn't support GPUs yet, mainly because the dev on it needs a GPU CI setup, but it should mostly just work (issue https://github.com/oxinabox/AutoPreallocation.jl/issues/10 )

Wow... This is mind blowing! I am surprised there has not been more fanfare about this package.

Re: Julia library for fast machine learning

#26

Post by s1t5 »

FabHK wrote: post #22
photon-torpedo wrote: post #18

Earlier quoted context omitted.

Or rather, a function from one package can be extended with methods by other packages.

And to add: Which of the many method of that function is called depends on the type of all the arguments (not only the first argument, as in single dispatch languages like C++). In other words, the implementation actually executed (and thus the source package) might vary depending on the argument type. If I understand Julia correctly.

I still don't get why multiple dispatch was chosen over having seperate functions with typed arguments. It just seems to add complexity with limited benefit.

Re: Julia library for fast machine learning

#27

Post by pjmlp »

ViralBShah wrote: post #23
systemvoltage wrote: post #9

Earlier quoted context omitted.

To add to that, even if Julia had excellent documentation literally everywhere and on every library, I wish there were better stack trace and meaningful error messages. Even if Julia performed 10x worse, this overlooked aspect of Julia would make up for it. It is rather unbelievable how much time I need to spend to figure out what's wrong with a particular piece of Julia code. Founders of Julia - please focus on erro…

As walnuss says below, these things are already being worked on. If you tried Julia a few months or couple of years ago, you'll find that stacktraces have already got quite a lot better. One of the things that the Julia community could greatly benefit from is more compiler contributors. Languages like Rust naturally attract compiler folks, and those like Go have the backing of Google. Julia has more complex compiler…

As bystander I love the work you guys are doing, in a way it feels like Dylan was reborn and found a place in scientific computing.

Re: Julia library for fast machine learning

#29

Post by StefanKarpinski »

s1t5 wrote: post #26
FabHK wrote: post #22

Earlier quoted context omitted.

And to add: Which of the many method of that function is called depends on the type of all the arguments (not only the first argument, as in single dispatch languages like C++). In other words, the implementation actually executed (and thus the source package) might vary depending on the argument type. If I understand Julia correctly.

I still don't get why multiple dispatch was chosen over having seperate functions with typed arguments. It just seems to add complexity with limited benefit.

This video is my best attempt to explain it (not a rickroll, I swear): https://youtu.be/kc9HwsxE1OY

Re: Julia library for fast machine learning

#30

Post by ddragon »

s1t5 wrote: post #26
FabHK wrote: post #22

Earlier quoted context omitted.

And to add: Which of the many method of that function is called depends on the type of all the arguments (not only the first argument, as in single dispatch languages like C++). In other words, the implementation actually executed (and thus the source package) might vary depending on the argument type. If I understand Julia correctly.

I still don't get why multiple dispatch was chosen over having seperate functions with typed arguments. It just seems to add complexity with limited benefit.

It's hard to say limited benefit when the entire Julia ecosystem evolved to so heavily rely on it, to allow for each package to work on it's own level of abstraction: For example, the Julia's standard library implements all of the basic math operators on the CPU, and libraries like Flux then define it's own methods to implement the higher level operators used in ML (such as convolutional layers and activation functions). And then someone writes those same basic math operators but instead of running on CPU they run on GPU, and for that they use a new type (CUArray). The original library knows nothing about CUArrays, it will call the same basic operators as always, but since they have a different type (received from the user) they'll dispatch to the GPU version.

This kind of interaction can grow indefinitely, for example if you use a complex number type/library it will change the basic operators to deal with both real and imaginary parts, and if you use the GPU types within it, then it will do complex math in the GPU (and ML on complex math on GPU..., without any of the libraries being aware of the other). You can see a more detailed explanation on:

https://www.youtube.com/watch?v=kc9HwsxE1OY

Post reply on HN