Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automatic Differentiation with Julia

blog.rogerluo.me

Post reply on HN
Print view View original

1–10 of 83 posts

Re: Automatic Differentiation with Julia

#3

Post by one-more-minute »

ncfausti wrote: post #2

For those curious about Julia, I just found this: https://www.infoworld.com/article/3284380/data-science/what-... Close to C speed in a dynamic language? Seems pretty great on paper. Is this generally the case?

Generally, yes. It sounds like magic at first, but it's really just like a very lazy C++ compiler. You can happily look through the generated LLVM and machine code if you want to make sure it's reasonable.

Re: Automatic Differentiation with Julia

#4

Post by one-more-minute »

You may like to take a look at Flux's implementation [1]; roughly the same idea but "professionalised" with performance work, tighter integration with the type system, nested AD and so on. It's a little less simple for that, of course, but is still under 500loc of fairly straightforward Julia code, and is generally a bit faster than PyTorch.

The Julia world has done a lot of experimentation with AD and is converging on some really cool things, so if you're interested in this field it's definitely worth a look.

[1]: https://github.com/FluxML/Flux.jl/blob/master/src/tracker/Tr...

Re: Automatic Differentiation with Julia

#5

Post by sischoel »

ncfausti wrote: post #2

For those curious about Julia, I just found this: https://www.infoworld.com/article/3284380/data-science/what-... Close to C speed in a dynamic language? Seems pretty great on paper. Is this generally the case?

While Julia is indeed a dynamically typed language, it also supports type annotations and can in most cases infer the static type of a variable. If you write a function

  function foo(x)
    #do something
  end
and you call foo(10) and foo("some string"), then the compiler will create specialized methods foo(x::Int) and foo(x::String). Then there is no need for tracking the dynamic type of x inside these functions.

Re: Automatic Differentiation with Julia

#7

Post by FridgeSeal »

0-_-0 wrote: post #6

Julia is a great language, but I'm still waiting for someone to create a Julia fork that uses 0-based indexing.

So, fun times: if the 1-based indexing throws you off that much, it is entirely straightforward to configure it to use 0 based indexing if you want (or any other kind of offset that you so desire)

https://docs.julialang.org/en/latest/devdocs/offset-arrays/

Having said that, I encourage you to try out the 1-based indexing, as I think you might find a lot of things become surprisingly more intuitive.

Re: Automatic Differentiation with Julia

#9

Post by FridgeSeal »

ncfausti wrote: post #2

For those curious about Julia, I just found this: https://www.infoworld.com/article/3284380/data-science/what-... Close to C speed in a dynamic language? Seems pretty great on paper. Is this generally the case?

I've been using it at work and it's pretty great.

The performance is awesome, the REPL is super great as well. There's libraries that I'm itching to try out as soon as I've got a relevant project (Flux ML being the top of that list).

There's been a lot of situations in using it where I've just gone "this is everything I needed/wanted": the performance, the language features and API design, etc.

Plus there's a lot of very interesting things going on with the language and ecosystem, I definitely recommend trying it out.

Re: Automatic Differentiation with Julia

#10

Post by 0-_-0 »

pjmlp wrote: post #8
0-_-0 wrote: post #6

Julia is a great language, but I'm still waiting for someone to create a Julia fork that uses 0-based indexing.

There are others that actually like 1-based indexing languages.

And they already have Julia!

Looks like you can't joke with array indexing...

Post reply on HN