Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automatic Differentiation in 38 lines of Haskell

gist.github.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Automatic Differentiation in 38 lines of Haskell

#11

Post by ttesmer »

xiphias2 wrote: post #10
sterlind wrote: post #4

This is an interesting approach. Haskell is not a symbolic language, but you take advantage of the abstractness of type parameters in function definitions to thread your implementation of "D x" through, and pattern match on that. It's a neat design pattern. I bet it'd work in Julia too.

Yes, but Julia has both forward and backward differention implemented (backwards it's harder).

As I wrote in the Markdown file, there's also a usable package for Haskell called `ad` on Hackage. It has both forward and backward autodiff and prevents expression swell, among other things. This gist is just for illustration purposes.

Re: Automatic Differentiation in 38 lines of Haskell

#14

Post by version_five »

sterlind wrote: post #4

This is an interesting approach. Haskell is not a symbolic language, but you take advantage of the abstractness of type parameters in function definitions to thread your implementation of "D x" through, and pattern match on that. It's a neat design pattern. I bet it'd work in Julia too.

For an example in julia, see Mike Innes tutorial: https://github.com/MikeInnes/diff-zoo

I'm only a beginner in Julia and not and AD expert, but I went through the exercise of porting this to python and found it very enlightening

Re: Automatic Differentiation in 38 lines of Haskell

#15

Post by chrsig »

one of the references is a talk by simon peyton jones on AD[0]

I'm neither a math expert nor a haskell expert, but I happen to enjoy both. It's been a while since I've watched a SPJ lecture, and I'd forgotten how much I want him to explain everything.

[0] https://www.youtube.com/watch?v=EPGqzkEZWyw

Re: Automatic Differentiation in 38 lines of Haskell

#16

Post by chrsig »

mghwaz wrote: post #13

How do you come up with code looking this good? I've been working in Haskell for 6 months professionally: currently - I wouldn't be able to come up with something like this in a day (or more).

generally this isn't something that you come up with in a day. it's distilled down and refined over time, you just see it when it reaches maturity.

can't speak to the OPs process though, maybe they shat it out on a whim :)

Re: Automatic Differentiation in 38 lines of Haskell

#17

Post by sordina »

One of my favourite Haskell "one-liners" is combining the AD package with Number.Symbolic:

    {-# LANGUAGE ImportQualifiedPost #-}
    
    module Module_1663406024_9206 where
    
    import Numeric.AD qualified as Ad
    import Data.Number.Symbolic qualified as Sym
    
    -- >>> f x = x^2 + 3 * x
    -- >>> Ad.diff f 1
    -- >>> Ad.diff f (Sym.var "a")
    -- 5
    -- 3+a+a

    -- >>> Ad.diff sin pi
    -- >>> Ad.diff sin (Sym.var "a")
    -- -1.0
    -- cos a
The package authors did not need to coordinate to make this possible which is pretty wild.

Saw it first here: https://twitter.com/GabriellaG439/status/647601518871359489 and https://www.reddit.com/r/haskell/comments/3r75hq/comment/cwm...

Re: Automatic Differentiation in 38 lines of Haskell

#18

Post by cryptonector »

green_on_black wrote: post #3

Curious: I don't see the `^` op defined, or is it translated inti `exp` im guessing?

What Haskell calls a "class" is what Java calls an "interface". What Haskell calls an "instance" of an interface is what Java would call a class implementing that interface.

`Num`, then, is a Haskell class (Java interface).

The `Num` class will have a bunch of what Java would call "default methods".

Now, the "instance" of `Num` defined here has only a few methods defined, but the other default methods of `Num` will use those. So if `Num` has a `^` defined in terms of ``, and you define an instance of `Num` that defines ``, then you get `^` for free if you don't implement it.

Re: Automatic Differentiation in 38 lines of Haskell

#19

Post by jhgb »

sterlind wrote: post #6
mountainriver wrote: post #2

Being purely functional makes this quite easy, still a beauty to see

it's not just the functionalness. you couldn't do it this way in Lisp/Scheme (I think?) because of the lack of multiple dispatch. If you did (f 'x) for instance, you'd end up with things like (* 2 'x) which would blow up, since Lisp would try to compute the answer instead giving you '(* 2 x) back.

I'm sure that systems like ScmUtils wouldn't have problems with this.

Re: Automatic Differentiation in 38 lines of Haskell

#20

Post by cryptonector »

sordina wrote: post #17

One of my favourite Haskell "one-liners" is combining the AD package with Number.Symbolic: {-# LANGUAGE ImportQualifiedPost #-} module Module_1663406024_9206 where import Numeric.AD qualified as Ad import Data.Number.Symbolic qualified as Sym -- >>> f x = x^2 + 3 * x -- >>> Ad.diff f 1 -- >>> Ad.diff f (Sym.var "a") -- 5 -- 3+a+a -- >>> Ad.diff sin pi -- >>> Ad.diff sin (Sym.var "a") -- -1.0 -- cos a The package auth…

> The package authors did not need to coordinate to make this possible which is pretty wild.

It works because `f` is polymorphic. The type of its `x` argument is not constrained in `f`'s definition, so you can plug in any `x` of any type you want provided that `x`'s type implements the methods used in `f`'s definition. With the `Dual` scheme you get to use as `x` a "dual" of `y` (`f x`, for some `f`) and `y'`, and then you get an `f` applied to that `x` where the actual `f` is parameterized by the actual `x`'s type, and so the methods called by `f` are those that apply to `x`'s type. So instead of the traditional numeric addition and multiplication, you'd get the "dual" addition and multiplication, and then everything "chains" through and you end up with `diff f x` being the `y'` in the dual of `y` and `y'` (you don't care about the `y`, just the `y'` because you want the `diff` -- the differential or derivative).

It's brilliant.

Post reply on HN