log (D u u') = D (log u) (scale (log u) u')
Automatic Differentiation in 38 lines of Haskell
31–40 of 60 posts
Re: Automatic Differentiation in 38 lines of Haskell
#32This is an interesting approach. Haskell is not a symbolic language, but you take advantage of the abstractness of type parameters in function definitions to thread your implementation of "D x" through, and pattern match on that. It's a neat design pattern. I bet it'd work in Julia too.
> Haskell is not a symbolic language
I'm not sure I understand what this means. What is a symbolic language that excludes languages like Haskell, F#, OCaml, etc.?
Re: Automatic Differentiation in 38 lines of Haskell
#33Forward-mode automatic differentiation is always fun to see because of the power but simplicity of the method. Any language with pattern matching makes it almost trivial to implement.
Although, I'm rarely interested in in lines of . The more interesting things are overall conciseness with regards to the problem, the expressiveness, and the clarity that the code produces.
Re: Automatic Differentiation in 38 lines of Haskell
#34Earlier quoted context omitted.
I still don't get how sin ends up as cos, without any coordination.
Presumably the Ad package has a list of known derivatives. The Sym package now "automatically" uses it, without ever having to have known of it. The "coordination" is that they both use the "symbol" sin to refer to the idea of sine function.
Right, I was confused because for some reason I imagined "sin" coming from the symbolic library, but I'm assuming it's just built-in so AD knows about.
Re: Automatic Differentiation in 38 lines of Haskell
#35Earlier quoted context omitted.
Presumably the Ad package has a list of known derivatives. The Sym package now "automatically" uses it, without ever having to have known of it. The "coordination" is that they both use the "symbol" sin to refer to the idea of sine function.
Right, I was confused because for some reason I imagined "sin" coming from the symbolic library, but I'm assuming it's just built-in so AD knows about.
The `sin` function comes from this bit at the end of TFA:
instance VectorSpace d => Floating (Dual d) where
pi = D pi zero
exp (D u u') = D (exp u) (scale (exp u) u')
log (D u u') = D (log u) (scale (log u) u')
--->sin (D u u') = D (sin u) (scale (cos u) u')
cos (D u u') = D (cos u) (scale (-sin u) u')
sinh (D u u') = D (sinh u) (scale (cosh u) u')
cosh (D u u') = D (cosh u) (scale (sinh u) u')
Re: Automatic Differentiation in 38 lines of Haskell
#36This is an interesting approach. Haskell is not a symbolic language, but you take advantage of the abstractness of type parameters in function definitions to thread your implementation of "D x" through, and pattern match on that. It's a neat design pattern. I bet it'd work in Julia too.
Yes, but Julia has both forward and backward differention implemented (backwards it's harder).
No post body was provided.
Re: Automatic Differentiation in 38 lines of Haskell
#37Earlier quoted context omitted.
Right, I was confused because for some reason I imagined "sin" coming from the symbolic library, but I'm assuming it's just built-in so AD knows about.
The `sin` function comes from this bit at the end of TFA: instance VectorSpace d => Floating (Dual d) where pi = D pi zero exp (D u u') = D (exp u) (scale (exp u) u') log (D u u') = D (log u) (scale (log u) u') --->sin (D u u') = D (sin u) (scale (cos u) u') cos (D u u') = D (cos u) (scale (-sin u) u') sinh (D u u') = D (sinh u) (scale (cosh u) u') cosh (D u u') = D (cosh u) (scale (sinh u) u') and the `sin` function…
Not quite. This subthread is about the extremely short, one-line implementation mentioned here https://news.ycombinator.com/item?id=32882825 (which merges two unrelated modules (autodiff and symbolic) and uses autodiff to implement symbolic differentiation). Your comment is true for the original 38-line implemention of autodiff at the very top of the thread, but not in this subthread. The 38-line implementation is similar to the aforementioned autodiff module though.
Re: Automatic Differentiation in 38 lines of Haskell
#38log (D u u') = D (log u) (scale (log u) u') That doesn't look like the derivative of the natural logarithm!
(It was fixed shortly after your comment was posted.)
Re: Automatic Differentiation in 38 lines of Haskell
#39This is an interesting approach. Haskell is not a symbolic language, but you take advantage of the abstractness of type parameters in function definitions to thread your implementation of "D x" through, and pattern match on that. It's a neat design pattern. I bet it'd work in Julia too.
> Haskell is not a symbolic language I'm not sure I understand what this means. What is a symbolic language that excludes languages like Haskell, F#, OCaml, etc.?
Perhaps they mean not homoiconic like lisp or forth.
Re: Automatic Differentiation in 38 lines of Haskell
#40To be clear this is a forward mode auto diff implementation, not reverse mode, as might be inferred by the reference to the SPJ talk, correct?
[deleted]