[deleted]
Automatic Differentiation in 38 lines of Haskell
21–30 of 60 posts
Re: Automatic Differentiation in 38 lines of Haskell
#22Fun exercise in Prolog too: https://www.codebymath.com/index.php/welcome/lesson/prolog-d....
Re: Automatic Differentiation in 38 lines of Haskell
#23One of my favourite Haskell "one-liners" is combining the AD package with Number.Symbolic: {-# LANGUAGE ImportQualifiedPost #-} module Module_1663406024_9206 where import Numeric.AD qualified as Ad import Data.Number.Symbolic qualified as Sym -- >>> f x = x^2 + 3 * x -- >>> Ad.diff f 1 -- >>> Ad.diff f (Sym.var "a") -- 5 -- 3+a+a -- >>> Ad.diff sin pi -- >>> Ad.diff sin (Sym.var "a") -- -1.0 -- cos a The package auth…
> The package authors did not need to coordinate to make this possible which is pretty wild. It works because `f` is polymorphic. The type of its `x` argument is not constrained in `f`'s definition, so you can plug in any `x` of any type you want provided that `x`'s type implements the methods used in `f`'s definition. With the `Dual` scheme you get to use as `x` a "dual" of `y` (`f x`, for some `f`) and `y'`, and th…
I still don't get how sin ends up as cos, without any coordination.
Re: Automatic Differentiation in 38 lines of Haskell
#24Is the Float' type there to not pollute the Float type with the defined typeclasses? Or is there something I am missing?
Re: Automatic Differentiation in 38 lines of Haskell
#25Is the Float' type there to not pollute the Float type with the defined typeclasses? Or is there something I am missing?
I have no clue. It looks totally redundant to me too.
Re: Automatic Differentiation in 38 lines of Haskell
#26To be clear this is a forward mode auto diff implementation, not reverse mode, as might be inferred by the reference to the SPJ talk, correct?
Re: Automatic Differentiation in 38 lines of Haskell
#27Earlier quoted context omitted.
> The package authors did not need to coordinate to make this possible which is pretty wild. It works because `f` is polymorphic. The type of its `x` argument is not constrained in `f`'s definition, so you can plug in any `x` of any type you want provided that `x`'s type implements the methods used in `f`'s definition. With the `Dual` scheme you get to use as `x` a "dual" of `y` (`f x`, for some `f`) and `y'`, and th…
I still don't get how sin ends up as cos, without any coordination.
Presumably the Ad package has a list of known derivatives. The Sym package now "automatically" uses it, without ever having to have known of it.
The "coordination" is that they both use the "symbol" sin to refer to the idea of sine function.
Re: Automatic Differentiation in 38 lines of Haskell
#28To be clear this is a forward mode auto diff implementation, not reverse mode, as might be inferred by the reference to the SPJ talk, correct?
Correct. Reverse mode seems much harder to express in Haskell..i was trying to understand AD some time ago and used Haskell for that (before running into Conal's paper). This way of writing forward AD was easy and it was awesome to see type inference and laziness help me with the understanding.
At that time, I tried to code up reverse mode AD and failed to do it with comparable simplicity.
If I may ...
1. First attempt - https://sriku.org/blog/2019/03/08/automatic-differentiation/
2. Dual numbers and Taylor numbers - http://sriku.org/blog/2019/03/12/automatic-differentiation-d...
3. Higher ranked beings - http://sriku.org/blog/2019/03/13/automatic-differentiation-h...
Re: Automatic Differentiation in 38 lines of Haskell
#29To be clear this is a forward mode auto diff implementation, not reverse mode, as might be inferred by the reference to the SPJ talk, correct?
Correct. Reverse mode seems much harder to express in Haskell..i was trying to understand AD some time ago and used Haskell for that (before running into Conal's paper). This way of writing forward AD was easy and it was awesome to see type inference and laziness help me with the understanding. At that time, I tried to code up reverse mode AD and failed to do it with comparable simplicity. If I may ... 1. First attem…
The struggle I had with reverse mode felt comparable to my struggle with coding up union find (for which mutation is crucial for its efficiency).
Re: Automatic Differentiation in 38 lines of Haskell
#30Is the Float' type there to not pollute the Float type with the defined typeclasses? Or is there something I am missing?
I have no clue. It looks totally redundant to me too.
You're absolutely right, it is totally redundant. You could reduce the entire thing even more in size by doing only `D Float Float`, removing the `TypeSynonymInstances` pragma and removing the `VectorSpace`, instead using normal Float operations for the derivatives in the Num, Floating and Fractional instances. Further, you can remove the `diff` function entirely if you want, since it's doing something very simple; calling `f` using a `D` instead of a normal Float, Int, etc. So you would simply get the function applied to x and it's derivative by doing `(\x -> x^2) (D 2 1)`, setting the derivative to 1 and x=2.
However, I think the Float' and `diff` etc. is at least a little helpful in understanding it. I got it from SPJ's talk, which I linked to in the file. Also, it makes it easier (e.g. in the case of `diff`) to later add onto the Autodiff, for example by implementing reverse mode, Jacobians, etc.