11–20 of 22 posts
Historical fact: Differentiable programming was a little known secret back in the 90's, used mainly by engineers simulating numerically stiff systems like nukes and chemicals in FORTRAN 95. It then disappeared for nearly 30 yrs before rediscovery by the ML/AI researchers!
I’m not deep into autodiff (just recall some calculus from university), but the syntax in this post reminds me a lot of ML (the programming language, not machine learning) I know autodiff isn’t lambda calculus, but the expression-based structure and evaluation rules feel similar. Couldn’t this be implemented in something like ML or Clojure? Just wondering what the custom DSL adds that existing functional languages wo…
Historical fact: Differentiable programming was a little known secret back in the 90's, used mainly by engineers simulating numerically stiff systems like nukes and chemicals in FORTRAN 95. It then disappeared for nearly 30 yrs before rediscovery by the ML/AI researchers!
Historical fact: Differentiable programming was a little known secret back in the 90's, used mainly by engineers simulating numerically stiff systems like nukes and chemicals in FORTRAN 95. It then disappeared for nearly 30 yrs before rediscovery by the ML/AI researchers!
A lot of the current new set of learning is that we have the compute power to do these things in more places. It is also something that has been long done in expensive environments that many of us just don't have access to.
I’m not deep into autodiff (just recall some calculus from university), but the syntax in this post reminds me a lot of ML (the programming language, not machine learning) I know autodiff isn’t lambda calculus, but the expression-based structure and evaluation rules feel similar. Couldn’t this be implemented in something like ML or Clojure? Just wondering what the custom DSL adds that existing functional languages wo…
Automatic differentiation can be implemented in essentially any language. Some just make it look "nicer".
Historical fact: Differentiable programming was a little known secret back in the 90's, used mainly by engineers simulating numerically stiff systems like nukes and chemicals in FORTRAN 95. It then disappeared for nearly 30 yrs before rediscovery by the ML/AI researchers!
I watched the ML/AI bros actively ignore previous research — even when they were requested to properly cite sources they were plagiarizing — in real time. The race to publish (even for big journals) was so important that it was easier to ignore the rank dishonesty than it was to correct their misbehavior. I'm 1000x happier to not have stayed around for all that crap.
Historical fact: Differentiable programming was a little known secret back in the 90's, used mainly by engineers simulating numerically stiff systems like nukes and chemicals in FORTRAN 95. It then disappeared for nearly 30 yrs before rediscovery by the ML/AI researchers!
My PhD dissertation included a chapter (originally from a 2006 paper) on generic programming in CAS' on the algorithmic differentiable ring (and operator). By the 1990s, algorithmic differentiation was easily 30–40 years old. Griewank & Monagan both knew guys who had built early electromagnetic naval targeting computers that used the methodology back by at least the early 60s "by hand". (Very literally.) I watched th…
That sounds like an interesting read. Do you have the chapter or the reference to the paper that you can share ?
Regarding th crop of deep neural network research their self-serving and willful blindness has a reputation that's well deserved.
A grad student from Hinton's lab mentioned one researcher who would misspell a citation on purpose so that the citation count of the cited paper does not go up.
Earlier quoted context omitted.
My PhD dissertation included a chapter (originally from a 2006 paper) on generic programming in CAS' on the algorithmic differentiable ring (and operator). By the 1990s, algorithmic differentiation was easily 30–40 years old. Griewank & Monagan both knew guys who had built early electromagnetic naval targeting computers that used the methodology back by at least the early 60s "by hand". (Very literally.) I watched th…
> on the algorithmic differentiable ring (and operator) That sounds like an interesting read. Do you have the chapter or the reference to the paper that you can share ? Regarding th crop of deep neural network research their self-serving and willful blindness has a reputation that's well deserved. A grad student from Hinton's lab mentioned one researcher who would misspell a citation on purpose so that the citation c…
Earlier quoted context omitted.
> on the algorithmic differentiable ring (and operator) That sounds like an interesting read. Do you have the chapter or the reference to the paper that you can share ? Regarding th crop of deep neural network research their self-serving and willful blindness has a reputation that's well deserved. A grad student from Hinton's lab mentioned one researcher who would misspell a citation on purpose so that the citation c…
My PhD, and its chapters are junk. Just read Griewank; I don't have anything more to say than his monograph. (We could never generalize the AD operator to the hybrid mode, so it's useless for real workloads.)
I think most people think so about their own dissertation. I have Griewank.
Earlier quoted context omitted.
My PhD, and its chapters are junk. Just read Griewank; I don't have anything more to say than his monograph. (We could never generalize the AD operator to the hybrid mode, so it's useless for real workloads.)
> My PhD, and its chapters are junk. I think most people think so about their own dissertation. I have Griewank.