Earlier quoted context omitted.
Maybe this is the piece I don't understand. What do you mean by "more than an expression-only language"? What is a non-expression in a language? And what would it mean to have a derivative for your non-expression?
You can easily use expressions to create trees rather than values in a library. Eg a + b need not compute a value, through a bit of operating overloading it can compute the tree plus(tree-a, tree-b). This “trick” does not extend to statements, however. You can’t override if or semicolon in most languages. You can encode statements as expressions, but then you have to worry about things like variable bindings on your…
Building expression trees is not the only way to do automatic differentiation. A simpler way is to just carry the value and it's derivative as a pair:
#include
#include
struct autodiff { double value, deriv; };
autodiff just(double x) { return { x, 1 }; }
autodiff operator +(autodiff a, autodiff b) {
return { a.value + b.value, a.deriv + b.deriv };
}
autodiff operator *(autodiff a, autodiff b) {
return { a.value * b.value, a.deriv*b.value + a.value*b.deriv };
}
autodiff sin(autodiff a) {
return { sin(a.value), cos(a.value)*a.deriv };
}
int main() {
autodiff x = just(.1);
for (int ii = 0; iiThis generalizes to partial derivatives for multivariate functions too:
template
struct autograd { double value, grad[dims}; };