The Elements of Differentiable Programming
1–10 of 84 posts
Re: The Elements of Differentiable Programming
#2[deleted]
Re: The Elements of Differentiable Programming
#3This is amazing! Seems like a perfect excuse to get back into Julia. I just wish Julia had more compile targets. Ideally I'd like to have the option to target the browser (wasm/webgpu).
Re: The Elements of Differentiable Programming
#4This is amazing! Seems like a perfect excuse to get back into Julia. I just wish Julia had more compile targets. Ideally I'd like to have the option to target the browser (wasm/webgpu).
Re: The Elements of Differentiable Programming
#5Re: The Elements of Differentiable Programming
#6This is amazing! Seems like a perfect excuse to get back into Julia. I just wish Julia had more compile targets. Ideally I'd like to have the option to target the browser (wasm/webgpu).
The Julia community seems like they've been out front when it comes to differentiable programming. For example: https://www.youtube.com/watch?v=rF2QAJLM730
Re: The Elements of Differentiable Programming
#7i don't know why people write these things. it's an absolute hodge-podge of theorem/proofs/results/techniques with no unifying theme other than "CALCULUS". so it's a pretty bad math book to actually learn math from (you can always spot a pedagogically unsound math book by its lack of exercises). the book doesn't even have any code in it which is surprising considering it has "programming" in the title.
actually i know why people write these but i still don't know why they publish them: this is a phase everyone goes through in their "math" life where they look back on everything they've learned hastily between undergrad/phd/postdoc (or whatever) and they have the urge to formalize/crystallize. everyone has the urge - i had a late-career QFT prof tell me that he was excited to take his sabbatical so that he could finally do all of the exercises in peskin&schroeder for real real and type it all up neatly.
i've done it too, in-the-small (some very nice notes that i'm proud of, on various things). you sit down, make your list of things, pull up all of the books/papers you're going to use as references and you start essentially transcribing - but you tell yourself you're putting your own spin on it (adding ample "motivation"). and it's all fine and healthy and gratifying for you and yourself alone. but i don't think i'd ever imagine to myself "well hmm my organization of these topics is going to be useful for other people i should put it out there for the world to see". but that's just me.
ninja edit:
before someone jumps down my throat about "what's the harm?". the harm is n00b/undergrads/young people/etc will look at this and think this is the right way to learn this material and some of them will even make an attempt to learn the material from this thing and they'll struggle and fail and be discouraged - i speak from experience! it's not a good thing for the community. sure maybe 1 in a 100 can learn this stuff from just reading a monograph (what these things used to be called...) but that's the exception that proves the rule.
Re: The Elements of Differentiable Programming
#8Would this be useful for general applications, or just numerical ones?
Re: The Elements of Differentiable Programming
#9This is a timely book. Maybe more interesting (at least to me) than the recent results of AI research is the application of techniques used in the field applied elsewhere.
For my work going forward catering to automatic differentiation in the code is a no-brainer.
Re: The Elements of Differentiable Programming
#10Would this be useful for general applications, or just numerical ones?
Your program takes various inputs, does processing and gives you results. To quote the book this applies to programs "including those with control flows and data structures" and are not entirely numerical in the most common sense. Broadly speaking, with AD you can modify the results and the program will spit out the required inputs to get those results.