Earlier quoted context omitted.
in what is your phd?
Operations research / transportation
cool!
Composability in Julia: Implementing Deep Equilibrium Models via Neural ODEs
51–53 of 53 posts
Re: Composability in Julia: Implementing Deep Equilibrium Models via Neural ODEs
#52Earlier quoted context omitted.
Yes, that's mostly right (instead of PETSc put FATODE since PETSc TS Adjoint was only published in 2019, but it's based heavily on FATODE's techniques https://epubs.siam.org/doi/10.1137/130912335?mobileUi=0 ). And that's why the Julia tools were so ready for it: we already had adjoint sensitivity analysis (implemented for parameter estimation in systems pharmacology), so neural ODEs were a freebee. Similar to DEQs us…
Ha, yeah I knew that analysis of how to do second order adjoints fast :), in what is basically documentation no less. I figured someone would probably do a ML paper on that eventually. I have had a hard time judging, what is non-trivial in the past, aswell. The next thing that they will probably discover is Jet spaces and Hopf algebras. I'm also slightly salty because a recent paper ( https://arxiv.org/pdf/2011.03902…
Just wondering, are the reasons you don't like Julia existential (eg, it's a dynamic language without static types), or stuff that we can fix? Also, I wouldn't be surprised if at some point in the next decade, there start to be good C/C++ libraries written in Julia.
Re: Composability in Julia: Implementing Deep Equilibrium Models via Neural ODEs
#53Earlier quoted context omitted.
Ha, yeah I knew that analysis of how to do second order adjoints fast :), in what is basically documentation no less. I figured someone would probably do a ML paper on that eventually. I have had a hard time judging, what is non-trivial in the past, aswell. The next thing that they will probably discover is Jet spaces and Hopf algebras. I'm also slightly salty because a recent paper ( https://arxiv.org/pdf/2011.03902…
Just wondering, are the reasons you don't like Julia existential (eg, it's a dynamic language without static types), or stuff that we can fix? Also, I wouldn't be surprised if at some point in the next decade, there start to be good C/C++ libraries written in Julia.
I would say I'm relatively pragmatic. My main use case would involve DifferentialEquations.jl modelling either certain quantum processes or model equations of biological neurons. I have a rough idea how that would work and what kind of abstractions I would like to implement. I mainly have not been able to get a good developer experience with it, so far. Things tend to either work or fail rather drastically (spectacular long back traces). But it clearly is very useful, so I will probably bite the bullet and try to use it again.