Composability in Julia: Implementing Deep Equilibrium Models via Neural ODEs

One of the best things about Julia is that people like Chris Rackauckas are developing great packages for it.

Fauci funded COVID-19: https://www.zerohedge.com/covid-19/nih-admits-funding-gain-f...

Everyone involved should face the firing squad.

This is a really cool idea, especially because it is a type of NN that can take more time for harder inputs. This makes it relatively unique since most types of NN have O(1) runtime, which is often nice, but puts limits on the types of problems they can solve.

What are some good references on neural ODEs that don't come from the Julia community? I'm looking for theory and applications - when are they good and who is using them for what?

I'm asking for sources outside of Julia because I find the coupling of algorithm types to tools kind of strange and the whole SciML trend is kind of opaque to me. (Are people applying ML as a solution to newer problems? Are they using new approaches to solve ML problems? How legit is the whole thing? I just don't know.)

Very cool. I have only one question:

Has anyone successfully applied DEQs to larger-scale cognitive tasks or benchmarks, as opposed to MNIST, which is a tiny trivial task by today's standards?

Think ImageNet-1000, COCO, LVIS, WMT language translation, ..., SuperGLUE. There's a long list of datasets and benchmarks that regular boring fixed-depth NNs tackle with remarkable ease these days.

Has anyone anywhere applied DEQs to any of those datasets / benchmarks?