Differentiable programming for gradient-based machine learning
forums.swift.org
Differentiable programming for gradient-based machine learning
1–10 of 17 posts
Re: Differentiable programming for gradient-based machine learning
#2Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
Re: Differentiable programming for gradient-based machine learning
#3Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
Re: Differentiable programming for gradient-based machine learning
#4Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
In my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor.
Examples of library issues: there is C++ support in tensor flow, but not for training.
Re: Differentiable programming for gradient-based machine learning
#5Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
The differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.
Re: Differentiable programming for gradient-based machine learning
#6Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
In my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor. Examples of library issues: there is C++ support in tensor flow, but not for training.
Agree to that completely. It is also hard for the libraries to spend their energy / time in building for languages that covers only a few percentage of total users. :(
Re: Differentiable programming for gradient-based machine learning
#7Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
In my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor. Examples of library issues: there is C++ support in tensor flow, but not for training.
For anything involving differential equations, Julia is way better than anything else. I also wouldn't be at all surprised if Julia catches up to python for way more machine learning within the next year
Re: Differentiable programming for gradient-based machine learning
#8Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
The differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.
Here's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...
Re: Differentiable programming for gradient-based machine learning
#9Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
Particularly outside deep learning, R remains popular especially for those closer to the stats world.
Matlab and Mathematica both have interesting machine learning features but are probably orders of magnitude lower usage levels compared to Python.
Re: Differentiable programming for gradient-based machine learning
#10Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
Attention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI