Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Differentiable Programming System to Bridge ML and Scientific Computing

arxiv.org

Post reply on HN
Print view View original

11–20 of 75 posts

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#11

Post by ChrisRackauckas »

endoftime5 wrote: post #8
6thaccount2 wrote: post #3

Earlier quoted context omitted.

Is the performance equivalent in your scenario?

We have taken a slight performance hit, but the code is cleaner and much easier to extend.

If there's any benchmarks on the DiffEq side, feel free to reach out to us with them and we can take a crack at optimizing it. If it's something that's allowed to be shared, we'd love to add it to our DiffEqBenchmarks.jl repository to track the performance over time and understand the performance characteristics of your domain.

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#12

Post by jefft255 »

akhilcacharya wrote: post #9
melling wrote: post #6

Earlier quoted context omitted.

Does Swift have an equivalent to Python‘s numpy? It seems to lack a nice way of doing vector and matrix operations.

They built in Python interop so that you can use numpy in TFSwift

Doesn’t that introduce serious overhead compared to first class native arrays as in Julia or FORTRAN?

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#13

Post by 6thaccount2 »

ChrisRackauckas wrote: post #11
endoftime5 wrote: post #8

Earlier quoted context omitted.

We have taken a slight performance hit, but the code is cleaner and much easier to extend.

If there's any benchmarks on the DiffEq side, feel free to reach out to us with them and we can take a crack at optimizing it. If it's something that's allowed to be shared, we'd love to add it to our DiffEqBenchmarks.jl repository to track the performance over time and understand the performance characteristics of your domain.

Is there a general way to do this with many of the libraries in Julia?

Any chance ya'll could eventually use a test model and script from me in a benchmark test suite.

Ex: I have a large synthetic test model from a university and plan on writing some code that uses a bunch of the sparse matrix and linear solve functionality of the language. This would be on a sparse ~70k row/column square matrix.

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#15

Post by ChrisRackauckas »

6thaccount2 wrote: post #13
ChrisRackauckas wrote: post #11

Earlier quoted context omitted.

If there's any benchmarks on the DiffEq side, feel free to reach out to us with them and we can take a crack at optimizing it. If it's something that's allowed to be shared, we'd love to add it to our DiffEqBenchmarks.jl repository to track the performance over time and understand the performance characteristics of your domain.

Is there a general way to do this with many of the libraries in Julia? Any chance ya'll could eventually use a test model and script from me in a benchmark test suite. Ex: I have a large synthetic test model from a university and plan on writing some code that uses a bunch of the sparse matrix and linear solve functionality of the language. This would be on a sparse ~70k row/column square matrix.

At least with DiffEq we keep track of a lot of different benchmarks here: https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl and it's always growing. Some of the best benchmarks have come from users who want us to track performance on their problem. Some other libraries have repos around, but not as formalized. We are getting things automated so that we can start helping others.

Your model sounds nice because it can use our automatic sparsity detection and matrix coloring. I'd love to give it a try.

https://github.com/JuliaDiffEq/SparseDiffTools.jl#automated-...

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#16

Post by phonebucket »

KenoFischer wrote: post #4

Happy that this paper finally made it to arxiv. The biggest reason for writing it was to try and showcase some of the breadth of applications we see for really high quality first class AD support at the language level. There are several communities that need this technology, so it makes sense to try and build one system that can address all of them and share tricks. I'm also hoping this gives people a sense of why ou…

Congratulations on some great work. I like the diversity shown in the examples. In particular, it's nice that you threw those working with stochastic processes a bone.

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#17

Post by jjoonathan »

akhilcacharya wrote: post #9
melling wrote: post #6

Earlier quoted context omitted.

Does Swift have an equivalent to Python‘s numpy? It seems to lack a nice way of doing vector and matrix operations.

They built in Python interop so that you can use numpy in TFSwift

Bridges always have trolls under them.

If you don't see the troll, that doesn't mean it isn't there, it's just waiting for you to cross.

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#19

Post by superlopuh »

jefft255 wrote: post #12
akhilcacharya wrote: post #9

Earlier quoted context omitted.

They built in Python interop so that you can use numpy in TFSwift

Doesn’t that introduce serious overhead compared to first class native arrays as in Julia or FORTRAN?

My impression is that this is to bootstrap adoption, there is a Swift-native TensorFlow package being developed also.

Re: A Differentiable Programming System to Bridge ML and Scientific Computing

#20

Post by tel »

akhilcacharya wrote: post #9
melling wrote: post #6

Earlier quoted context omitted.

Does Swift have an equivalent to Python‘s numpy? It seems to lack a nice way of doing vector and matrix operations.

They built in Python interop so that you can use numpy in TFSwift

Bridging into NumPy isn’t going to be useful unless it covers some AD system too (XLA?).
Post reply on HN