Live data from Hacker News

Differentiable Fortran with LFortran and Enzyme

docs.pasteurlabs.ai

11–20 of 20 posts

Re: Differentiable Fortran with LFortran and Enzyme

#11

Earlier quoted context omitted.

When you say you 'wrote this up', you mean you had an AI write (at least) chunks of it.

Indeed, just like I let my compiler write (at least) chunks of my AD logic. Not great when the tool becomes a leaky abstraction, but overall net positive don't you think?

Disagree; it irks me to read AI slop, and writing is meant to be persuasive and engaging to human readers.

Re: Differentiable Fortran with LFortran and Enzyme

#12

Earlier quoted context omitted.

Indeed, just like I let my compiler write (at least) chunks of my AD logic. Not great when the tool becomes a leaky abstraction, but overall net positive don't you think?

Disagree; it irks me to read AI slop, and writing is meant to be persuasive and engaging to human readers.

Happy to take the blame for the lack of persuasian and engagement then :) Thanks for the feedback, although I’d like to believe there’s a way to have that cake and eat it too.

Re: Differentiable Fortran with LFortran and Enzyme

#13
post #3

Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…

Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…

I would also like to know this. Fortran itself is column-major, so I would guess the internal layout isn't same for multi-dimensional arrays when compared to row-major C? I'm not sure how LFortran represents arrays internally though.

Re: Differentiable Fortran with LFortran and Enzyme

#14
post #3

Earlier quoted context omitted.

Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…

I would also like to know this. Fortran itself is column-major, so I would guess the internal layout isn't same for multi-dimensional arrays when compared to row-major C? I'm not sure how LFortran represents arrays internally though.

LFortran internally uses column-major, so interchanging data with C should be done carefully for multi-dimensional arrays. If row-major representation is highly needed feature, We can introduce a flag to do that. I'm not totally sure about that but it's doable under some conditions for sure.

Re: Differentiable Fortran with LFortran and Enzyme

#15

Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…

Very interesting stuff. How would I get GPU offload working? I have a rather complex scientific code I'm working on with JAX. Most of it can be expressed well with JAX's programming model, but the last 10% really sucks. It's still worth it so I don't have to mess around with offload onto whatever XPU flavor of the week. But going to C++ would really make my life easier, as long as I could use e.g. Kokkos.

Re: Differentiable Fortran with LFortran and Enzyme

#16

Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…

Very interesting stuff. How would I get GPU offload working? I have a rather complex scientific code I'm working on with JAX. Most of it can be expressed well with JAX's programming model, but the last 10% really sucks. It's still worth it so I don't have to mess around with offload onto whatever XPU flavor of the week. But going to C++ would really make my life easier, as long as I could use e.g. Kokkos.

No idea lol. I assume it’s possible since both Enzyme and GPU programming are pervasive in Julia. Let us know if you end up trying.

Re: Differentiable Fortran with LFortran and Enzyme

#17
Really nice work. I love Enzyme, and used it in my project about differentiable atomic descriptors. Idea was that I can quickly gobble up existing C++ and fortran codes alike for atomic descriptors and create a encompassing library what differentiate against hyper-params as well! But at time Enzyme was very early ~0.0.50 version or so. In our observations also Enzyme was fast enough that performance wise it matched the analytical gradients (when embedded inside entire pipeline) ![libdescriptor](https://libdescriptor.readthedocs.io/en/latest/_images/E_F_C...) .

Re: Differentiable Fortran with LFortran and Enzyme

#18

Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…

Very interesting stuff. How would I get GPU offload working? I have a rather complex scientific code I'm working on with JAX. Most of it can be expressed well with JAX's programming model, but the last 10% really sucks. It's still worth it so I don't have to mess around with offload onto whatever XPU flavor of the week. But going to C++ would really make my life easier, as long as I could use e.g. Kokkos.

You can write the cuda kernels and directly differentiate them using Enzyme. Last time I checked KOKKOS was still WIP: https://github.com/EnzymeAD/Enzyme/issues?q=kokkos

Re: Differentiable Fortran with LFortran and Enzyme

#19

Earlier quoted context omitted.

I would also like to know this. Fortran itself is column-major, so I would guess the internal layout isn't same for multi-dimensional arrays when compared to row-major C? I'm not sure how LFortran represents arrays internally though.

LFortran internally uses column-major, so interchanging data with C should be done carefully for multi-dimensional arrays. If row-major representation is highly needed feature, We can introduce a flag to do that. I'm not totally sure about that but it's doable under some conditions for sure.

We can internally do both row-major and column-major arrays. For Fortran we enable the column-major flag by default, but the door is open to have both at the same time.

Re: Differentiable Fortran with LFortran and Enzyme

#20
Great work, I am glad LFortran worked well for this work. We are very close to beta quality now, many codes just work, pretty much all Fortran features are implemented now, but some codes still don't compile due to various small bugs, so we've been fixing them all in the last couple months.
Post reply on HN