Differentiable Fortran with LFortran and Enzyme
docs.pasteurlabs.ai
Differentiable Fortran with LFortran and Enzyme
1–10 of 20 posts
Re: Differentiable Fortran with LFortran and Enzyme
#2Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :)
I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare say so.
Would be awesome to see this applied to a real scientific codebase, and I hope that the demo is enough to convince people that it’s worth trying.
Re: Differentiable Fortran with LFortran and Enzyme
#3Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ?
A shared layout and a shared calling convention would be very nice.
Sorry about my naive question. Haven't touched Fortran directly in three decades I think.
EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is very nice.
Re: Differentiable Fortran with LFortran and Enzyme
#4Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…
Not naive! The answer is yes and no.
Array layout: yes, for the simple cases this post relies on. In the demo the work arrays are literally calloc'd in a C wrapper and handed straight to the Fortran routine, and it just works. (Column-major vs. C's row-major is still on you to keep straight, of course.) This wouldn't be so easy for fancier Fortran array features — allocatables, assumed-shape (:) dummies, pointer arrays — which in general get a descriptor struct (bounds, strides, etc.) rather than a bare pointer. Flang uses those descriptors much more aggressively, but I don't know how that would manifest at the C-Fortran bridge. Would be interesting to repeat the experiment with Flang (if at all possible) and comparing the pain.
Calling convention: there are well established contracts, much older than the layout story. Fortran passes everything by reference, so a double precision :: x argument is an x* at the ABI level — which is why every argument in the C wrapper is a pointer (&n_, &k0_, ...). Combine that with C-compatible name mangling (a trailing underscore historically, controllable via LFortran/bind(C)) and Bob's your uncle. Nothing here is new, the pipeline is really just leaning on that decades-old interop contract and then letting Enzyme differentiate across the boundary.
Re: Differentiable Fortran with LFortran and Enzyme
#5Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…
Lots of scientific code in Fortran has sparse arrays, so a NxN array that only has values on 5 diagonals will store that as 5xN array to save memory allowing you to run a larger problem.
Re: Differentiable Fortran with LFortran and Enzyme
#6Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…
[dead]
Re: Differentiable Fortran with LFortran and Enzyme
#7Earlier quoted context omitted.
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…
Lots of scientific code in Fortran has sparse arrays, so a NxN array that only has values on 5 diagonals will store that as 5xN array to save memory allowing you to run a larger problem.
That's a very orthogonal issue.
Sparse arrays are supported on C libraries too. I have done my time with CSC and CSR even inside Python that called out to C libraries.
Re: Differentiable Fortran with LFortran and Enzyme
#8Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ? A shared layout and a shared calling convention would be very nice. Sorry about my naive question. Haven't touched Fortran directly in three decades I think. EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is v…
[deleted]
Re: Differentiable Fortran with LFortran and Enzyme
#9Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
When you say you 'wrote this up', you mean you had an AI write (at least) chunks of it.
Re: Differentiable Fortran with LFortran and Enzyme
#10Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :) I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare…
When you say you 'wrote this up', you mean you had an AI write (at least) chunks of it.
Indeed, just like I let my compiler write (at least) chunks of my AD logic. Not great when the tool becomes a leaky abstraction, but overall net positive don't you think?