Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LFortran: Modern interactive LLVM-based Fortran compiler

lfortran.org

Post reply on HN
Print view View original

61–70 of 73 posts

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#61

Post by pjmlp »

MaxBarraclough wrote: post #51
yodelshady wrote: post #48

Modern fortran is surpisingly pleasant to write. The last time optimisation came up, I tried the author's problem - multiplying two 4096x4096 matrices - in numpy, FORTRAN and Rust, on a standard laptop. Supposedly it took 9 hours in naive Python (I didn't try). Results were: Python3 (numpy 1.18.4) 1.3s FORTRAN (gfortran 9.3.0) 6.0s Rust (1.43.1, debug) >60s Rust (1.43.1, release) 4.0s What surprised me is that python…

So to be clear, numpy gives you a highly optimised matrix multiplication algorithm, and your FORTRAN solution was just something you threw together yourself, right? Little surprise that numpy roundly outperforms it, especially if it's using a different algorithm, such as Strassen rather than naive matrix multiplication. [0] > every single way I tried had "gotchas" and utterly astonishing behaviour in it As someone wh…

Actually the table is more like

    ┌────────────────┬─────────────────┬──────┐
    │ Language       │     Detail      │ Time │
    ├────────────────┼─────────────────┼──────┤
    │ Python3 and C  │ numpy 1.18.4    │ 1.3s │
    │ FORTRAN        │ gfortran 9.3.0  │ 6.0s │
    │ Rust           │ 1.43.1, debug   │ >60s │
    │ Rust           │ 1.43.1, release │ 4.0s │
    └────────────────┴─────────────────┴──────┘

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#62

Post by MaxBarraclough »

pjmlp wrote: post #61
MaxBarraclough wrote: post #51

Earlier quoted context omitted.

So to be clear, numpy gives you a highly optimised matrix multiplication algorithm, and your FORTRAN solution was just something you threw together yourself, right? Little surprise that numpy roundly outperforms it, especially if it's using a different algorithm, such as Strassen rather than naive matrix multiplication. [0] > every single way I tried had "gotchas" and utterly astonishing behaviour in it As someone wh…

Actually the table is more like ┌────────────────┬─────────────────┬──────┐ │ Language │ Detail │ Time │ ├────────────────┼─────────────────┼──────┤ │ Python3 and C │ numpy 1.18.4 │ 1.3s │ │ FORTRAN │ gfortran 9.3.0 │ 6.0s │ │ Rust │ 1.43.1, debug │ >60s │ │ Rust │ 1.43.1, release │ 4.0s │ └────────────────┴─────────────────┴──────┘

I could have put 'host language', but I figured it was clear from context.

I'd assumed numpy used FORTRAN, but you're right, its heavy-lifting is done in C.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#63

Post by celrod »

waltpad wrote: post #45
celrod wrote: post #13

Earlier quoted context omitted.

I think Eigen's template strategy is probably hard to beat from the perspective of combining operations. How well Fortran does is probably mostly up to the compiler implementation. In some of my benchmarks, gfortran sometimes seems to fuse the operations and end up much slower than if it has performed them separately in succession, but I wouldn't be surprised if compiling with `-fexternal-blas` (and linking MKL) woul…

If you have a look at the different fortran implementations of BLAS gemm (matrix multiplication), you'll see that the transposed matrix cases are treated specifically. In fact, IIRC, the gemm function has flags to indicate for each matrix if it is transposed.

That's what I did when calling OpenBLAS and MKL, but I confess I don't know the internal details of a non-inlined `matmul` call in gfortran when you don't use `-fexternal-blas`.

Just writing three loops and letting the compiler optimize it was much faster for `A * B'`, so it must be a pretty naive implementation getting called.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#64

Post by CyberDildonics »

enriquto wrote: post #43
CyberDildonics wrote: post #37

Earlier quoted context omitted.

That's ridiculous, you can make a C++ class and overload operator() like Eigen to easily make a multi-dimensional array.

By that reasoning, all turing-complete languages support multi-dimensional arrays.

I don't think every turing complete language lets you create an interface and memory layout that would be identical to fortran.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#65

Post by certik »

celrod wrote: post #36
certik wrote: post #34

Earlier quoted context omitted.

Thanks for the benchmarks. Do you see some intrinsic reason why a Fortran compiler couldn't do these optimizations? I would think it would know all the information so it would be the ideal place to optimize it.

There is no reason it could not. Those optimizations just have to be implemented. Flang (the one merged into LLVM) is using MLIR, which has all the required code-gen abilities. That just leaves the cost modeling / deciding which optimizations/transformations to apply.

Yes, once MLIR is more mature, we plan to have a backend for LFortran to also use MLIR.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#66

Post by CyberDildonics »

baggy_trough wrote: post #39
CyberDildonics wrote: post #37

Earlier quoted context omitted.

That's ridiculous, you can make a C++ class and overload operator() like Eigen to easily make a multi-dimensional array.

"natively"

What is different to fortran's multi-dimensional arrays? The interface can be almost identical and the memory layout in C++ can be whatever you want it to be.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#67

Post by dm319 »

ironmagma wrote: post #27
dm319 wrote: post #26

Earlier quoted context omitted.

fortran - one of only a handful of languages that natively supports multidimensional arrays, maybe the only low level one?

C kind of does, right? A multidimensional array in C isn't just an array of pointers.

Is that more of a jagged / ragged array, or array of arrays? Sometimes these are called multidimensional arrays, but these are not the same thing. Dimensions and lengths of the dimensions should be a property of the multidimensional object.

[1] https://stackoverflow.com/questions/597720/what-are-the-diff...

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#68

Post by ironmagma »

dm319 wrote: post #67
ironmagma wrote: post #27

Earlier quoted context omitted.

C kind of does, right? A multidimensional array in C isn't just an array of pointers.

Is that more of a jagged / ragged array, or array of arrays? Sometimes these are called multidimensional arrays, but these are not the same thing. Dimensions and lengths of the dimensions should be a property of the multidimensional object. [1] https://stackoverflow.com/questions/597720/what-are-the-diff...

I was talking about

  int a[2][3];
in C. Is that jagged? I wouldn’t have thought so, since AFAIK you could only store 6 elements in that and no fewer. Interestingly you cannot take the value of &&a[0][0], assign it to an

  int **b;
pointer b, and then access it via b[0][0] because it’s stored as a contiguous block of integers rather than an array of pointers that point to arrays as you might expect.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#69

Post by celrod »

certik wrote: post #65
celrod wrote: post #36

Earlier quoted context omitted.

There is no reason it could not. Those optimizations just have to be implemented. Flang (the one merged into LLVM) is using MLIR, which has all the required code-gen abilities. That just leaves the cost modeling / deciding which optimizations/transformations to apply.

Yes, once MLIR is more mature, we plan to have a backend for LFortran to also use MLIR.

For BLAS in particular, this paper can give you an idea of some of MLIR's capabilities: https://arxiv.org/pdf/2003.00532.pdf (But maybe you already know them better than I do.)

LoopVectorization can't do many of these yet, so its performance will fall off a cliff shortly after the largest size on the plots (and at much smaller sizes for CPUs with a smaller L2 cache). I had to add code to perform packing/tiling in my actual matmul code on top of what it did. So that MLIR can generate that sort of code already looks promising.

Still, the work of telling it what to do isn't easy.

I'm not involved in any of those projects, so everything I say here is pure speculation. But I imagine pragmas and the like would be important whenever the compiler doesn't know the sizes at compile time. Otherwise, you probably don't want it to generate massive amounts of code through multiple extra blocking loops, massive unrolling in a main kernel, and multiple clean up kernels for every random loop nest.

Re: LFortran: Modern interactive LLVM-based Fortran compiler

#70

Post by [deleted] »

yodelshady wrote: post #48

Modern fortran is surpisingly pleasant to write. The last time optimisation came up, I tried the author's problem - multiplying two 4096x4096 matrices - in numpy, FORTRAN and Rust, on a standard laptop. Supposedly it took 9 hours in naive Python (I didn't try). Results were: Python3 (numpy 1.18.4) 1.3s FORTRAN (gfortran 9.3.0) 6.0s Rust (1.43.1, debug) >60s Rust (1.43.1, release) 4.0s What surprised me is that python…

[deleted]
Post reply on HN