Making Julia as Fast as C++ (2019)

A lot of people have focussed on the LSP in their replies when it is was only one of the problems I mentioned.

rust-analyzer is a great LSP and paired with clippy it can teach you the language itself. Also, writing numerical code is extremely easy in Rust. I can write code and just run cargo run to see the output. Julia, on the other hand, forced a REPL-based workflow which never has made sense to me. REPL-based workflow makes sense when you just want to do some script stuff. But when writing a code which will run for a long duration on a HPC? I don't get it. Part of the problem is I'm not "holding it correctly", but again, out of the box experience isn't good. You define a struct and later add or remove a field from it. Often you'll get an error because Revise.jl didn't recompile things. It was a sub-par experience and I was hoping to people would share their dev workflow in more detail

Actually the column-major order of Fortran is more efficient for some linear algebra operations than the order of C, which has been inherited by many modern languages that do not care about high performance in scientific computations.

So I would say that the culprit for interoperability is C and its descendants, not Fortran or Julia. The designers of C and of the languages that have imitated C have not given any thought about which order for multi-dimensional arrays is better, so the users of such languages do not have any right to blame for interoperability other languages that have done the right thing. Even if the Fortran order had not been better, it had already been used for 20 years before C, so there was no reason to choose a different order.

C has chosen to store arrays in the order in which they are typically read by humans when written on paper, but this is a choice like the choice between big-endian and little-endian, where big-endian was how Europeans wrote numbers, but little-endian is more efficient on computers.

An example of why column-major order is preferable, is the matrix-vector product, i.e. the evaluation of a function that maps linear spaces.

The matrix-vector product should not be done as it is typically taught in schools, by scalar products of rows of the matrix with the vector, because this is less efficient, by making more memory accesses. The right way to compute a matrix-vector product is by doing AXPY operations between columns of the matrix and the vector operand (segments of the output of the AXPY operations are held in registers until all partial AXPY operations are accumulated, avoiding memory accesses). In this case, you need to read columns of the input matrix for each AXPY operation, which is much more efficient when the elements of a column are stored compactly in memory, avoiding the need of strided accesses.

The same thing happens for matrix-matrix products, which must not be done in the naive way taught in schools, by scalar products of rows of the first matrix with columns of the second matrix, but it must be done by tensor products of columns of the first matrix with rows of the second matrix.

Julia is reasonably fast. I returned to a language comparison project specific to my math research, to see how I might do better. My agents and I studied the advice in the post and various more recent links from the comments, but we were already mostly on target and nothing left moved the needle.

My work is more combinatorial. Julia does excel at numerical computation. There's a tribal divide in math between people who can't go 30 seconds away from the real or complex numbers, and those whose tolerance is about that long. I try to keep an open mind, but I'm closer to the second camp. Julia is good enough to consider either way.

A development in recent months, AI can assist in general purpose Lean 4 programming, no longer getting confused by the dominant proof-oriented training corpus. If one is a functional programmer who believes that Haskell was on the right track, then Lean is the most interesting language choice for shaping one's thoughts. Benchmarks are inherently misleading if a better language makes it possible to express algorithms out of reach of more primitive languages.

https://github.com/Syzygies/Compare

            C++  100    13.08s  ±0.08s
           Rust   99    13.16s  ±0.02s
          Julia   90    14.54s  ±0.01s
             F#   90    14.54s  ±0.04s
  Kotlin-native   88    14.79s  ±0.01s
         Kotlin   86    15.18s  ±0.01s
          Scala   79    16.50s  ±0.08s
   Scala-native   76    17.14s  ±0.02s
            Nim   65    20.17s  ±0.01s
          Swift   64    20.54s  ±0.04s
          Ocaml   52    25.38s  ±0.04s
           Chez   49    26.64s  ±0.02s
        Haskell   37    34.96s  ±0.06s
           Lean   29    45.39s  ±0.15s

And yet Julia is used for large-scale simulations on giant HPC machines and Rust is not.

Recent versions of Revise let you redefine structs in the REPL.

You are not forced to use the REPL, ever. It’s a fantastic convenience, however.

My dev workflow is to write my code in Neovim, sometimes with a REPL attached to the editor to try out code snippets. I don’t need or use LSPs. I do enjoy the Aerial plugin, which pops up an outline of my code for easy navigation.

So when I was googling it I was seeing a few other corporate activities seemingly coming from the other major contributors of the language outside of Julia hub? It looks like pumas AI has a few of the same people as Julia hub. Or am I misunderstanding the situation?

From a quick Google search it looked kind of like a bunch of MIT staff/professors(?) are getting students to churn out code for a variety of business interests. Just doesn't seem right in the surface and does make me wonder about what other things happen knowing what I know about human behavior.

I am personally not interested that's for sure. Thanks for sharing your experiences though.