For someone who does not know much about NIFs, will it be a lot slower if the Rust service is coded as a command line application and then invoked using System.cmd? Is that a good alternative?
Using Rust with Elixir for code reuse and performance
11–20 of 20 posts
Re: Using Rust with Elixir for code reuse and performance
#12> Unsurprisingly, Rust outperforms Elixir by 2 orders of magnitude.
That seems very surprising to me. Is this a common result? I've heard that the BEAM is not the best for "number crunching" code, is this one of those scenarios? Is this really just the raw performance of the languages, or a difference in algorithms? There's also no mention of the version of Elixir and the OTP. Did the JIT change anything?
Re: Using Rust with Elixir for code reuse and performance
#13One thing to keep in mind is that even the simplest Rust NIF will significantly slow your builds and increase your repo/artifact sizes.
Author here. This is a good point. In this case we found it does slow down clean builds somewhat since we're pulling down + compiling Rust dependencies now as well as Elixir. While actually editing Elixir code I haven't experienced any noticeable slowdown for incremental builds - the Rust code is mostly static.
Re: Using Rust with Elixir for code reuse and performance
#14One thing to keep in mind is that even the simplest Rust NIF will significantly slow your builds and increase your repo/artifact sizes.
It's one reason I prefer lighter languages for NIF's. My personal preference is to use Nim and Nimler [1]. It generally compiles quicker than Rust while providing most of the same benefits. To be fair compiling a small Rust library doesn't take too much time. Especially compared to the performance numbers from those Rust NIFs!
Re: Using Rust with Elixir for code reuse and performance
#15For someone who does not know much about NIFs, will it be a lot slower if the Rust service is coded as a command line application and then invoked using System.cmd? Is that a good alternative?
It will be faaaar slower. If it is a good alternative depends on your need. Benchmark your situation
Re: Using Rust with Elixir for code reuse and performance
#16> Unsurprisingly, Rust outperforms Elixir by 2 orders of magnitude. That seems very surprising to me. Is this a common result? I've heard that the BEAM is not the best for "number crunching" code, is this one of those scenarios? Is this really just the raw performance of the languages, or a difference in algorithms? There's also no mention of the version of Elixir and the OTP. Did the JIT change anything?
Heavy duty string bashing is a pessimal case for BEAM, yes. The two core data structures you could use for this, the binaries or the 'strings' (which are linked lists of characters), each have their own problems with this sort of algorithm. Erlang was designed to fling chunks around, maybe take a header apart before deciding where to fling a chunk around, not to grovel over every byte of some fairly-large string for detailed parsing.
Based on performance numbers I've seen, I'd expect a one-order-of-magnitude difference to be a more common case in general.
Re: Using Rust with Elixir for code reuse and performance
#17One thing to keep in mind is that even the simplest Rust NIF will significantly slow your builds and increase your repo/artifact sizes.
It’s also important to note that a NIF can bring down your whole BEAM.
Re: Using Rust with Elixir for code reuse and performance
#18> Unsurprisingly, Rust outperforms Elixir by 2 orders of magnitude. That seems very surprising to me. Is this a common result? I've heard that the BEAM is not the best for "number crunching" code, is this one of those scenarios? Is this really just the raw performance of the languages, or a difference in algorithms? There's also no mention of the version of Elixir and the OTP. Did the JIT change anything?
For me, Elixir with Rust NIFs seems like a match made in heaven.
The BEAM VM is amazing at many things but it is still a VM and while the BeamAsm Just In Time compiler added in 2020 can offer performance improvement gains of 130%, there are areas where the BEAM still won't outperform native code.
Erlang excels at network programming and binary data processing. Computation intensive math and heavy string processing are both good NIF use cases.
An article on string performance [1],pre-JIT, describes initial Elixir benchmark of 140s vs C's 3.74s. They managed to make a number of tradeoffs and get the Elixir benchmark improved to 13s. Part of how they did that was to not use unicode (IO.binstream instead of IO.stream), that alone gained them ~4s.
[1] https://blog.jola.dev/elixir-string-processing-optimization
Re: Using Rust with Elixir for code reuse and performance
#19For someone who does not know much about NIFs, will it be a lot slower if the Rust service is coded as a command line application and then invoked using System.cmd? Is that a good alternative?
It will be faaaar slower. If it is a good alternative depends on your need. Benchmark your situation
Thank you.