Elixir and Rust is a good mix • Hacker News Board

Re: Elixir and Rust is a good mix

#11

Post by OkayPhysicist » Thu, Apr 13, 2023, 7:09 PM UTC

pawelduda wrote: post #4

I used this in past, but there were a lot of people saying that NIFs/ports are dangerous to use. Did any of this change or are there tried and tested practices that can be applied? What I used this for wasn't critical app so I didn't mind it going down. IIRC one threat was Rust sharing memory with BEAM which could exhaust it and cause OOM crash?

NIFs are dangerous, ports are not. Basically a NIF runs in the same memory space as the BEAM, so misbehavior of the NIF can crash the entire application. On the other hand, they have wildly better performance than ports, which have to operate through STDI/O.

The Rust / BEAM memory sharing problem does exist, but it's not nearly as bad as in more traditional C NIFs, because almost all C programs leak memory due to bad manual memory management. Hence all the buzz about Elixir+Rust.

Re: Elixir and Rust is a good mix

#12

Post by throwawaymaths » Thu, Apr 13, 2023, 7:10 PM UTC

waynesonfire wrote: post #5

IO in erlang is slow, is Rustler a good candidate to improve IO intensive applications?

Where did you get this idea? Depending on your task, Erlang can be very fast with IO because it will aggressively use writev/readv instead of write. Obviously, you can do this if you "know about it" in low level langs, but it might be a pain.

Erlang is generally considered to be compute-slow (which is generally the case without dropping to nifs).

Re: Elixir and Rust is a good mix

#13

Post by eikenberry » Thu, Apr 13, 2023, 7:12 PM UTC

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

Could also be nice as it means the Rust parts are smaller and so its slow compiler speed isn't as much of a bother. You get the safety and runtime speed of Rust with the fast iteration cycle of Elixir. Best of both worlds type of thing.

Re: Elixir and Rust is a good mix

#14

Post by zinclozenge » Thu, Apr 13, 2023, 7:12 PM UTC

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

I pretty much agree with this. As much as I liked the core language, I absolutely hated using anything else in the ecosystem, and in my opinion a lot of that was due to lack of static types. Using Ecto as the main way of interacting with the database was a miserable experience. Part of that was due to skill issue I'm sure.

Re: Elixir and Rust is a good mix

#15

Post by toast0 » Thu, Apr 13, 2023, 7:13 PM UTC

waynesonfire wrote: post #5

IO in erlang is slow, is Rustler a good candidate to improve IO intensive applications?

Is IO in Erlang slow? I never got that impression; iolists are a good fit for scatter/gather I/O, which reduces copying, and Erlang supports kqueue, epoll, and I believe similar on Windows. I don't follow Erlang on Linux closely, so I don't know if there's any movement towards io_uring, which does seem promising to help with throughput.

Either way, I wouldn't expect doing I/O in NIFs or ports to substantially improve throughput, unless you're also moving significant processing into that layer as well, or you're going to end up doing substantially the same level of marshaling work as ERTS does, just in a different language. Setting up a different set of kqueue/epoll descriptors sounds like a lot of work for not much gain too, IMHO; again, maybe io_uring would be useful, but I think you'd be better served to bite the bullet and integrate it with ERTS.

Re: Elixir and Rust is a good mix

#16

Post by sergiotapia » Thu, Apr 13, 2023, 7:14 PM UTC

zinclozenge wrote: post #14

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

I pretty much agree with this. As much as I liked the core language, I absolutely hated using anything else in the ecosystem, and in my opinion a lot of that was due to lack of static types. Using Ecto as the main way of interacting with the database was a miserable experience. Part of that was due to skill issue I'm sure.

On the flipside for me Ecto is the perfect balance between ORM and writing raw SQL queries. I've used many different ORMs like Entity Framework, ActiveRecord, Prisma, Ecto - Ecto stands alone.

Re: Elixir and Rust is a good mix

#17

Post by satvikpendem » Thu, Apr 13, 2023, 7:17 PM UTC

eikenberry wrote: post #13

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

Could also be nice as it means the Rust parts are smaller and so its slow compiler speed isn't as much of a bother. You get the safety and runtime speed of Rust with the fast iteration cycle of Elixir. Best of both worlds type of thing.

I often find FFI more annoying than it should be (ie something always breaks somewhere), I try to avoid it where possible.

Re: Elixir and Rust is a good mix

#18

Post by shiryel » Thu, Apr 13, 2023, 7:17 PM UTC

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

As it usually goes in programming, "it depends on your objectives", there are things that are easier accomplished with the ErlangVM than Rust. Also, if you want a language that uses the ErlangVM and has static types, maybe you should take a look at Glean[1].

In my case I prefer to work with Elixir because of the community, as I find easier to work professionally with Elixir than some other mainstream languages, as mostly projects follows the same good practices, use the same tools and have good documentation.

[1] - https://gleam.run/

Re: Elixir and Rust is a good mix

#19

Post by alberth » Thu, Apr 13, 2023, 7:18 PM UTC

Doesn't the use of Rust have to be extremely minimal because ErlangVM has hard time limits on their preemptive scheduler and if your Rust code hasn't finished when preempted, that causes lots of problems.

EDIT: thanks for pointing out where in the article this is talked about.

Re: Elixir and Rust is a good mix

#20

Post by kerkeslager » Thu, Apr 13, 2023, 7:18 PM UTC

satvikpendem wrote: post #7

I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology). Using Rust afterwards was great, plus it was faster than the BEAM. I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language? I've been using Axum and it works pretty well. The only time I had to do FFI with Rust was with Flutter via flutter_rust_bridge,…

> I used to use Elixir, but the lack of static types got to me (especially since I prefer the type-driven development methodology).