Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Managing mutable data in Elixir with Rust

lambdafunctions.com

Post reply on HN
Print view View original

41–50 of 61 posts

Re: Managing mutable data in Elixir with Rust

#41

Post by ilikehurdles »

wredue wrote: post #22
dimitrios1 wrote: post #16

Earlier quoted context omitted.

There isn't some universe where there exists a list of axiomatic, unfalisifable proofs for what or what doesn't constitute the foundations of scalability and robustness, rather, you have a tradition of development practices that have much more often lead you to scalability and robustness than than the alternatives, and Joe Armstrong was someone who trailblazed that tradition in blood sweat and tears so to speak, as w…

The author claimed that immutability is a foundation for scalability and robustness. I reject that claim. In this comment, you simultaneously agree and disagree with me. I don’t give a shit what Joe Armstrong says about immutability because the facts are the facts: 1) immutability cause performance problems 2) immutability significantly limits how you can manage data, which is counter to what computers are meant to d…

You continue to express strong claims on strong claims without backing a single one of them up. And then you drop to mocking others for challenging those claims.

Have you ever heard the saying, “assertions made without evidence can be dismissed without evidence?” My experience differs.

Immutability has been the foundation of many of our large scale programs. It makes safe concurrent programming easier, and languages built around immutable data structures usually optimize memory handling in ways that are not available when simply writing “functional style” code in non-functional languages. ie under the hood they’re using persistent data structures, structural sharing, tail call optimization, etc.

Re: Managing mutable data in Elixir with Rust

#42

Post by evnu »

mgdev wrote: post #25

Rustler is great. Though this gets me thinking about how you can maintain as many Elixir invariants and conventions as possible, even while escaping them under the covers. Being able to call FeGraph.set/2 and have db actually be mutated violates Elixir's common call patterns, even if it's technically allowed. For example: I wonder if it wouldn't be more "erlangy"/"elixiry" to model the mutable ops behind a genserver…

> For example: I wonder if it wouldn't be more "erlangy"/"elixiry" to model the mutable ops behind a genserver that you send messages to.

It depends on the use case. For example, when creating a resource (basically a refcounted datastructure), it might make sense to allow mutable access only through a process as the "owner" of the resource. But if you have only read-only data behind that resource, sharing the resource similar to ETS might be what you want.

Re: Managing mutable data in Elixir with Rust

#43

Post by clarkema »

rkangel wrote: post #36
clarkema wrote: post #27

Earlier quoted context omitted.

That's an interesting point that I should perhaps have covered in the original article. The real code that this is based on is in fact hidden behind a GenServer for this exact reason -- to maintain the expectations of other Elixir code that has to interact with it. The advantage of the escape hatch, as another commenter mentions, is allowing efficient sparse mutations of a large chunk of data, without having to pay a…

Did you consider a port (written in Rust) instead of a NIF? When you're presenting a GenServer like message passing interface a port is a natural fit, with none of the risks related to linking a NIF into the VM itself. (admittedly those risks are much lower with Rust than C)

In our case one of these NIF stores is created per user for a specific task; ironically, with the amount of polish that Rustler puts around NIFs I suspect it would have been more work and more risk to go down the port route and manage everything manually.

Re: Managing mutable data in Elixir with Rust

#44

Post by toast0 »

jerf wrote: post #18
WolfeReader wrote: post #11

Earlier quoted context omitted.

I'm not sure Joe Armstrong would agree with your comment.

Sadly, due to his untimely passing, I don't think Joe Armstrong ever really got a good chance to analyze Rust's borrow checker from his perspective. So I'd be reluctant to be too dogmatic about what he'd think about it. Personally I think that if you can stomach the additional complexity (which is a non-trivial "if", but a doable one), Rust's approach supercedes immutability. Full immutability was an interesting theo…

> Sadly, due to his untimely passing, I don't think Joe Armstrong ever really got a good chance to analyze Rust's borrow checker from his perspective. So I'd be reluctant to be too dogmatic about what he'd think about it.

I agree that Joe was a great explorer of ideas. I'm not sure if he expressed thoughts on Rust, but he would probably look at it again from time to time.

> In my considered opinion, in the end, immutability isn't even important to Erlang. What matters in Erlang is that you can't send references across messages, so there is no way to witness mutation done by another process.

In some ways, you may be right. But you can always mutate the process dictionary if immutable data really bothers you. But even with the process dictionary, it's not possible to construct a self refering datastructure as an Erlang term, which is important! That makes garbage collection simple.

Also, functional programming makes Erlang processes effectively preemptive, when they're built from cooperative user space threads. Tokio tasks can loop and tie up the OS thread it's running on; but an Erlang process will always come to a function call in finite time and can be descheduled at that time, so all runable processes will get a share of cpu.

Edit to add: It's also important to note that Immutability is a property of Erlang (and Elixir), not a property of BEAM. The BEAM vm has opcodes for mutation, and the Erlang compiler will emit them in certain sequences --- if you never use the old value again, it's ok to mutate it rather that create a new modified value; you're most likely to see that with Tuples, IIRC.

Re: Managing mutable data in Elixir with Rust

#45

Post by sodapopcan »

wredue wrote: post #22
dimitrios1 wrote: post #16

Earlier quoted context omitted.

There isn't some universe where there exists a list of axiomatic, unfalisifable proofs for what or what doesn't constitute the foundations of scalability and robustness, rather, you have a tradition of development practices that have much more often lead you to scalability and robustness than than the alternatives, and Joe Armstrong was someone who trailblazed that tradition in blood sweat and tears so to speak, as w…

The author claimed that immutability is a foundation for scalability and robustness. I reject that claim. In this comment, you simultaneously agree and disagree with me. I don’t give a shit what Joe Armstrong says about immutability because the facts are the facts: 1) immutability cause performance problems 2) immutability significantly limits how you can manage data, which is counter to what computers are meant to d…

It doesn't sound like you've used a natively mutable language before. If all you've ever used is immutable.js or something like that then I can understand why you think this way. Otherwise, you've gotta be trolling.

Re: Managing mutable data in Elixir with Rust

#46

Post by filmor »

Miner49er wrote: post #6
elbasti wrote: post #4

Cool writeup. A little ironic, since Erlang's `digraphs` are also mutable!

Erlang's digraphs are stored in an ETS table, so aren't they only mutable in the same way that ETS tables are mutable? I don't normally see people consider (D)ETS tables as mutable, however.

ETS tables are absolutely mutable, they even have specific functions to iterate over them while being mutated (https://www.erlang.org/doc/man/ets#safe_fixtable-2). I use them extensively to share data in a "lock-free" fashion with other processes (a `gen_server` that gets all messages and aggregates data in ETS tables, retrieval via direct reads on a known table name instead of gen_server:call). Mnesia is also (usually) ETS down below.

Re: Managing mutable data in Elixir with Rust

#47

Post by filmor »

cybrox wrote: post #19
impulser_ wrote: post #14

Earlier quoted context omitted.

Rust perfect for this because Rust code can be very reliable which is needed for NIFs in Erlang because a NIF can crash the whole VM. So using C and Zig libraries without fully understanding them can be a death trap while in Rust as long as it doesn't use unsafe code you can feel pretty good about using it.

This has nothing to do with Rust itself. While the compiler does prevent a lot of common pitfalls, you can still write erroneous code with it. It's entirely the rustler project's effort (and goal) to wrap any kind of Rust program so that it will not bring down the BEAM under any circumstance, which they have done a great job achieving.

It's still to a large degree Rust itself. It's the language design which makes it possible to wrap the NIF C API in a safe fashion (e.g. using lifetimes, phantom data, etc.). The only additional safety feature we use is catch_unwind (https://doc.rust-lang.org/std/panic/fn.catch_unwind.html) to prevent panics from unwinding into the BEAM (and killing it).

Re: Managing mutable data in Elixir with Rust

#48

Post by filmor »

unoti wrote: post #17
wredue wrote: post #10

Immutable data is not a “foundation of scalability and robustness”.

> Immutable data is not a “foundation of scalability and robustness”. It may not be the only way to get to scalability and robustness. But it certainly is the cornerstone of how Erlang gets there. 1. First, the way Erlang treats data ensures that every piece of data can be sent over the wire by default. This helps pave the way for another amazing characteristic of Erlang, and that is when you refer to and use an obje…

As an aside: The particular technique used in the OP (resource objects) is actually not transparent. If you send the object handle to another machine, it will have an opaque handle that it can't do anything with apart from storing and sending around. However, if the other machine sends it back (and the resource object hasn't been deallocated in between) it will be the same as the one that you sent.

Re: Managing mutable data in Elixir with Rust

#49

Post by senderista »

mrkeen wrote: post #30
wredue wrote: post #13

Earlier quoted context omitted.

I don’t really care what people making claims say when they make claims without evidence. ”Who” makes a claim has no bearing on its truth. Immutability is a tool, not a rule, and I am free to reject any assertion otherwise when those assertions provide no evidence, or shitty anecdotes. Prove your claims. Certainly, immutability is a foundation for performance problems. Another provable rule in computing is that more…

> Another demonstrable fact is that Haskell based programs have just as many bugs as any other programming language whether you have immutability or not. Therefore, immutability is not a bastion of robustness. Bugs happen when you think you can program something correctly, but can't. If you look at the implementations of transactions in any other language... Oh wait there aren't any! People keep trying to implement i…

I mostly agree with this (having worked on a simple STM for C++), but with a couple caveats:

- Clojure doesn't enforce purity (it can't), but from what I hear its STM seems to work pretty well (aside from some perf issues possibly? haven't used it). That's because "mostly pure" functional programming is encouraged by both the language itself and the culture and ecosystem around it, so uncontrolled side effects are less likely to be a problem.

- I think STM can work "well enough" in unmanaged languages as long as you don't try to boil the ocean and make it perfectly transparent, safe, and fast under all circumstances (Microsoft, IBM, Intel and several others tried for years and failed). That means there will inevitably be huge footguns for non-expert programmers (e.g., any side effect might be invoked every time a transaction is optimistically and transparently retried). These footguns can be mitigated by affordances like commit/abort handlers and infallible transactions.

Re: Managing mutable data in Elixir with Rust

#50

Post by wredue »

WolfeReader wrote: post #21
wredue wrote: post #13

Earlier quoted context omitted.

I don’t really care what people making claims say when they make claims without evidence. ”Who” makes a claim has no bearing on its truth. Immutability is a tool, not a rule, and I am free to reject any assertion otherwise when those assertions provide no evidence, or shitty anecdotes. Prove your claims. Certainly, immutability is a foundation for performance problems. Another provable rule in computing is that more…

You begin with "I don’t really care what people making claims say when they make claims without evidence". May I hold you to your own standards? Because the rest of your post is pretty LOL-worthy in light of your opening sentence.

Sure:

1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed.

Source 2: logic - copying data is slower than not copying it

Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both.

2) immutability requires more code and loc is the best predictor of defects

Clarification: runtime immutability requires more code

Source: it takes more lines of code to return deep copies of objects than to not do that.

Source: https://www.researchgate.net/publication/316922118_An_Invest...

Package densities are the best predictors of defects

3) Haskell projects have as many bugs as any other language

Source: the best evidence we have here is “the large scale study of programming languages on GitHub”, but I suggest that you look deeper here, as the authors qualifications of defects is somewhat questionable (a project that never fixes defects would have low defect rates in this study, it additionally doesn’t properly compare projects sizes and other things). Anyways, in responses that do have better controls in place (and hilariously even in this paper itself, where we see Haskell programs tend of see higher defects as projects go on while c projects tend to have fewer), we see that Haskell does absolutely no better than anything else for bugs and defects.

Post reply on HN