Live data from Hacker News

Managing mutable data in Elixir with Rust

lambdafunctions.com

51–60 of 61 posts

Re: Managing mutable data in Elixir with Rust

#51
post #32

Earlier quoted context omitted.

Explorer does actually use Rust (and polars) for a lot of its work -- its one on the libraries I looked at while figuring out my memory management issues.

But would it benefit from mutating the value of one reference? At the moment it does not do that, right?

Probably not ideal since Polars builds on Apache Arrow and that tends to want to treat the structure as immutable if I recall correctly.

Re: Managing mutable data in Elixir with Rust

#52
post #50

Earlier quoted context omitted.

You begin with "I don’t really care what people making claims say when they make claims without evidence". May I hold you to your own standards? Because the rest of your post is pretty LOL-worthy in light of your opening sentence.

Sure: 1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed. Source 2: logic - copying data is slower than not copying it Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both. 2) immutability requires more code and loc is the best predictor of def…

When you say "the large scale study of programming languages on GitHub", are you referring to this? https://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf

"Table 7: Functional languages have a smaller relationship to defects than other language classes where as procedural languages are either greater than average or similar to the average."

"The data indicates functional languages are better than procedural languages; it suggests that strong typing is better than weak typing; that static typing is better than dynamic; and that managed memory usage is better than un-managed."

You got owned by your own source.

As for your un-sourced claim that "copying data is slower than not copying it", I'd suggest learning how immutable-first languages practice data sharing between objects to minimize the amount of copying needed.

Re: Managing mutable data in Elixir with Rust

#53
post #50

Earlier quoted context omitted.

You begin with "I don’t really care what people making claims say when they make claims without evidence". May I hold you to your own standards? Because the rest of your post is pretty LOL-worthy in light of your opening sentence.

Sure: 1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed. Source 2: logic - copying data is slower than not copying it Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both. 2) immutability requires more code and loc is the best predictor of def…

> Source 2: logic - copying data is slower than not copying it

> Source: it takes more lines of code to return deep copies of objects than to not do that.

Defensive copying and deep copying is not a thing you have to do in immutable languages. Even under the covers, it's not happening the way you seem to think it is. If I had a large immutable map in use by some other process, and needed a version of it with an element changed or added, why would I deep copy it when I can just point to that same map instance, and add a pointer to the key-value pair I want to substitute [1]? I think this is a common reservation people have about immutable programming because they come into it with a OO mindset. At least, I know I did.

In a really simplified example, a = (1, 2, 3, ..., 100) and b = (2, 3, ..., 100) are not allocated as two full lists in memory space. a contains 1 followed by a pointer to b. Because you have guarantees that b will never change, the single instance of b can be recycled in other data structures (or passed to many other functions and threads) and you avoid the complexity of managing race conditions, mutexes, semaphores, which are a significant source of bugs in other languages.

See [2] for a more realistic implementation.

1. https://en.wikipedia.org/wiki/Radix_tree

2. https://en.wikipedia.org/wiki/Hash_array_mapped_trie

Re: Managing mutable data in Elixir with Rust

#54
post #50

Earlier quoted context omitted.

You begin with "I don’t really care what people making claims say when they make claims without evidence". May I hold you to your own standards? Because the rest of your post is pretty LOL-worthy in light of your opening sentence.

Sure: 1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed. Source 2: logic - copying data is slower than not copying it Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both. 2) immutability requires more code and loc is the best predictor of def…

I wanted to write that making defensive copies is something you need to do in mutable situations to preserve safety, (not in immutable situations!), but it looks like enough commenters have hit that point.

So lets disabuse your mistrust of immutability in another domain!

Here is some typical "go fast and mutable!" nonsense code:

    int foo(int i, int j) {
      while (i 
Let's compile it with https://godbolt.org/, turn on some optimisations and inspect the IR (-O2 -emit-llvm). Copying out the part that corresponds to the while loop:

  4:
    %5 = sub i32 9, %0, !dbg !20
    %6 = add nsw i32 %0, 1, !dbg !20
    %7 = mul i32 %5, %6, !dbg !20
    %8 = zext i32 %5 to i33, !dbg !20
    %9 = sub i32 8, %0, !dbg !20
    %10 = zext i32 %9 to i33, !dbg !20
    %11 = mul i33 %8, %10, !dbg !20
    %12 = lshr i33 %11, 1, !dbg !20
    %13 = trunc i33 %12 to i32, !dbg !20
    tail call void @llvm.dbg.value(metadata i32 poison, metadata !17, metadata !DIExpression()), !dbg !18
    tail call void @llvm.dbg.value(metadata i32 poison, metadata !16, metadata !DIExpression()), !dbg !18
    %14 = add i32 %1, %0, !dbg !20
    %15 = add i32 %14, %7, !dbg !20
    %16 = add i32 %15, %13, !dbg !20
    br label %17, !dbg !21

  17:
    %18 = phi i32 [ %1, %2 ], [ %16, %4 ]
Well, would you look at that! Clang decided (even in this hot loop) never to re-assign any of the left-hand-sides, even though my instructions were just: "mutate j in-place. mutate i in-place."

Re: Managing mutable data in Elixir with Rust

#55

Earlier quoted context omitted.

I'm not sure Joe Armstrong would agree with your comment.

After a certain scale, it actually does start to slow you down https://discord.com/blog/using-rust-to-scale-elixir-for-11-m...

That was an excellent write up. And great case study in how to iteratively approach a performance problem. First address the algorithmic issues. Then once the algorithm is no longer the bottleneck, look into dropping into a higher performance, lower overhead language like Rust. Jumping directly to Rust with the original algorithm, would not have helped much. And in the general case, getting the algorithm right might make dropping down into Rust unnecessary.

Very nice work.

Re: Managing mutable data in Elixir with Rust

#56

Earlier quoted context omitted.

There isn't some universe where there exists a list of axiomatic, unfalisifable proofs for what or what doesn't constitute the foundations of scalability and robustness, rather, you have a tradition of development practices that have much more often lead you to scalability and robustness than than the alternatives, and Joe Armstrong was someone who trailblazed that tradition in blood sweat and tears so to speak, as w…

> you have a tradition of development practices that have much more often lead you to scalability and robustness than than the alternatives I'm not GP, but these traditions are usually not backed by any evidence but by cargo-culting and cult-of-personalities. Not to mention people who over-hype their favourite technologies to high heavens, poisoning the well for everyone else (no, most telecom industry doesn't run Er…

Do they need to be backed by evidence to be correct? What if they are just right? Do other companies not running Erlang mean its wrong? Because that's a bold argument that could be restated to disprove many things -- X technology claims to do Y well and has a track record of doing it really well, but Z companies don't use X technology, therefore...[insert whatever].

I don't believe in blindly believing things without evidence either, especially if I have never encountered them before, but I also don't believe in blindly dismissing experience of world renowned experts in their field because they didn't provide me a point by point prooftext of every claim they made (Again we aren't sitting here discussing a dissertation or mathematical proof). Their experience and what they've provided to the world is the evidence. We took this 19th century german ultra-materialist philosophy too far here in the west, and that's what gave us post modernism/poststructuralism with its disastrous consequences, but it still seems like we haven't learned anything from that.

The ancients had it right that theres different types of knowledge, and different ways of knowing things (and knowing them to be true, at least as far as it mattered). We here in the modern era with the most unfettered access to information have quite possibly the narrowest definition, ironically.

Re: Managing mutable data in Elixir with Rust

#57
post #50

Earlier quoted context omitted.

Sure: 1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed. Source 2: logic - copying data is slower than not copying it Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both. 2) immutability requires more code and loc is the best predictor of def…

> Source 2: logic - copying data is slower than not copying it > Source: it takes more lines of code to return deep copies of objects than to not do that. Defensive copying and deep copying is not a thing you have to do in immutable languages. Even under the covers, it's not happening the way you seem to think it is. If I had a large immutable map in use by some other process, and needed a version of it with an eleme…

In order to flatten these to regain cache hits, a deep copy is required all the way down.

There’s a reason that the fastest Haskell game engine looks like ps2 graphics on modern hardware.

Re: Managing mutable data in Elixir with Rust

#58
post #20
post #13

Earlier quoted context omitted.

I don’t really care what people making claims say when they make claims without evidence. ”Who” makes a claim has no bearing on its truth. Immutability is a tool, not a rule, and I am free to reject any assertion otherwise when those assertions provide no evidence, or shitty anecdotes. Prove your claims. Certainly, immutability is a foundation for performance problems. Another provable rule in computing is that more…

If you're requesting proofs, don't make so many bold statements you cannot prove yourself.

Provided above.

Re: Managing mutable data in Elixir with Rust

#59
post #54
post #50

Earlier quoted context omitted.

Sure: 1) immutability has performance problems: source: literally every measurement of immutable vs not data structures ever performed. Source 2: logic - copying data is slower than not copying it Source 3: cache lines: modern CPUs rely pretty heavily on cache lines and branch prediction to improve performance. Immutability measurably harms both. 2) immutability requires more code and loc is the best predictor of def…

I wanted to write that making defensive copies is something you need to do in mutable situations to preserve safety, (not in immutable situations!), but it looks like enough commenters have hit that point. So lets disabuse your mistrust of immutability in another domain! Here is some typical "go fast and mutable!" nonsense code: int foo(int i, int j) { while (i Let's compile it with https://godbolt.org/ , turn on som…

Listen dude. I’m not going to argue about this:

Immutability is measurably slower. Full stop.

The fact that you can come up with silly, overly simplistic, non-idiomatic anecdotes showing that sometimes a compiler will prefer calculation doesn’t change that. It is a commonly known fact in low level programming that just because you’re calculating something doesn’t make it slower by default.

When Haskell devs can produce a game engine that doesn’t look like PS2 on a 4090, we can chat again about how immutability is supposedly not slow.

Re: Managing mutable data in Elixir with Rust

#60

Earlier quoted context omitted.

> you have a tradition of development practices that have much more often lead you to scalability and robustness than than the alternatives I'm not GP, but these traditions are usually not backed by any evidence but by cargo-culting and cult-of-personalities. Not to mention people who over-hype their favourite technologies to high heavens, poisoning the well for everyone else (no, most telecom industry doesn't run Er…

Do they need to be backed by evidence to be correct? What if they are just right? Do other companies not running Erlang mean its wrong? Because that's a bold argument that could be restated to disprove many things -- X technology claims to do Y well and has a track record of doing it really well, but Z companies don't use X technology, therefore...[insert whatever]. I don't believe in blindly believing things without…

> The ancients had it right that theres different types of knowledge

People hawking these traditions usually do for consulting money, not spiritual fulfilment. I am all for non-materialism, but only as long as it's not used to exploit me. Belonging to a post-Colonial country, I know exactly where that leads.

Post reply on HN