Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unpacking Elixir: Concurrency

underjord.io

Post reply on HN
Print view View original

101–110 of 138 posts

Re: Unpacking Elixir: Concurrency

#101

Post by rozap »

fndex wrote: post #51
tombert wrote: post #39

Disckaimer: I have never used Elixir in any serious capacity, but I have done a good chunk of Erlang. Concurrency in Erlang sort of frustrates me...not because it's bad, but because when I use it I start getting pissed at how annoying concurrency is in nearly every other language. So much of distributed systems tooling in 2023 is basically just there to port over Erlang constructs to more mainstream languages. Obviou…

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

Cargo culting happens regardless of language. It's true that fully meshed distribution won't fly for Google or Amazon scale. But 99% of companies will never get to that scale, despite what they'd like to believe. Fully meshed distribution works just fine for many use cases.

Re: Unpacking Elixir: Concurrency

#102

Post by sangnoir »

rozap wrote: post #101
fndex wrote: post #51

Earlier quoted context omitted.

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

Cargo culting happens regardless of language. It's true that fully meshed distribution won't fly for Google or Amazon scale. But 99% of companies will never get to that scale, despite what they'd like to believe. Fully meshed distribution works just fine for many use cases.

> It's true that fully meshed distribution won't fly for Google or Amazon scale.

I'm not sure I agree at product level: WhatsApp seems to be scaling rather well. I can't say if their use of Erlang is "fully meshed distribution" or not, but it seems to be flying just fine as the world's number 1 messaging platform.

Re: Unpacking Elixir: Concurrency

#103

Post by darkmarmot »

josevalim wrote: post #72
fndex wrote: post #51

Earlier quoted context omitted.

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

> I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... Many companies are using Distributed Erlang but not the way you described: they are using it to exchange messages between nodes running the same version of the software. Imagine that you are building a web application, you can directly exchange live messages between nodes…

We use it across multiple versions of our software running in the same cluster. As long as you dark launch API changes, it's not much of an issue.

https://www.youtube.com/watch?v=pQ0CvjAJXz4

(Doh, just realized I was replying to Mr. Elixir himself! And you're familiar with our project anyway :)

Re: Unpacking Elixir: Concurrency

#104

Post by conradfr »

karmajunkie wrote: post #100
conradfr wrote: post #78

Earlier quoted context omitted.

ETS is quite faster than using Redis though.

single node though, so you have to add distribution in some manner. For some situations, whether its the "right" way or not, redis ends up being an easier way to go. ETA: As another couple of comments pointed out, ETS also dies with the node so you've got to handle that also when rolling it. ETS is cool, but its not a panacea.

Nebulex has different adapters although I've only use it on a local node which uses ETS and with transient data so I can't comment on them too much.

https://github.com/cabol/nebulex

Re: Unpacking Elixir: Concurrency

#105

Post by ch4s3 »

toast0 wrote: post #98
ch4s3 wrote: post #43

Earlier quoted context omitted.

No really. The scheduler isn’t really like an event loop. The scheduler will only allow a task so many units of execution before pulling it off the processing queue and scheduling other work. You can block an event loop, but not the scheduler.

Adding preemption doesn't make it not an event loop, IMHO.

The context of the discussion was the node event loop, which doesn’t have preemption. The BEAM scheduler is only like an event loop when you’re locked to one physical processor core, otherwise the schedule is spreading work across multiple OS level threads. You could squint and call this an event loop manager with preemption, but I think the phrase “event loop” detracts from the clarity of any correct description.

Re: Unpacking Elixir: Concurrency

#106

Post by di4na »

sodapopcan wrote: post #64
giraffe_lady wrote: post #49

Earlier quoted context omitted.

I love elixir and am very comfortable with functional programming and generally prefer it. But being told you have to write a state machine instead of just += a var is an excellent experience in believing elixir is harder than python.

Of course you example is hyperbole and I certainly understand what you are saying. But the attitude that many programmers take when learning a new language that something is "harder" when it's just different than what they are used to bothers me (to be clear, I'm not directing that at you). But comparing: i = 0 for x in [1, 2, 3, 4]: i += x vs i = Enum.reduce([1, 2, 3, 4], fn x, i -> i + x end) There is nothing inher…

Especially when

  Enum.sum([1, 2, 3, 4])
Exist ;)

Re: Unpacking Elixir: Concurrency

#107

Post by _v7gu »

weatherlight wrote: post #58
westoque wrote: post #52

Earlier quoted context omitted.

regarding concurrency, language plays an important role and pretty much dictates how code is written which is i believe where it has most of the frustration. in erlang it’s in a functional style, in javascript it’s in an asynchronous style. what i’ve come to realize is that it’s still better and more maintainable to think synchronously and have the core of the tech handle concurrency, for example golang with it’s gor…

thats the thing though, elixir you can write it in a synchronous style and to make it concurrent is usually very easy because the semantics of regular and concurrent code is essentially the same. for example refactoring something like this: File.stream!("path/to/some/file") |> Stream.flat_map(&String.split(&1, " ")) |> Enum.reduce(%{}, fn word, acc -> Map.update(acc, word, 1, & &1 + 1) end) |> Enum.to_list() to be as…

Node's "experimental" stream api also leads to code with similar semantics. It gives some additional concurrency options, but of course no parallelism outside of waits.

    createReadStream("file.txt")
      .flatMap((chunk) => chunk.split(" "))
      .reduce((acc, word) => {
        acc[word] = (acc[word] ?? 0) + 1;
        return acc;
      }, {})

Re: Unpacking Elixir: Concurrency

#108

Post by Joel_Mckay »

fndex wrote: post #51
tombert wrote: post #39

Disckaimer: I have never used Elixir in any serious capacity, but I have done a good chunk of Erlang. Concurrency in Erlang sort of frustrates me...not because it's bad, but because when I use it I start getting pissed at how annoying concurrency is in nearly every other language. So much of distributed systems tooling in 2023 is basically just there to port over Erlang constructs to more mainstream languages. Obviou…

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

Elixir/Erlang often simply replaces most of what Kafka, Redis and GRPC offer.

Also, have a look at the Phoenix Framework Channels examples, as it essentially replaces most simple micro-services architectures.

This recipe book covers the common design examples:

"Designing Elixir Systems with OTP: Write Highly Scalable, Self-Healing Software with Layers" (James Edward Gray, II, Bruce A. Tate)

One day, you too may get annoyed with IT complexity, and articles mostly written by LLM chatbots.

Happy computing, =)

Re: Unpacking Elixir: Concurrency

#109

Post by dangets »

fndex wrote: post #51
tombert wrote: post #39

Disckaimer: I have never used Elixir in any serious capacity, but I have done a good chunk of Erlang. Concurrency in Erlang sort of frustrates me...not because it's bad, but because when I use it I start getting pissed at how annoying concurrency is in nearly every other language. So much of distributed systems tooling in 2023 is basically just there to port over Erlang constructs to more mainstream languages. Obviou…

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

Sasa Juric makes this point in 'Elixir In Action' and some of his talks, where in other languages you need to pull in these other technologies near the start, whereas in Elixir you can use the built-in primitives until you start running into scaling problems.

- https://youtu.be/JvBT4XBdoUE?si=Xo0QXgVSI2HCg8pj&t=2198

Post reply on HN