Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unpacking Elixir: Concurrency

underjord.io

Post reply on HN
Print view View original

61–70 of 138 posts

Re: Unpacking Elixir: Concurrency

#61

Post by sodapopcan »

andsoitis wrote: post #55
fndex wrote: post #51

Earlier quoted context omitted.

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

> they didn't use Distributed Elixir at all, just plain old Kubernetes, Kafka, Redis and GRPC There must be a good rationale for that decision. Do you know what it is?

There's a good article about BEAM + k8s by José Valim [0]

[0] https://dashbit.co/blog/kubernetes-and-the-erlang-vm-orchest...

You could certainly get away without some of the other stuff but, as another comment has mentioned, it requires some infra know-how. Like, you can't "just" use ETS as a Redis replacement without setting it up in a way that its data won't get blown away during a blue-green deploy.

Re: Unpacking Elixir: Concurrency

#62

Post by jolux »

jerf wrote: post #60
jolux wrote: post #45

Earlier quoted context omitted.

Note that Erlang has maps since OTP 17, which I believe are implemented more efficiently than dicts.

Good point, though I think my point about user types still holds. It is very difficult to create any efficient user types of your own with the primitives Beam provides.

Yeah, the lack of user-defined types is definitely one of the biggest downsides of the BEAM.

Re: Unpacking Elixir: Concurrency

#63

Post by pawelduda »

conradfr wrote: post #3

I think a few code example would have been nice. The thing I like about concurrency in Elixir is that it's there if you need it but it's mostly not mandatory in your code, compared to Javascript where its kind of imposed on you even when it's an hindrance. I remember one trick I used to do with LiveView on click events etc was to put all async (or "asyncable") code in a spawn function, which would speed up the return…

I remember building a Liveview dashboard that eventually had A LOT of things in a table being updated in real time, that the browser tab actually used like 15% of CPU. I implemented batching of websocket updates and added sleep() in interim genserver loop in like 1h... Problem solved, I could just change the sleep value depending on how fast I wanted the updates to happen

Re: Unpacking Elixir: Concurrency

#64

Post by sodapopcan »

giraffe_lady wrote: post #49
throwawaymaths wrote: post #31

Earlier quoted context omitted.

You really shouldn't be using agent for mutable state. Kind of annoying that the docs push you to that. Agent is most useful when you need to tie the transient lifetime of some state with something else (think: compilers). Otherwise, it's generally better to use a database, (or ets, if you need performance), GenServer/gen_statem if you need reactive events attached to the state. Agent is just a wrapper over GenServer…

I love elixir and am very comfortable with functional programming and generally prefer it. But being told you have to write a state machine instead of just += a var is an excellent experience in believing elixir is harder than python.

Of course you example is hyperbole and I certainly understand what you are saying. But the attitude that many programmers take when learning a new language that something is "harder" when it's just different than what they are used to bothers me (to be clear, I'm not directing that at you).

But comparing:

    i = 0
    for x in [1, 2, 3, 4]:
        i += x
vs

    i = Enum.reduce([1, 2, 3, 4], fn x, i -> i + x end)
There is nothing inherently "harder" about the functional version, it's just different (and of course comes with its own benefits).

Re: Unpacking Elixir: Concurrency

#65

Post by fndex »

weatherlight wrote: post #57
fndex wrote: post #51

Earlier quoted context omitted.

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

Well, in big orgs, that adopt Elixir/Erlang, along with other technologies with poor concurrency stories, those other ecosystems still need Kubernetes, Kafka, Redis and GRPC, to get by. elixir isn't going to make ruby or python apps magically concurrent. So that make sense. However, in orgs that are primarily Elixir shops, I don't see a lot of Kafka or gRPC. (Redis is just useful, its more than just a queue and K8s a…

>those other ecosystems still need Kubernetes, Kafka, Redis and GRPC, to get by

And what makes Elixir not need Kafka, Redis or GRPC?

Instead of Redis, you could use ETS for caching. But once you have 2+ instances of your app, you will need to have a centralized caching mechanism, otherwise, each instance will have its own ETS with its own memory, not sharing anything. Unless you decide to use Distributed Erlang and connect all the nodes of your application, which comes with a lot of trouble. Much easier to just use Redis.

And lets say you have multiple teams, each team has its own service(built with Elixir), and you need to have some async communication between those services. What native Elixir solution would you use instead of Kafka?

Same for GRPC. What's the alternative? Connecting all the nodes and using Erlang message-passing?

Re: Unpacking Elixir: Concurrency

#66

Post by whalesalad »

hinkley wrote: post #59
tombert wrote: post #39

Disckaimer: I have never used Elixir in any serious capacity, but I have done a good chunk of Erlang. Concurrency in Erlang sort of frustrates me...not because it's bad, but because when I use it I start getting pissed at how annoying concurrency is in nearly every other language. So much of distributed systems tooling in 2023 is basically just there to port over Erlang constructs to more mainstream languages. Obviou…

Any sufficiently complicated distributed program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang

I've done this multiple times. I will not make the mistake again. Next big system is being born in Elixir.

Re: Unpacking Elixir: Concurrency

#67

Post by cultofmetatron »

fndex wrote: post #51
tombert wrote: post #39

Disckaimer: I have never used Elixir in any serious capacity, but I have done a good chunk of Erlang. Concurrency in Erlang sort of frustrates me...not because it's bad, but because when I use it I start getting pissed at how annoying concurrency is in nearly every other language. So much of distributed systems tooling in 2023 is basically just there to port over Erlang constructs to more mainstream languages. Obviou…

This is coming from someone who likes Elixir. Not much for its distributed systems features, but mostly because of the language design. I keep hearing everyone talk about how Erlang/Elixir gives everything out of the box and you don't need to worry about Queues, RPC or whatever... But in reality, people don't really recommend using Distributed Erlang that much, on most Elixir gigs I worked, they didn't use Distribute…

well my startup uses distributed elixir. we use it horde to distribute certain long lived processes accross our cluster. that does not exclusive to kuernetes. we user kubernetes to manage and handle vm crashes (its only happened twice ever) as well as porvide consistent network toplogy between the nodes.

that said, having the ability to send messages between machines without having to bring in an external dependency like redis is AWESOME from a maintenance perspective. one less thing to worry about and the system just works. our uptime has been pretty amazing given our small team.

Re: Unpacking Elixir: Concurrency

#68

Post by malkosta »

brabel wrote: post #14
cutler wrote: post #12

The deal-breaker for me with Elixir has always been the lack of real Elixir vectors rather than the crappy Erlang array library. Yes, I know you can use a Map with numeric keys but that's not the same.

What would be the difference between a "real vector" and a Map with numeric keys?

> What would be the difference between a "real vector" and a Map with numeric keys?

Access in O(1) instead of O(nlogn).

Re: Unpacking Elixir: Concurrency

#69

Post by chefandy »

weatherlight wrote: post #58
westoque wrote: post #52

Earlier quoted context omitted.

regarding concurrency, language plays an important role and pretty much dictates how code is written which is i believe where it has most of the frustration. in erlang it’s in a functional style, in javascript it’s in an asynchronous style. what i’ve come to realize is that it’s still better and more maintainable to think synchronously and have the core of the tech handle concurrency, for example golang with it’s gor…

thats the thing though, elixir you can write it in a synchronous style and to make it concurrent is usually very easy because the semantics of regular and concurrent code is essentially the same. for example refactoring something like this: File.stream!("path/to/some/file") |> Stream.flat_map(&String.split(&1, " ")) |> Enum.reduce(%{}, fn word, acc -> Map.update(acc, word, 1, & &1 + 1) end) |> Enum.to_list() to be as…

Yeah— as a primarily python guy, I find concurrency much more palatable in elixir than in js. As a relatively infrequent js user, I have tripped on its asynchronicity for decades. After learning in Perl, shell scripting, and a smidge off C in the early 2ks, then PHP and python in the subsequent decade, I rage quit js every time I picked it up for anything significant until like 10 years ago. For the first few years, I always forgot basic facets of the language like the scope of "this" in anonymous callback functions.

While it's not a close analog, the way Unreal Engine's node-based "no code" Blueprints language approaches asynchronicity just feels so much more natural. Even hopeful pre-hello-world coders seem to conceptually get "well I can't get that because this hasn't happened yet." Having graphical representations of things both in nodes and in-game obviously helps in ways that wouldn't make sense in js, but being built from the ground up to handle it does show that it can be approached more intuitively.

Re: Unpacking Elixir: Concurrency

#70

Post by cultofmetatron »

out_of_protocol wrote: post #6

Unlike javascript, elixir makes you write synchronous code, e.g. def process_items(items) do items |> Task.async_stream(&Processor.process/1, max_concurrency: 2, timeout: 7000, on_timeout: :kill_task) |> Enum.to_list() end Semantics of regular and concurrent code is the same

you can replicate this using primises

source: nodejs dev for 8 years, now fulltime elixir dev for the last 5

Post reply on HN