Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir GenServer Explained

papercups.io

Post reply on HN
Print view View original

71–80 of 91 posts

Re: Elixir GenServer Explained

#71

Post by lostcolony »

Osiris wrote: post #70
lostcolony wrote: post #66

Earlier quoted context omitted.

Specific design questions, yes. The "45 minute to design a system" ones, not so much. The hard part of those is not getting too bogged down in details, and this just means I have one more, extremely important, likely unappreciated by the interviewer, detail I can get bogged down in: what happens when things fail.

What happens when things fail is an absolutely critical part of design / architecture. It's disappointing that interviewers works consider that an unimportant detail.

Again, 45 minutes for system design interviews. I could go on a long diatribe of why the FAANG interview strategies are flawed, but everyone knows that, FAANG interviewers included. You play the game if you want the role; if you're just, you know, good, and don't want to learn to play the game, then you're an acceptable false negative.

Re: Elixir GenServer Explained

#72

Post by lostcolony »

silly-silly wrote: post #64
therein wrote: post #16

Earlier quoted context omitted.

Furthermore I have heard it is actually possible for casts to be dropped under load. I am having a hard time confirming this, and had a hard time confirming it back then too. And even worse, I realize most of the time we code as if the casts will be reliably delivered. Can someone chime in on this?

Erlang casts are always delivered, or the sender crashes before that point.

To local pids. Remote, no guarantee is made. Ergo, generally better to assume no guarantees. Same as with everything else; you have at most once, or at least once; there is no exactly once (except in marketing speak that uses an identifier to allow at least once to dedupe within a window).

Re: Elixir GenServer Explained

#73

Post by lostcolony »

therein wrote: post #16
jimbokun wrote: post #5

Since these insert events are being buffered, what happens if the process dies? Are all of those inserts lost? I feel like Erlang/Elixir is designed to handle cases like this robustly, but it's not clear to me how this code avoids losing data when a process crashes, potentially up to 5s worth of updates!

Furthermore I have heard it is actually possible for casts to be dropped under load. I am having a hard time confirming this, and had a hard time confirming it back then too. And even worse, I realize most of the time we code as if the casts will be reliably delivered. Can someone chime in on this?

https://medium.com/learn-elixir/message-order-and-delivery-g... is what you want.

I'll chime in explicitly to your question though - messages don't just disappear without a reason, but the reason may not be visible to you. It's possible a receiving process got a message, crashed partway through processing it (possibly after you even sent other messages, so dropping those, too), restarted, and now is handling the next incoming messages, making it look like a message dropped. It's possible a receiving process is on another node (in distributed Erlang) and the packet dropped. Etc etc.

These all look the same. The way to handle them is always the same; accept you have at most once delivery, or include an 'ack or retry' mechanism and write your system to handle at least once delivery. As that article mentions, Erlang makes you start thinking of your system as a distributed system from the get go. This initially feels incredibly inconvenient, but as you go down the distributed system, fault tolerant, CAP-bound rabbit hole, it becomes incredibly helpful. The system doesn't hide the things it can't guarantee from you; it forces you to feel uneasy about them.

(Whereas, for example, in Java, I don't know that that worker thread has terminated. So some other thread's fiddling with some bit of shared memory to communicate something to that thread does nothing. Which I've had happen in production and led to a single instance out of the fleet have stale data, which caused us no end of grief. Erlang forces you to ask, as you've just done, "what happens if this message doesn't get received because something happened to the receiver"?)

Re: Elixir GenServer Explained

#74

Post by lostcolony »

ammanley wrote: post #67

Thank you so much for sharing this. I've not yet finished a decade in the industry, nor ever laid an eye on Elixir before, and this made sense pretty quickly. Great job explaining it all! On a side note, its pretty amazing to see the impact on readability and understanding that native language features have on something like async. I've worked with similar systems to the example in the past in JS/Go/etc, and while yo…

It flows better because it is writing synchronous code. The code inside of every process in Erlang is synchronous.

You achieve concurrency by spinning up processes (spawning functions). This isn't that different to firing an async function in Javascript, or starting a goroutine, etc.

And you communicate between the processes by sending async messages.

Those are the primitives. Asynchronously send messages between processes that are synchronously doing stuff. Go is actually not -that- different, except sending messages is synchronous, too, unless you set a buffer on the channel (and then you have to size it). Though I agree, because Erlang/Elixir keep it simpler than that, and are dynamically typed, it's a lot cleaner to write.

Re: Elixir GenServer Explained

#75

Post by namelosw »

It is very interesting to me that components in OTP like GenServer, Register, and Supervisor are surprisingly intuitive if one tries to learn and play Erlang first, but not so obvious and possibly looks artificial if people don't learn Erlang in the first place.

At first, I was surprised that the Erlang processes resemble the Actor model in such a minimalistic interface. Then I play with them for a while, I realized I wanted to extract something similar to GenServers. Then when I played with GenServers, soon I was bothered by a bunch of Pids, and yearning for Registry. In the end, Supervisor is also pretty natural because things in the universe generally cannot revive themselves.

In a retrospective, OTP resembles a lot of concepts we are familiar with, the Registry just looks like how the web works because we need to resolve the address problem. And vice versa: Docker, Service Discovery, and Kubernetes look like GenServer, Registry, Supervisor respectively just at different scales.

It seems to me to let our systems fulfill some properties, some recurring themes are required. We'll reuse them or rediscover them one way or another.

Re: Elixir GenServer Explained

#76

Post by nesarkvechnep »

Philip-J-Fry wrote: post #49
Zarathu wrote: post #38

> In particular, I was itching to learn more about handling concurrency in Elixir. This, of course, led me to GenServers. Might be a nitpicking here, but GenServers aren't useful for concurrency. They just manage state, and only process one message at a time. If you're using this as a cache, your reads will be bottlenecked by however quickly the GenServer can handle the read requests.

If you've got 5 gen_servers and you cast each one a message then that is concurrency. The whole point of a gen_server is that it's a separate process doing it's own thing. I can dump 1000s of things into its message queue and then do something else. It'll keep working away. It's like saying threads aren't useful for concurrency because they can only do one thing at a time.

I would advise to not rely on casting too much because it lacks backpressure.

Re: Elixir GenServer Explained

#77

Post by dnautics »

intergalplan wrote: post #54
dnautics wrote: post #52

Earlier quoted context omitted.

Version 1.0.4, first paragraph: https://hexdocs.pm/elixir/1.0.4/GenServer.html#content `The advantage of using a generic server process (GenServer)` current release (1.11.4), first paragraph; https://hexdocs.pm/elixir/1.11.4/GenServer.html#content `The advantage of using a generic server process (GenServer)`

TIL: ignore elixir-lang.org, go to hexdocs.

Elixir-lang.org > docs > elixir takes you straight to hexdocs.

Re: Elixir GenServer Explained

#78

Post by mulmen »

> I'm ashamed to say most of my Elixir education has been through trial and error, figuring things out as I go along.

Oh, gosh. This is how I learned... everything? I feel like I just walked into homeroom without my pants on.

Re: Elixir GenServer Explained

#79

Post by HH_GU »

dnautics wrote: post #68
ammanley wrote: post #67

Thank you so much for sharing this. I've not yet finished a decade in the industry, nor ever laid an eye on Elixir before, and this made sense pretty quickly. Great job explaining it all! On a side note, its pretty amazing to see the impact on readability and understanding that native language features have on something like async. I've worked with similar systems to the example in the past in JS/Go/etc, and while yo…

Oh that's so funny, because I'm an elixir user and we all claim that genserver (internal) organization is terrible. But yes, it is way better than callback hell... It just could be better still.

not genserver, what will you use?

Re: Elixir GenServer Explained

#80

Post by fn1 »

lostcolony wrote: post #74
ammanley wrote: post #67

Thank you so much for sharing this. I've not yet finished a decade in the industry, nor ever laid an eye on Elixir before, and this made sense pretty quickly. Great job explaining it all! On a side note, its pretty amazing to see the impact on readability and understanding that native language features have on something like async. I've worked with similar systems to the example in the past in JS/Go/etc, and while yo…

It flows better because it is writing synchronous code. The code inside of every process in Erlang is synchronous. You achieve concurrency by spinning up processes (spawning functions). This isn't that different to firing an async function in Javascript, or starting a goroutine, etc. And you communicate between the processes by sending async messages. Those are the primitives. Asynchronously send messages between pro…

The formal framework for this is called CSP ("Communicating sequential processes"). I discovered this 15 years ago, and once you understand it, you will never have problems in multithreading again.

https://arild.github.io/csp-presentation/#3

Post reply on HN