Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir GenServer Explained

papercups.io

Post reply on HN
Print view View original

41–50 of 91 posts

Re: Elixir GenServer Explained

#41

Post by dnautics »

nrmitchi wrote: post #7
jimbokun wrote: post #5

Since these insert events are being buffered, what happens if the process dies? Are all of those inserts lost? I feel like Erlang/Elixir is designed to handle cases like this robustly, but it's not clear to me how this code avoids losing data when a process crashes, potentially up to 5s worth of updates!

I'd like a solid answer to this too; it's my biggest concern with any sort of "batch online requests" functionality in any language; unsafe shutdowns can/will always happen at some point , and it feels like this is always a data loss risk.

To some degree there's always data loss risk... A backhoe could cut a line, you could lose power to a PSU, rack, data center, a meteor could destroy the willamette valley, etc. What you want your system to provide you with is the framework to understand the risks and what the recovery looks like and when recovery is a lost cause.

Re: Elixir GenServer Explained

#42

Post by lostcolony »

rjknight wrote: post #33
lostcolony wrote: post #24

Earlier quoted context omitted.

Short Answer - yes, data will be lost. THAT SAID - this is true of any buffering solution, it's why even the 'sync' style of request can still time out. Long Answer - What happens in any language where you batch stuff in memory? If the system breaks down, you lose that data. This isn't unique to Erlang/Elixir. There's a tradeoff that persistent storage is slower than memory, but it's persistent. So do you want to be…

> What happens in any language where you batch stuff in memory? If the system breaks down, you lose that data. This isn't unique to Erlang/Elixir. A little over a decade ago I did a project using Erlang. My next project was using Ruby, and I was suddenly _horrified_ by the fact that I had no idea what would happen if the application crashed (and Ruby apps, at least in those days, crashed fairly often). Erlang/Elixir…

100%

I consider the couple years I built systems in Erlang to be fundamental for me. It's affected, for better and worse, my entire approach to system design, at every level. It's meant the stuff I or (now that I'm in management) my teams tend to write is incredibly resilient (compared with the other teams in the department), but also meant that I have a really hard time with any Silicon Valley interview.

Re: Elixir GenServer Explained

#43

Post by [deleted] »

JohnCurran wrote: post #34

This line in the opening paragraph really rubs me the wrong way: > “I'm ashamed to say most of my Elixir education has been through trial and error, figuring things out as I go along“ This attitude is so prevalent in software as if we are all supposed to be divined with programming knowledge the moment our IDE spins up. In every other industry that’s exactly how you learn: Get your hands dirty, make mistakes, and fix…

[deleted]

Re: Elixir GenServer Explained

#44

Post by _qbjt »

Zarathu wrote: post #38

> In particular, I was itching to learn more about handling concurrency in Elixir. This, of course, led me to GenServers. Might be a nitpicking here, but GenServers aren't useful for concurrency. They just manage state, and only process one message at a time. If you're using this as a cache, your reads will be bottlenecked by however quickly the GenServer can handle the read requests.

> GenServers aren't useful for concurrency.

Sure, maybe not by themselves but typically you would run many GenServers concurrently in your application as part of a supervision tree. Libraries like Broadway (and the underlying GenStage) are essentially just leveraging GenServer to make it easier to orchestrate concurrency and state synchronization across multiple processes in your application. But you could build a comparable system on your own just using GenServer and a dynamic supervisor.

Re: Elixir GenServer Explained

#45

Post by areichert »

JohnCurran wrote: post #34

This line in the opening paragraph really rubs me the wrong way: > “I'm ashamed to say most of my Elixir education has been through trial and error, figuring things out as I go along“ This attitude is so prevalent in software as if we are all supposed to be divined with programming knowledge the moment our IDE spins up. In every other industry that’s exactly how you learn: Get your hands dirty, make mistakes, and fix…

OP here -- I know what you mean. "Ashamed" is probably too strong a word. I think the feeling I was trying to convey was just that I'm not exactly an authority on Elixir, so take everything I say with a grain of salt, and nitpick away :)

Re: Elixir GenServer Explained

#46

Post by brightball »

edisonywh wrote: post #32
jimbokun wrote: post #5

Since these insert events are being buffered, what happens if the process dies? Are all of those inserts lost? I feel like Erlang/Elixir is designed to handle cases like this robustly, but it's not clear to me how this code avoids losing data when a process crashes, potentially up to 5s worth of updates!

The article touched on it briefly, but perhaps it wasn't clear. If the process dies, the Supervisor invokes the `terminate/2` callback so you're still able to process events. Here are the relevant lines: https://github.com/plausible/analytics/blob/b724def948d51a0f... Keep in mind you need to trap exit signal to tell the supervisor to invoke the callback, as done so here: https://github.com/plausible/analytics/blob/b7…

I usually utilize this with an ETS cache to save and recover GenServer state in the event of a crash.

Re: Elixir GenServer Explained

#47

Post by toast0 »

therein wrote: post #16
jimbokun wrote: post #5

Since these insert events are being buffered, what happens if the process dies? Are all of those inserts lost? I feel like Erlang/Elixir is designed to handle cases like this robustly, but it's not clear to me how this code avoids losing data when a process crashes, potentially up to 5s worth of updates!

Furthermore I have heard it is actually possible for casts to be dropped under load. I am having a hard time confirming this, and had a hard time confirming it back then too. And even worse, I realize most of the time we code as if the casts will be reliably delivered. Can someone chime in on this?

I don't think that's quite true, at least not of a gen_X cast.

If you use erlang:send/3 with options or erlang:send_nosuspend/2, you can have some cases where messages would be dropped without trying. I think that may be the source of the drop under load you're thinking of?

Without nosuspend, if you send to a node that's connected, dist will queue it to be sent, but there's no guarantee it's received because networking, and it may not even be sent if the other side has gone away and the send queue is already too large; There is a hard to express constraint that if your message doesn't get received in this case, dist will disconnect eventually, but it's important to note that the tick time-out doesn't guarantee timely delivery either; as long as some data is flowing, you can get quite a backlog; I think I've gotten net_adm:ping times above 30 minutes in some cases. Also, if the dist connection is dropped, but both nodes are online, it will likely reconnect shortly, and some messages will have been lost.

If you send to a node that's not connected, and didn't specify no_connect, dist will queue the message while attempting to connect, but if that attempt fails, the messages will be dropped.

It's also possible for code running as a gen_server (or GenServer, I suppose) to check how many messages are queued for it, and run different logic. I've written gen_servers that would drop optional requests if the queue was large. Also, if client timeouts are known, there are ways to approximate the time spent waiting in queue, and drop requests if they are received after the client already timed out; it's a little tricky to do this though.

Re: Elixir GenServer Explained

#49

Post by Philip-J-Fry »

Zarathu wrote: post #38

> In particular, I was itching to learn more about handling concurrency in Elixir. This, of course, led me to GenServers. Might be a nitpicking here, but GenServers aren't useful for concurrency. They just manage state, and only process one message at a time. If you're using this as a cache, your reads will be bottlenecked by however quickly the GenServer can handle the read requests.

If you've got 5 gen_servers and you cast each one a message then that is concurrency. The whole point of a gen_server is that it's a separate process doing it's own thing.

I can dump 1000s of things into its message queue and then do something else. It'll keep working away.

It's like saying threads aren't useful for concurrency because they can only do one thing at a time.

Re: Elixir GenServer Explained

#50

Post by muxator »

areichert wrote: post #48

In case anyone was curious what I used for the diagrams in this article, it was an awesome open source [0] tool called Excalidraw! [1] [0] https://github.com/excalidraw/excalidraw [1] https://excalidraw.com/

Excalidraw is a gem. I discovered it weeks ago, and cannot but praise it. The online instance is free and works really well, plus you have the possibility of hosting your own copy and be completely independent.

I also saw that there is the possibility of doing collaborative real time drawing, but still did not try it.

I have seen that the exported SVGs could be simplified (lots of repeated markup), and I am thinking about giving a try to do a PR.

Excalidraw is superb.

Post reply on HN