I assume because every process has its own GC that messages are always copied, even on the same node? AFAICT, there is no shared/copy-on-write messages in the BEAM, except for a few cases where you explicitly pass a shared byte buffer for special circumstances? I would think this is worth the cost in most cases, but for some workloads would be prohibitive.
A Brief Guide to OTP in Elixir
21–30 of 100 posts
Re: A Brief Guide to OTP in Elixir
#22> In a sense, supervisors are very similar to Kubernetes, but they work on the application level instead of the cluster level.
Good article, and very worth sharing, but it could have done without that part. That's a terribly unhelpful analogy. For those who Kubernetes it's wildly misleading, and for those who don't it's meaningless.
Re: A Brief Guide to OTP in Elixir
#23> Forget using a million different technologies for things like background jobs, OTP can supply you with everything. It's true but in practice this usually doesn't pan out. For example with just background jobs alone there's the idea of queues, tracking failures / successes, exponential back-off retries, guaranteeing uniqueness, draining, periodic tasks and everything else you'd likely want in a production ready app.…
I think these blog posts usually pretend it's simpler than it is. There's usually (almost always) durable storage required somewhere. But to respond to your example specifically, OTP offers quite a bit. To your points:
Queues: a coordinator genserver process per node, which uses poolboy to only run X jobs at a time
Tracking failures/sucesses: postgres
Exponential backoff: a function
Uniqueness across the cluster: stdlib global locking (https://erlang.org/doc/man/global.html#trans-2)
draining: remote console
periodic tasks: a simple library that knows how to cron
I built a sharded cron-like scheduler in a few hundred lines of code. It's not doing a billion transactions per second, but it's been running in production for a few years without much drama. It's not complicated because it's just postgres and some standard erlang/elixir stuff.
Erlang has plenty of drawbacks, but this isn't it.
Re: A Brief Guide to OTP in Elixir
#24> In a sense, supervisors are very similar to Kubernetes, but they work on the application level instead of the cluster level. Good article, and very worth sharing, but it could have done without that part. That's a terribly unhelpful analogy. For those who Kubernetes it's wildly misleading, and for those who don't it's meaningless.
I use both kubernetes and OTP supervisors and don't find it misleading, rather I quite agree but maybe I'm missing something. I'm curious what you mean by that?
Re: A Brief Guide to OTP in Elixir
#25> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…
There are libraries that do this, like https://github.com/sasa1977/exactor but the Elixir community nowadays seems more conservative about macro "magic" than it was a few years ago. Sasa has written an entire book about OTP and if he doesn't recommend using his own library I wouldn't argue with him, but I haven't written a Genserver myself in years despite writing Elixir for a living the past 3.5 years. If you are go…
> There are libraries that do this, like https://github.com/sasa1977/exactor
At the very top of the Extractor readme:
> I don't maintain this project anymore. In hindsight, I don't think it was a good idea in the first place. I haven't been using ExActor myself for years, and I recommend sticking with regular GenServer instead :-)
This both confirms what you said about the Elixir community shying away from macro magic, and makes this library a very bad idea to use in new projects, imo.
Re: A Brief Guide to OTP in Elixir
#26Earlier quoted context omitted.
Of course it's easier to understand. It's also much easier to get wrong. Just yesterday I messed around with running a small process on to ship subscribed messages across node boundaries and forgot the recursive call in one of 4 cases, leading to mysterious stopping. In Elixir you don't even have to define all of this boilerplate, just the functions you actually need. Also, the callbacks are not classical methods, a…
That's what I was thinking about, client side. Probably the Elixir compiler could detect it and generate the code to bridge the gap. It feels OO-ish but we're dealing with mutable state anyway. I prefer "easy" than "pure".
Seems like you're losing something here -- ISTM that the goodness of a reified massage is now lost. A message, as a piece of data, can be tested, stored, passed on, etc., while a "call" (the .get()) cannot.
Re: A Brief Guide to OTP in Elixir
#27> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…
There are libraries that do this, like https://github.com/sasa1977/exactor but the Elixir community nowadays seems more conservative about macro "magic" than it was a few years ago. Sasa has written an entire book about OTP and if he doesn't recommend using his own library I wouldn't argue with him, but I haven't written a Genserver myself in years despite writing Elixir for a living the past 3.5 years. If you are go…
[deleted]
Re: A Brief Guide to OTP in Elixir
#28> In a sense, supervisors are very similar to Kubernetes, but they work on the application level instead of the cluster level. Good article, and very worth sharing, but it could have done without that part. That's a terribly unhelpful analogy. For those who Kubernetes it's wildly misleading, and for those who don't it's meaningless.
I use both kubernetes and OTP supervisors and don't find it misleading, rather I quite agree but maybe I'm missing something. I'm curious what you mean by that?
Interesting, thanks for the feedback. I find it misleading because Kubernetes is a massive platform that does all manner of things besides just supervising your Pods, and in fact I had to think about it for a while before concluding that the supervisory function is likely what the author was alluding to.
It strikes me as saying that an oxygen mask in a hospital room is like an airplane: sure they both provide a facility for providing oxygen to the user, but the airplane is significantly more complex than simply an oxygen delivery system. Probably a terrible analogy, just thinking off the top of my head.
Re: A Brief Guide to OTP in Elixir
#29> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…
What you said is rarely stated in public, but I've often felt it too: OTP obscures the underlying beauty of the Erlang platform.
OTP is well engineered of course, but the basic notion of the spawn -> receive -> loop cycle is so clean and illuminating that I wish newbies would hold out before learning OTP sometimes.
It's natural to think of case-specific abstractions around the primitives that are more germane to the domain at hand than OTP's.
Re: A Brief Guide to OTP in Elixir
#30OTP = Open Telecom Platform [1], though I think the full name is avoided nowadays. On an unrelated note, I've always been fascinated with the Erlang VM. The idea that you could hot-load modules and have multiple versions of the same module running at once seems really useful. I wonder why other runtimes haven't adopted these features? [1] https://en.wikipedia.org/wiki/Open_Telecom_Platform
Or "One Time Password" if you're an idiot like me.
Gotta love an introductory article on "OTP" which even includes a section entitled "What is OTP?" but doesn't expand the acronym.