A Brief Guide to OTP in Elixir • Hacker News Board

Re: A Brief Guide to OTP in Elixir

#11

Post by macintux » Thu, Oct 01, 2020, 5:42 PM UTC

yetihehe wrote: post #6

abetlen wrote: post #3

OTP = Open Telecom Platform [1], though I think the full name is avoided nowadays. On an unrelated note, I've always been fascinated with the Erlang VM. The idea that you could hot-load modules and have multiple versions of the same module running at once seems really useful. I wonder why other runtimes haven't adopted these features? [1] https://en.wikipedia.org/wiki/Open_Telecom_Platform

> I wonder why other runtimes haven't adopted these features? Probably mainly due to shared memory. When you know your data can be modified only in one place, changing data structure is easier. Try changing a struct when some other code is using it. It would require a lock for every object in memory.

One of the great things about Erlang and the BEAM is that by designing a very opinionated language around concepts like immutability and ubiquitous messaging between lightweight actors, you can define the VM to support those.

So there’s this collection of features between the language and VM that support each other, and are generally hard to reproduce outside that ecosystem because they are so interdependent.

Re: A Brief Guide to OTP in Elixir

#12

Post by ing33k » Thu, Oct 01, 2020, 5:44 PM UTC

jeremyjh wrote: post #2

I'd just like to point out that while knowledge of OTP is a good thing for an Elixir developer, it isn't really needed for most application development. There are excellent libraries and frameworks that leverage OTP to provide a lot of value to their users, and most of us who use Elixir in industry just use those libraries.

depends on the type of project for sure. possessing knowledge of OTP is a superpower. when working on any serious project, it will always help if there is at least one developer in the team who has knowledge about it.

Re: A Brief Guide to OTP in Elixir

#13

Post by dqv » Thu, Oct 01, 2020, 5:47 PM UTC

pmontra wrote: post #5

> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…

Do you do that with proc_lib or is it just a completely otp-unaware process like Joe would do?

Re: A Brief Guide to OTP in Elixir

#14

Post by pythonaut_16 » Thu, Oct 01, 2020, 5:48 PM UTC

matt2000 wrote: post #8

jeremyjh wrote: post #2

I'd just like to point out that while knowledge of OTP is a good thing for an Elixir developer, it isn't really needed for most application development. There are excellent libraries and frameworks that leverage OTP to provide a lot of value to their users, and most of us who use Elixir in industry just use those libraries.

Would you mind linking to a couple of them? I don't know anything about Elixir (but am very comfortable in a lot of other languages), and am interested in the kind of thing more experienced people are using with Elixir. Thanks!

Phoenix is a good example. It makes great use of OTP, but you don't really need to touch OTP when using it to build web applications. You can, and your life as an Elixir dev will certainly be easier if you have some understanding of OTP, but it works great as a web framework even if you don't.

Re: A Brief Guide to OTP in Elixir

#15

Post by atonse » Thu, Oct 01, 2020, 5:49 PM UTC

nickjj wrote: post #9

> Forget using a million different technologies for things like background jobs, OTP can supply you with everything. It's true but in practice this usually doesn't pan out. For example with just background jobs alone there's the idea of queues, tracking failures / successes, exponential back-off retries, guaranteeing uniqueness, draining, periodic tasks and everything else you'd likely want in a production ready app.…

Yep exactly. We process millions of jobs using Exq, mainly because of the retry, persistence of jobs, etc. I don't want to risk leaving that in elixir only.

We use Exq because I've used Sidekiq for years. I should check out Oban, since it could reduce the complexity in our stack (we wouldn't need redis anymore).

Re: A Brief Guide to OTP in Elixir

#16

Post by filmor » Thu, Oct 01, 2020, 6:00 PM UTC

pmontra wrote: post #5

> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…

Of course it's easier to understand. It's also much easier to get wrong. Just yesterday I messed around with running a small process on to ship subscribed messages across node boundaries and forgot the recursive call in one of 4 cases, leading to mysterious stopping.

In Elixir you don't even have to define all of this boilerplate, just the functions you actually need. Also, the callbacks are not classical methods, a handle_call doesn't have to reply immediately, for example. Distinguishing info, call and cast is also very relevant.

If you wanted something like

    {:ok, agent} = Agent.start_link ...

    agent.get()

I think you would need a change in the BEAM to recognise that agent is a "particular kind of PID".

Re: A Brief Guide to OTP in Elixir

#17

Post by mrdoops » Thu, Oct 01, 2020, 6:02 PM UTC

OTP is definitely more of an expert's toolset than a framework for throwing together features quickly. This is a good thing because it's not trying to be more of an abstraction than necessary. With OTP you can make fault-tolerant, concurrent, self-healing applications that handle stateful operations with ease, but you'll still need a fair bit of need-to-know about the callbacks, how to manage process lifecycles, etc.

What a lot of people seem to want is a batteries included framework that does the OTP for you, but doing at a high level of abstraction is difficult. So more commonly you'll see the OTP managed by libraries for more concrete use cases (web servers, database connection pooling, back pressured job processing, etc). Trying to wrap or rebuild OTP from the primitives available in the BEAM is possible, but definitely not an easy task.

Could we potentially have an OTP-like library that cuts down on the boilerplate significantly? Yes. But how much effort would that take and is it much of a win over OTP as-is? Not sure the trade-off is entirely there yet - OTP really isn't that bad. You're still writing less boilerplate than Java in general.

I think we're much better off writing good library-level abstractions over OTP that can be embedded in a supervision tree or used in more specific use cases. If we were to actually attempt some kind of higher level abstraction in the scope of OTP - maybe we can do so with a very different approach like decorated dataflow graphs and runtime property based testing.

Re: A Brief Guide to OTP in Elixir

#18

Post by sergiotapia » Thu, Oct 01, 2020, 6:03 PM UTC

nickjj wrote: post #9

> Forget using a million different technologies for things like background jobs, OTP can supply you with everything. It's true but in practice this usually doesn't pan out. For example with just background jobs alone there's the idea of queues, tracking failures / successes, exponential back-off retries, guaranteeing uniqueness, draining, periodic tasks and everything else you'd likely want in a production ready app.…

This is my experience as well, working with Elixir fulltime for about 5 years.

Re: A Brief Guide to OTP in Elixir

#19

Post by davidw » Thu, Oct 01, 2020, 6:22 PM UTC

Knowing how to use OTP is definitely something worthwhile if you're going to stick with the BEAM platform:

https://ferd.ca/ten-years-of-erlang.html

Re: A Brief Guide to OTP in Elixir

#20

Post by pmontra » Thu, Oct 01, 2020, 6:26 PM UTC

filmor wrote: post #16

pmontra wrote: post #5

> In real life, we don’t need to write code with receive do loops. Instead, we use one of the behaviours created by people much smarter than us. I make more than half of my income from Elixir. That said, the naive receive loop is much easier to understand than any of the GenServer examples. They pollute the module logic with all that handle_* boilerplate. I believe that neither Erlang nor Elixir got the right abstrac…

Of course it's easier to understand. It's also much easier to get wrong. Just yesterday I messed around with running a small process on to ship subscribed messages across node boundaries and forgot the recursive call in one of 4 cases, leading to mysterious stopping. In Elixir you don't even have to define all of this boilerplate, just the functions you actually need. Also, the callbacks are not classical methods, a…

That's what I was thinking about, client side. Probably the Elixir compiler could detect it and generate the code to bridge the gap. It feels OO-ish but we're dealing with mutable state anyway. I prefer "easy" than "pure".