91–100 of 100 posts
Earlier quoted context omitted.
I never heard of your OTP, and I immediately knew which OTP they were talking about due to the Elixir keyword.
Do you program in elixir often?
Earlier quoted context omitted.
Elixir already started the process of extending OTP: Task is a hugely important addition to the family of genserver, genstatem, etc.
Isn't Task an entirely different beast to the gen_* behaviours? To me it looks like a convenience wrapper around Erlang's primitives. I usually use them directly if I need the flexibility to "await" later. If I don't, I have a small function "run_in_subprocess" (mostly to not run into binary gc problems) that spawns and receives within one function. The only real advantage I see is that it is separately supervised. W…
Task does the same thing but for a usage model that wasn't previously well supported. It basically removes the last reason I ever did spawn_link directly in Erlang.
Earlier quoted context omitted.
At WhatsApp we had a bunch of mnesia-backed services. I can't say that it was the best experience. Pretty much anything non-trivial, such as healing a cluster after a partition, or expanding a cluster, or cross-DC replication was a complete PITA to do properly. There's also very few people that have operational experience with it and very little documentation about it compared to pretty much any other storage engine.…
I think we would have found any database to be a PITA at write volumes and data volumes we were using, with our expectations of availability. I don't think we would have been able to get that performance out of MySQL, or Postgres; and anyway, we didn't have a dedicated team for database management to make sure nobody wrote bad queries if we did. Sync, status and reg ran on MySQL and php when I started, and it was a l…
Which is why I'm happy others have paved the way.
A huge amount of web apps can get by with a Postgres backed queue that won't break a sweat handling a few hundred writes per second on a low end cloud hosted VPS.
While leveraging something like Oban you don't even need to be an expert. You can just use the library, throw work into it with a reasonable amount of foresight / best practices and you can start developing features in your app.
> Forget using a million different technologies for things like background jobs, OTP can supply you with everything. It's true but in practice this usually doesn't pan out. For example with just background jobs alone there's the idea of queues, tracking failures / successes, exponential back-off retries, guaranteeing uniqueness, draining, periodic tasks and everything else you'd likely want in a production ready app.…
Phoenix ships with Ecto ORM-style-binding to Postgres underneath (usually) for a good reason! :) And if you need that in memory magic where the serial nature of a genserver won't suffice for your load, you basically have built-in Redis with Erlang's ets. And if you really need insane performance on longer-cached items than ets is optimal for, Discord released https://github.com/discord/fastglobal And if you don't wan…
The big difference there is if you have a Dockerized app, Redis won't be restart on every deploy but the BEAM will be so you will lose your ETS cache or whatever you're storing.
Now, I know cache is meant to be disposable but this model of using ETS instead of Redis reminds me of a Python or Ruby app storing its cache in memory. I mean, sure you can do it but for the longest time (7-8 years?) the community as a whole kind of landed on keeping that stuff out of your app's process.
Earlier quoted context omitted.
Elixir already started the process of extending OTP: Task is a hugely important addition to the family of genserver, genstatem, etc.
Isn't Task an entirely different beast to the gen_* behaviours? To me it looks like a convenience wrapper around Erlang's primitives. I usually use them directly if I need the flexibility to "await" later. If I don't, I have a small function "run_in_subprocess" (mostly to not run into binary gc problems) that spawns and receives within one function. The only real advantage I see is that it is separately supervised. W…
2) Task also implements the $callers process library key. What this means is that if you spawn a process with Task, it knows which process was responsible for calling it (note that this is in general different from "the process that supervises it"). In tests, you can use the $callers value to shard global state in a concurrency-friendly fashion.
Examples:
1. Make a mock in Mox. Spawn using Task. Call your mock from the task, Mox knows that the parent test is and serves the "correct mock".
2. Check out a database sandbox. Spawn using Task. Use the database in your task, Ecto knows what is parent test (and checkout) and serves the correct db view.
3. Make an HTTP request from your test. Stuff the $callers parameter into an HTTP header with term_to_binary and hex encoding (probably user-agent is a good choice). Use a plug to put $callers into your phoenix connection genserver. Spawn a Task that accesses your DB. Ecto knows what is the parent test (and checkout) and serves the correct DB view. So the cool thing is that YOUR REQUEST LEFT THE VM and it still worked! and this is composable too, if you do it right.
Earlier quoted context omitted.
Isn't Task an entirely different beast to the gen_* behaviours? To me it looks like a convenience wrapper around Erlang's primitives. I usually use them directly if I need the flexibility to "await" later. If I don't, I have a small function "run_in_subprocess" (mostly to not run into binary gc problems) that spawns and receives within one function. The only real advantage I see is that it is separately supervised. W…
1) Task makes it easy to supervise non-complicated async workflows. In prod, you REALLY want to supervise. Don't use spawn_link, it doesn't show up in the supervisin tree. 2) Task also implements the $callers process library key. What this means is that if you spawn a process with Task, it knows which process was responsible for calling it (note that this is in general different from "the process that supervises it")…
2) Every OTP behaviour keeps the ancestor info as well, I could use proc_lib instead of spawning directly.
Earlier quoted context omitted.
Well, the sys behavior is easy to implement. I also am not a fan of the complexity and overhead of gen_server so instead I use metal ( http://github.com/lpgauth/metal ). It's a simple receive loop with an optional init and terminate callback. It implement sys and can be supervised.
Does gen_server have a measurable overhead compared to this? The main event loop of gen_server looks remarkably similar to what you are doing but handles calls, casts, hibernation, debugging, provides stacktraces and doesn't "force" exit trapping. In the "happy-path", I don't see any additional overhead to what you are doing. I would also like to see "handle_call" and "handle_cast" (maybe even "handle_info" with a bl…
Maybe if it were instead a macro/parse-transform, injecting all the common "framework" code into the module implementing the server callbacks, you could then at least get the optimizations that result from all the calls into and out of the "framework" being local rather than remote calls. Private functions could be inlined/fused, etc.
Further, if HiPE was made to work again on Erlang 25, such a "statically-compiled" server module could also "stay native", with no context-switches to interpreted code, in a way that HiPE can't presently manage for gen_servers due to the code in `gen_server` and `proc_lib` themselves not executing natively.
Earlier quoted context omitted.
1) Task makes it easy to supervise non-complicated async workflows. In prod, you REALLY want to supervise. Don't use spawn_link, it doesn't show up in the supervisin tree. 2) Task also implements the $callers process library key. What this means is that if you spawn a process with Task, it knows which process was responsible for calling it (note that this is in general different from "the process that supervises it")…
1) It shows up in the application view of the observer. Also, if one both processes dies the other one dies too, what more supervision would I want? A Task is also not magically rerun if there is a problem. 2) Every OTP behaviour keeps the ancestor info as well, I could use proc_lib instead of spawning directly.
I'd just like to point out that while knowledge of OTP is a good thing for an Elixir developer, it isn't really needed for most application development. There are excellent libraries and frameworks that leverage OTP to provide a lot of value to their users, and most of us who use Elixir in industry just use those libraries.
Earlier quoted context omitted.
1) It shows up in the application view of the observer. Also, if one both processes dies the other one dies too, what more supervision would I want? A Task is also not magically rerun if there is a problem. 2) Every OTP behaviour keeps the ancestor info as well, I could use proc_lib instead of spawning directly.
ancestor is not the same as caller.