Live data from Hacker News

Elixir/Erlang Hot Swapping Code (2016)

kennyballou.com

101–110 of 119 posts

Re: Elixir/Erlang Hot Swapping Code (2016)

#101

Live updating a drone running Erlang in 10ms while it was flying with no application restart and no loss of state impressed me when I saw it in 2021: https://www.youtube.com/watch?v=XQS9SECCp1I But I almost never hear Erlang/Elixir/Gleam folks talk about this benefit of the Erlang VM now, even though it seems fairly unique and interesting. Has the community moved away from it? Is it just not that useful?

A lot of web apps are just well-enough served with a blue-green deployment model. It is less risky. But if you really need it, it's really great to have that option (e.g. very long running systems which are split in front/back etc), and it can be used in creative ways too (like the Drone example). Here is a lightning talk I gave about how to use hot-reload for music / MIDI interactions: https://www.youtube.com/watch?…

Extremely cool, thanks for sharing!

Re: Elixir/Erlang Hot Swapping Code (2016)

#102

Live updating a drone running Erlang in 10ms while it was flying with no application restart and no loss of state impressed me when I saw it in 2021: https://www.youtube.com/watch?v=XQS9SECCp1I But I almost never hear Erlang/Elixir/Gleam folks talk about this benefit of the Erlang VM now, even though it seems fairly unique and interesting. Has the community moved away from it? Is it just not that useful?

Huh, really? I feel like I see Elixir folks sing the praises of beam pretty regularly. Specifically the OTP supervisor stuff for fault-resistant server deployments. I haven’t looked specifically for that though recently so maybe people are taking it for granted?

Re: Elixir/Erlang Hot Swapping Code (2016)

#103
Hot code upgrades on the BEAM are awesome, but they're not a piece of cake. If you're also interested in the challenges of making them production safe, I gave a talk about this topic on CodeBEAM Sto earlier this year:

https://youtu.be/epORYuUKvZ0?si=gkVBgrX2VpBFQAk5

OP talks in the summary about the importance of understanding the process. It's very much true, but you need to understand not only the process your tooling provides, but also what's going on in the background and what hasn't been taken care for you by your tools. I'm afraid these things are rarely understood about hot upgrades, even by experienced Erlang engineers.

Re: Elixir/Erlang Hot Swapping Code (2016)

#104

Is this like a similar feature in Smalltalk/Pharo and Lisp?

Yes, the basics are there in Smalltalk and there's more support built into Erlang.

Also:

"Live program changes in the Dart VM"

https://github.com/dart-lang/sdk/blob/main/docs/Hot-reload.m...

"Live reloading for your ESP32"

https://github.com/toitlang/jaguar

Re: Elixir/Erlang Hot Swapping Code (2016)

#105
post #62

Earlier quoted context omitted.

Erlang absolutely has closures, you are mistaken. What you are referring to are "function captures", which bind a function reference as a value, and there is no environment to close over with those. However, you can define closures which as you'd expect, can close over bindings in the environment in which the closure is defined. The interaction between hot reloads and function captures in general is a bit subtle, par…

> but is force upgraded after two consecutive hot upgrades, as only two versions of a module are allowed to exist at the same time. Force upgraded is maybe misleading. When a module is loaded for the 3rd time, any processes that still have the first version in their stack are killed. That may result in a supervisor restarting them with new code, if they're supervised.

Ah right, good point - I was trying to remember the exact behavior, but couldn't recall if an error is raised (and when), or if the underlying module is just replaced and "jesus take the wheel" after that.

Re: Elixir/Erlang Hot Swapping Code (2016)

#106
post #65

Earlier quoted context omitted.

Erlang absolutely has closures, you are mistaken. What you are referring to are "function captures", which bind a function reference as a value, and there is no environment to close over with those. However, you can define closures which as you'd expect, can close over bindings in the environment in which the closure is defined. The interaction between hot reloads and function captures in general is a bit subtle, par…

What does is it look like? I was talking about this thing: Val = 1, SumFun = fun(X) -> X + Val end, SumFun(2). It looks like you define arity 1 function that captures Val, while in fact you define arity 2 function and bind 1 as a first argument. Since you can't redefine Val anyway, it's as good as a closure, but technically it doesn't capture the environment. Maybe I'm mistaken and there is another way to express it?

The example you've given here does not work the way you think it does. I would agree however that the mechanics of closure environments is simpler in Erlang due to the fact that values are immutable, as opposed to closures in other languages where mutability must be accounted for.

I would also note that, for the example you've given, the compiler _could_ constant-fold the whole thing away, but for the sake of argument, let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound.

The closure will be constructed at the point it is captured, using the `make_fun` BIF, with a given number of free var slots (in this case, 1 for the capture of `Val`). `Val` is written to the slot in the closure environment at this time as well. See the implementation of the BIF [here](https://github.com/erlang/otp/blob/6cefa05a2a977864150908feb...) if you are curious.

At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars. In my own Erlang compiler, the closure environment was given via pointer, as the first argument to the function, and then instructions were emitted to load free variables relative to that pointer. I believe BEAM does the same thing, but it may differ in the specific details, but conceptually that is how it works.

The compiler obviously must generate a new free function definition for closures with free variables (hence the name of the function you see in the interactive shell, or in debug output). The captured MFA of the closure is this generated function. The runtime distinguishes between the two types of closures (function captures vs actual closures) based on the metadata of the func value itself.

Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard.

Re: Elixir/Erlang Hot Swapping Code (2016)

#107
post #65

Earlier quoted context omitted.

What does is it look like? I was talking about this thing: Val = 1, SumFun = fun(X) -> X + Val end, SumFun(2). It looks like you define arity 1 function that captures Val, while in fact you define arity 2 function and bind 1 as a first argument. Since you can't redefine Val anyway, it's as good as a closure, but technically it doesn't capture the environment. Maybe I'm mistaken and there is another way to express it?

The example you've given here does not work the way you think it does. I would agree however that the mechanics of closure environments is simpler in Erlang due to the fact that values are immutable, as opposed to closures in other languages where mutability must be accounted for. I would also note that, for the example you've given, the compiler _could_ constant-fold the whole thing away, but for the sake of argumen…

> let's assume that `Val` is an argument to the current function in which `SumFun` is defined, and so the compiler cannot reason about the actual value that was bound.

That was exactly the case I was talking about, because otherwise there is no need to even make arity 2 function. If the value is known at compile time, the constant is embedded into the body of inlined function.

>At runtime, when the closure is executed, the underlying function receives the closure environment, from which it loads any free vars.

To my understanding, no it doesn't, as the value is resolved when the function pointed is created, not when the underlying function executes, which the code you linked shows too. I know it uses the "env" as a structure field, but it's partial application, not the actual closure which has access to parent scope. Consider two counter examples in python:

    for x in range(1,10): ret.append(partial(lambda y: y*2, x)) # that's what erlang does

    for x in range(1,10): ret.append(partial(x, lambda y: y*2)) # that's an actual closure, as all lambdas will return 18 because x is captured from the parent context
But then again, it doesn't matter since variables are assigned only once.

>Like I mentioned near the top, it's worth bearing in mind that the compiler can also do quite a bit of simplification and optimization during compilation to BEAM - so there may be cases where you end up with a function capture instead of a closure, because the compiler was able to remove the need for the free variable in cases like your example, but I can't recall what erlc specifically does and does not do in that regard.

I was looking into it a week ago, and erlc does what I described when it can't figure out the constant at compile time.

add: If we are at it, BEAM doesn't even know about variables, only values and registers anyway, so it has nothing to capture anyway.

Re: Elixir/Erlang Hot Swapping Code (2016)

#109

Live updating a drone running Erlang in 10ms while it was flying with no application restart and no loss of state impressed me when I saw it in 2021: https://www.youtube.com/watch?v=XQS9SECCp1I But I almost never hear Erlang/Elixir/Gleam folks talk about this benefit of the Erlang VM now, even though it seems fairly unique and interesting. Has the community moved away from it? Is it just not that useful?

[deleted]

Re: Elixir/Erlang Hot Swapping Code (2016)

#110
post #66

Earlier quoted context omitted.

If we to wedge how Erlang does hot code swapping into a container metaphor, then to get what Erlang does, you'd need to have a container per function call. Given that it would be absurdly wasteful to use OS processes in containers to clone Erlang's code reload system, AnotherGoodName might take ten minute to watch Erlang: The Movie to get a better sense of the capabilities of that system. The movie is available from…

>If we to wedge how Erlang does hot code swapping into a container metaphor, then to get what Erlang does, you'd need to have a container per function call. You have a container that responds to HTPP requests sitting behind a load balancer, then you spawn a new container and tell load balancer to redirect calls to the new one. From the point of view of whoever is calling the load balancer you have hot swapping. You m…

Sure, you can shut down and restart your entire application. You could do that back in 1990 without containers, too.

The thing is that Erlang does hot reload at a per-function (or -according to Hebert- sometimes more-fine-grained) level, so nuking the entire program and paying the cost to start it up again is not at all the same thing as -say- using a not-absurdly-priced AWS Lambda [0] or similar to get per-function hot reloading.

By the way, have you read "A Pipeline Made of Airbags"? If not, you should give it a read: https://ferd.ca/a-pipeline-made-of-airbags.html>. It might be old news to you, maybe, but maybe not.

[0] Good luck finding one, though.

Post reply on HN