Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Erlang/OTP by Example

erlangbyexample.org

Post reply on HN
Print view View original

41–50 of 108 posts

Re: Erlang/OTP by Example

#41

Post by ilovecaching »

jhgg wrote: post #31
ilovecaching wrote: post #25

Earlier quoted context omitted.

I content Erlang can be very hard to reason about. I've seen some extremely gnarly Erlang where the author didn't write a -spec and it was almost impossible to tell what shape a tuple parameter would take. In Rust the compiler enforces all of this for you. Rust gives you: - Maintainability: Rust signatures not only tell you the types of arguments, but also their lifetimes. A signature in Rust is an extremely strong a…

Rust however does not offer nearly the same level of power when it comes to runtime introspection that BEAM does. Coming from a company that uses elixir heavily and has made a significant investment in rust, I don’t think we would ever use solely rust on our distributed systems. However, we have rewritten some code that elixir was too slow at in rust and exposed it as a NIF on BEAM - and that has worked well. (Blog p…

Yes, we can do that through flamegraphs. For any container running on our system, we can get the call stack of the running process, time taken, resources used etc. We can attach to containers, proxy traffic etc, all through Kubernetes.

I said this earlier, but we've essentially separated operation concerns from our applications, and that opens us up to relying more on knowledge of Linux which is easier to hire for, and we can reuse that knowledge and all our tooling with any other languages we want to use.

Re: Erlang/OTP by Example

#42

Post by pera »

zinclozenge wrote: post #20
pera wrote: post #14

Earlier quoted context omitted.

Are you actually using Rust for professional web development? I've tried it earlier this year and my experience wasn't that great, so I'm quite surprised to read this comment.

What did you end up using for Rust? The actix ecosystem is pretty good, and quite fast.

I did like Rust (as a language), but I'm using Elixir, Phoenix and Ecto right now.

Re: Erlang/OTP by Example

#43

Post by jhgg »

ilovecaching wrote: post #41
jhgg wrote: post #31

Earlier quoted context omitted.

Rust however does not offer nearly the same level of power when it comes to runtime introspection that BEAM does. Coming from a company that uses elixir heavily and has made a significant investment in rust, I don’t think we would ever use solely rust on our distributed systems. However, we have rewritten some code that elixir was too slow at in rust and exposed it as a NIF on BEAM - and that has worked well. (Blog p…

Yes, we can do that through flamegraphs. For any container running on our system, we can get the call stack of the running process, time taken, resources used etc. We can attach to containers, proxy traffic etc, all through Kubernetes. I said this earlier, but we've essentially separated operation concerns from our applications, and that opens us up to relying more on knowledge of Linux which is easier to hire for, a…

We run between 200k and 2m processes per beam VM. I don't know how it'd be possible to get as precise metrics as we need just from relying on linux utilities. And in the same cluster, some processes although identical in code have dramatically different workloads.

Re: Erlang/OTP by Example

#44

Post by pera »

ilovecaching wrote: post #28
pera wrote: post #14

Earlier quoted context omitted.

Are you actually using Rust for professional web development? I've tried it earlier this year and my experience wasn't that great, so I'm quite surprised to read this comment.

What issues did you have? We haven't had very many issues. We rebuilt some internals tools, but for the most part it's been smooth sailing. We do use hyper and h2.

My main issue was with the ecosystem, which seemed too green and volatile/experimental. IIRC I used Actix and Diesel, for a toy web app.

Re: Erlang/OTP by Example

#45

Post by [deleted] »

ilovecaching wrote: post #5

As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust. Erlang requires a lot of TLC to get right, it's super slow, and it's hard to burst. Like, super hard to burst. Erlang nodes are meant to cluster as a k graph and never go down. Modern ops, especially container ops, does availability through ephemerality of services. The BEAM just doesn't like to be…

[deleted]

Re: Erlang/OTP by Example

#46

Post by RobertKerans »

bthornbury wrote: post #16
ilovecaching wrote: post #15

Earlier quoted context omitted.

I'm aware of all of these claims. Any Erlang developer has heard this shtick a thousand times. The reality is that you do eventually have to twist the BEAM once you reach a certain level of scale, and there are plenty of bugs. Like I said, it's also difficult to burst, so it's not always as black and white as you say. It's also missing a lot of the introspection you need to operationalize at scale. We've had to patch…

As an Erlang fan, who hasn't used it in production services, I'm wondering if you can let us know some specific scaling issues you encountered.

Out-of-the-box, it's only really designed to scale to a certain point. It's all generally nice, predictable, and with with low latency up until that point. But because the nodes are fully meshed, the TCP heartbeats alone kill performance once you go past that point. So for ex. 100 nodes gives you 5050 TCP connections (100+99+98+97...etc). As parent says, there are methods to deal with this (Riak Core would be an example). But they're non-trivial.

+ you possibly need to bear in mind the design goals: that Erlang is designed to run as a highly reliable, self contained system in _a single geographic location_, with that system possibly left to run on its own for long periods of time (years)

Re: Erlang/OTP by Example

#47

Post by dxhdr »

ilovecaching wrote: post #5

As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust. Erlang requires a lot of TLC to get right, it's super slow, and it's hard to burst. Like, super hard to burst. Erlang nodes are meant to cluster as a k graph and never go down. Modern ops, especially container ops, does availability through ephemerality of services. The BEAM just doesn't like to be…

> As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust.

I'm surprised to read this, Erlang and Rust have almost nothing in common. It's understandable that BEAM doesn't suit your needs but suggesting that everyone should write Rust instead is grossly misleading. They are suited to solving different problems!

Re: Erlang/OTP by Example

#48

Post by ilovecaching »

dxhdr wrote: post #47
ilovecaching wrote: post #5

As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust. Erlang requires a lot of TLC to get right, it's super slow, and it's hard to burst. Like, super hard to burst. Erlang nodes are meant to cluster as a k graph and never go down. Modern ops, especially container ops, does availability through ephemerality of services. The BEAM just doesn't like to be…

> As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust. I'm surprised to read this, Erlang and Rust have almost nothing in common. It's understandable that BEAM doesn't suit your needs but suggesting that everyone should write Rust instead is grossly misleading. They are suited to solving different problems!

They have a lot in common syntactically, semantically, and philosophically. They are both languages designed to promate safety and robustness. They're both suitable for writing highly concurrent, massively scale-able applications. The future is lots of cores, and the Rust community is on board with this.

I'd also like to point out that at both of my jobs, we used Erlang for uses cases that are most heavily associated with the language, and for which popular open source alternatives in Elixir/Erlang already exist. We have had better success scaling, operating, and hiring for Rust. We've saved man hours, time, and money.

So I am saying that Rust is a replacement for Erlang. It's not a tool for a different problem.

Re: Erlang/OTP by Example

#50

Post by namelos »

ilovecaching wrote: post #5

As someone who has worked two jobs now writing, deploying, and operating Erlang clusters, I recommend switching to Rust. Erlang requires a lot of TLC to get right, it's super slow, and it's hard to burst. Like, super hard to burst. Erlang nodes are meant to cluster as a k graph and never go down. Modern ops, especially container ops, does availability through ephemerality of services. The BEAM just doesn't like to be…

I also find erlang does not play well with modern ops work flow like docker and kubernetes. But as for Erlang, a lot of people uses Erlang for building in-memory massive stateful services, and a very fault tolerant system on top of it. I'm interested in both language, but haven't start playing with Rust yet. I tried Akka and find it's pretty good but the virtual machine is not as good as BEAM, but Scala is a powerful language. I'm curious about Rust. Could you please share some insights about if Rust is capable of such stateful services as Erlang does, or there could be other advantages in terms of stateful/fault tolerance?
Post reply on HN