Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Scaled Elixir to 5M Concurrent Users

blog.discordapp.com

Post reply on HN
Print view View original

51–60 of 260 posts

Re: How Discord Scaled Elixir to 5M Concurrent Users

#51

Post by mononcqc »

janekm wrote: post #46
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

I read it a little differently. The whole article is about how Erlang/Elixir fails at its core reason for existence (fast message passing between distributed processes) and all the complicated work-arounds they had to implement to avoid actually using this core feature of Erlang.

I have never seen a language or framework used at a large scale that did not require digging, understanding, and tweaking of the underlying machinery for its specific use case.

The true failure of the platform happens when you cannot do these tweaks and adaptation, or that their cost is shadowed by having written it in a more appropriate technology at a lower time/cost/effort budget.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#52

Post by quaunaut »

janekm wrote: post #46
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

I read it a little differently. The whole article is about how Erlang/Elixir fails at its core reason for existence (fast message passing between distributed processes) and all the complicated work-arounds they had to implement to avoid actually using this core feature of Erlang.

I don't think that's what Erlang was made for at all?

Erlang was never about speed, it's about reliability, availability, and ease of concurrency. And Elixir just makes it easier to access those tools.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#53

Post by janekm »

mononcqc wrote: post #51
janekm wrote: post #46

Earlier quoted context omitted.

I read it a little differently. The whole article is about how Erlang/Elixir fails at its core reason for existence (fast message passing between distributed processes) and all the complicated work-arounds they had to implement to avoid actually using this core feature of Erlang.

I have never seen a language or framework used at a large scale that did not require digging, understanding, and tweaking of the underlying machinery for its specific use case. The true failure of the platform happens when you cannot do these tweaks and adaptation, or that their cost is shadowed by having written it in a more appropriate technology at a lower time/cost/effort budget.

That's a good point. The article is certainly a good summary of what's needed to make Erlang/Elixir scale and a reminder that there are no "magic bullets".

Re: How Discord Scaled Elixir to 5M Concurrent Users

#54

Post by nichochar »

ConanRus wrote: post #35

I do not see there any Elixir specific, it is all basically Erlang/Erlang VM/OTP stuff. When you using Erlang, you think in terms of actors/processes and message passing, and this is (IMHO) a natural way of thinking about distributed systems. So this article is a perfect example how simple solutions can solve scalability issues if you're using right platform for that.

You're right. Elixir doesn't pretend to do anything except make using the wonderful erlang VM/OTP stuff easier.

The VM is an absolute marvel of engineering, and it's insane to me that it doesn't have more adoption yet in big tech companies.

My best theory is that engineers in top engineering companies are actually not the best engineers but simply career engineers that learn one skill (python/java/C++) and then explain to every employer that this technology is the best for the problem they have, over and over again.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#55

Post by pmarreck »

bsder wrote: post #38
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

> I see a bright future for Elixir, and a breath of fresh air for Erlang. Is Erlang really that much of a barrier? Erlang was a touch odd, but I didn't find the language itself that mind-bending. Wrapping my head around the proper way to structure things and the proper use of the OTP libraries was much more time-consuming. That level of architectural thought doesn't magically go away because you changed the language.…

Syntax is a funny thing. It seems that some developers (perhaps most) care about it (disclaimer: myself included) but there's a large number I've encountered who do not. It seems to be a subjective preferential thing, because most rational discussions I've attempted to have about it result in circling around a drain and coming to no conclusion. I'd say that they're simply blind to it (similar to how a colorblind person can't tell when their clothing color choices just "look wrong"), but they could (perhaps justifiably) argue that they're simply agnostic to it. It's just an aspect of a language that they don't care about and which doesn't influence whether they wish to work with the language or not.

... I'm not like that. I took one look at Elixir and said "HOLY CRAP IT'S A FUNCTIONAL SCALEABLE RUBY!" (of course that was merely scratching the surface, but...)

Re: How Discord Scaled Elixir to 5M Concurrent Users

#56

Post by cutler »

bigtunacan wrote: post #27
itbeho wrote: post #17

Earlier quoted context omitted.

It's still early. I've bet on the wrong horse (stack) before, so take my comments with a grain of salt, but I know several companies that are currently adopting Elixir/Phoenix with the same level of excitement that I recall from the early days of Ruby/Rails. It may never be "Rails big", but there is definitely some momentum building.

When Rails arrived on the scene it was very different from everything else, but also there weren't 1000 new languages/ frameworks popping up all at once. I'm not implying in anyway that Elixir is bad. I just think there are too many horses these days to know which to bet on. Elixir? Node? Go? Rust? Something else?

Exactly. If Elixir didn't have to compete with Clojure, Rust, Go, Node, Scala and Kotlin it might have had a chance to become mainstream. Beyond the hype job stats are the ultimate indicator.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#57

Post by vertex-four »

bigtunacan wrote: post #27
itbeho wrote: post #17

Earlier quoted context omitted.

It's still early. I've bet on the wrong horse (stack) before, so take my comments with a grain of salt, but I know several companies that are currently adopting Elixir/Phoenix with the same level of excitement that I recall from the early days of Ruby/Rails. It may never be "Rails big", but there is definitely some momentum building.

When Rails arrived on the scene it was very different from everything else, but also there weren't 1000 new languages/ frameworks popping up all at once. I'm not implying in anyway that Elixir is bad. I just think there are too many horses these days to know which to bet on. Elixir? Node? Go? Rust? Something else?

Depends on what you're doing, really.

Rust is lovely for security-sensitive code - I'm writing a customer identity management system with Rust+Postgres+Redis.

Go is... acceptable... for "glue" code where PHP would've been used a decade ago and Perl before that - all the successes of it I've seen fall into that sort of pattern.

Elixir is great when you need to think about networked, stateful systems on the scale of a rack of machines - it provides many of the components to help you design systems at that scale.

So... as ever, they all do quite seriously different things. I don't think many people need to build the sorts of systems Elixir is good for - it'll always have its niche in large-scale communications systems, though. A good many webapps fall into the patterns that Go is good for - take user input, munge it, send it to some backend system. A fair bit of code that drives the foundations of what those webapps are built on will eventually be written in Rust.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#58

Post by rubyn00bie »

toolz wrote: post #37
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

I think the next big push in elixir is tooling around metrics. Once there are hard numbers to market to non-techies I think we'll see a large shift towards elixir.

Just look for Erlang tools, there are loads available. The only problem with Elixir right now is sometimes people forget to look at Erlang for already solved problems/tooling/etc.

Erlang/BEAM has been alive and well in some very serious mission critical applications for 20 years. Just look at how telecoms used it (its origin).

Re: How Discord Scaled Elixir to 5M Concurrent Users

#60

Post by didibus »

So, at this point, every language was scaled to very high concurrent loads. What does that tell us? Sounds to me like languages don't matter for scale. In fact, that makes sense, scale is all about parallel processes, horizontally distributing work can be achieved in all language. Scale is not like perforance, where if you need it, you are restricted to a few languages only.

That's why I'd like to hear more about productivity and ease now. Is it faster and more fun to scale things in certain languages then others. Beam is modeled on actors, and offer no alternatives. Java offers all sorts of models, including actors, but if actors are the currently most fun and procudctive way to scale, that doesn't matter.

Anyways, learning how team scaled is interesting, but it's clear to me now languages aren't limiting factors to scale.

Post reply on HN