Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Scaled Elixir to 5M Concurrent Users

blog.discordapp.com

Post reply on HN
Print view View original

141–150 of 260 posts

Re: How Discord Scaled Elixir to 5M Concurrent Users

#141

Post by rdtsc »

mononcqc wrote: post #93
jlouis wrote: post #86

A fun idea is to do away with the "guild" servers in the architecture and simply run message passes from the websocket process over the Manifold system. A little bit of ETS work should make this doable and now an eager sending process is paying for the work itself, slowing it down. This is exactly the behavior you want. If you are bit more sinister you also format most of the message in the sending process and makes…

The true evil approach is to send the socket around, not the message, so that there is no copying required no matter what ;)

Wah. Easy there, Satan :-)

That is cool trick though. So it's basically sending the port itself around and changing its ownership, with something like port_connect(Port,NewOwner)?

And btw, thank you for writing https://www.erlang-in-anger.com and http://learnyousomeerlang.com !

Re: How Discord Scaled Elixir to 5M Concurrent Users

#142

Post by aphextron »

jakebasile wrote: post #39
lordCarbonFiber wrote: post #25

Earlier quoted context omitted.

A closed source walled garden chat service that survives purely on the free flow of VC capital 100% has no long term viability. No federation means as soon as the "next thing" shows up and wins way the VC dollars they'll disappear. It's so incredibly frusterating that so many companies and users make/support these closed environments even as we enter a new golden age of open sourced and federated technologies.

If there was an open source and federated equivalent to the features Discord provides I'd use it. There is no such product. Matrix is interesting, but the experience is no where near as polished as Discord and friends and that matters for mass adoption.

>If there was an open source and federated equivalent to the features Discord provides I'd use it.

But will you pay for Discord though? The features and quality Discord is able to provide are artificially propped up by VC funding. When it runs dry, we will be left with open source offerings, or the next product to take it's place and repeat the cycle.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#143

Post by scaryspooky »

bsder wrote: post #38
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

> I see a bright future for Elixir, and a breath of fresh air for Erlang. Is Erlang really that much of a barrier? Erlang was a touch odd, but I didn't find the language itself that mind-bending. Wrapping my head around the proper way to structure things and the proper use of the OTP libraries was much more time-consuming. That level of architectural thought doesn't magically go away because you changed the language.…

I have never had an easier job writing an MPEG-TS parser than when I used Erlang. It was incredibly easy and concise.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#144

Post by [deleted] »

pmarreck wrote: post #80
ShaneWilton wrote: post #9

Thanks for putting this writeup together! I use Elixir and Erlang every day at work, and the Discord blog has been incredibly useful in terms of pointing me towards the right tooling when I run into a weird performance bottleneck. FastGlobal in particular looks like it nicely solves a problem I've manually had to work around in the past. I'll probably be pulling that into our codebase soon.

Note that Erlang 20 may have solved the problem that FastGlobal tries to fix (that of not copying large amounts of data unnecessarily)

[deleted]

Re: How Discord Scaled Elixir to 5M Concurrent Users

#145

Post by ShaneWilton »

pmarreck wrote: post #80
ShaneWilton wrote: post #9

Thanks for putting this writeup together! I use Elixir and Erlang every day at work, and the Discord blog has been incredibly useful in terms of pointing me towards the right tooling when I run into a weird performance bottleneck. FastGlobal in particular looks like it nicely solves a problem I've manually had to work around in the past. I'll probably be pulling that into our codebase soon.

Note that Erlang 20 may have solved the problem that FastGlobal tries to fix (that of not copying large amounts of data unnecessarily)

Erlang 20 fixes the case where you're copying a constant literal, but unfortunately won't help if you're sharing a dynamically generated, but infrequently modified, term; like Discord does in this post.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#146

Post by rdtsc »

didibus wrote: post #60

So, at this point, every language was scaled to very high concurrent loads. What does that tell us? Sounds to me like languages don't matter for scale. In fact, that makes sense, scale is all about parallel processes, horizontally distributing work can be achieved in all language. Scale is not like perforance, where if you need it, you are restricted to a few languages only. That's why I'd like to hear more about pro…

> Sounds to me like languages don't matter for scale.

If by languages you mean syntax, perhaps. If you mean platforms, then it does matter. And it's non obvious things such as fault tolerance for example - ability not just to have lots small concurrent processes but that they have isolated heaps.That's not just gimmick but it allows designing systems and operating them in a different way. For example having whole subsystems crash and restart safely without affecting the rest of the service.

Or even silly things like being able to hot reload code or log into a live node and add a dynamic trace or hotpatch a module to get extra debugging info without stopping.

Now you can sort of do that with other frameworks, it's just it's much nicer in Erlang because it comes built-in and it just feels like using the right tool for the job. As in using a hammer to hammer nails vs say using the pliers to hammer nails.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#147

Post by sriram_malhar »

I really like elxir the language, but find myself strangely hamstrung by the _mix_ tool. There is only an introduction to the tool, but not a reference to all the bells and whistles of the tool. I'm not looking for extra bells and whistles, but simple stuff like pulling in a module from GitHub and incorporate it. Is there such documentation? How do you crack Mix?

Re: How Discord Scaled Elixir to 5M Concurrent Users

#148

Post by bradmwalker »

pmarreck wrote: post #55
bsder wrote: post #38

Earlier quoted context omitted.

> I see a bright future for Elixir, and a breath of fresh air for Erlang. Is Erlang really that much of a barrier? Erlang was a touch odd, but I didn't find the language itself that mind-bending. Wrapping my head around the proper way to structure things and the proper use of the OTP libraries was much more time-consuming. That level of architectural thought doesn't magically go away because you changed the language.…

Syntax is a funny thing. It seems that some developers (perhaps most) care about it (disclaimer: myself included) but there's a large number I've encountered who do not. It seems to be a subjective preferential thing, because most rational discussions I've attempted to have about it result in circling around a drain and coming to no conclusion. I'd say that they're simply blind to it (similar to how a colorblind pers…

Syntax (and to some extent, expressiveness/conciseness) is superficial. How the code feels under change matters more.
Post reply on HN