Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Scaled Elixir to 5M Concurrent Users

blog.discordapp.com

Post reply on HN
Print view View original

191–200 of 260 posts

Re: How Discord Scaled Elixir to 5M Concurrent Users

#191

Post by pmarreck »

eggy wrote: post #140
pmarreck wrote: post #55

Earlier quoted context omitted.

Syntax is a funny thing. It seems that some developers (perhaps most) care about it (disclaimer: myself included) but there's a large number I've encountered who do not. It seems to be a subjective preferential thing, because most rational discussions I've attempted to have about it result in circling around a drain and coming to no conclusion. I'd say that they're simply blind to it (similar to how a colorblind pers…

I tried translating some of the book "The Handbook of Neuroevolution Through Erlang" into LFE (Lisp Flavored Erlang), and Elxir (somebody's done this already I believe). Robert Virding, a co-creator of Erlang, creator of LFE, also wrote Luerl - Lua implemented in Erlang. So if syntax is important you have many choices to use the BEAM VM/OTP in a Ruby-like language (Elixr), a Lisp (LFE) or in Lua (Luerl)! I am now loo…

How far did you get with the "Neuroevolution Through Erlang" book? I happen to own it (there must be like 10 of us!) but haven't gotten around to it yet...

Re: How Discord Scaled Elixir to 5M Concurrent Users

#192

Post by _asummers »

denisw wrote: post #135
_asummers wrote: post #72

Earlier quoted context omitted.

Ecto is hands down the best "ORM" I've used. At no point has it ever gotten in the way of us dynamically creating queries on the fly, just following the Ecto internals (not recommended, but totally doable) and has an extremely expressive syntax. The team behind it has been rolling out new features and are extremely responsive to requests on the mailing list. I encourage you to give it a chance. The mental shift is th…

Could you explain which changes to Elixir 1.5 and OTP 20 should result in better Dialyzer error messages? I didn't find anything relevant to that in the respective changelogs. (For context, Dialyzer is a static analysis tool detecting type errors which is part of the core Erlang distribution.)

It's kind of cryptic, but the Erlang release notes say:

  OTP-14369    Application(s): compiler, dialyzer, stdlib
               Related Id(s): PR-1367

               The format of debug information that is stored in BEAM
               files (when debug_info is used) has been changed. The
               purpose of the change is to better support other
               BEAM-based languages such as Elixir or LFE.

               All tools included in OTP (dialyzer, debugger, cover,
               and so on) will handle both the new format and the
               previous format. Tools that retrieve the debug
               information using beam_lib:chunk(Beam, [abstract_code])
               will continue to work with both the new and old format.
               Tools that call beam_lib:chunk(Beam, ["Abst"]) will not
               work with the new format.

               For more information, see the description of debug_info
               in the documentation for beam_lib and the description
               of the {debug_info,{Backend,Data}} option in the
               documentation for compile.
There aren't specific notes in the Elixir 1.5 release notes, but given that it is OTP20 compatible, I assume it would be able to leverage this, perhaps with some work by Dialyxir.

http://erlang.org/download/otp_src_20.0-rc2.readme

Re: How Discord Scaled Elixir to 5M Concurrent Users

#193

Post by Arathorn »

jakebasile wrote: post #39
lordCarbonFiber wrote: post #25

Earlier quoted context omitted.

A closed source walled garden chat service that survives purely on the free flow of VC capital 100% has no long term viability. No federation means as soon as the "next thing" shows up and wins way the VC dollars they'll disappear. It's so incredibly frusterating that so many companies and users make/support these closed environments even as we enter a new golden age of open sourced and federated technologies.

If there was an open source and federated equivalent to the features Discord provides I'd use it. There is no such product. Matrix is interesting, but the experience is no where near as polished as Discord and friends and that matters for mass adoption.

We are working as hard as we can on polishing Matrix. Making it mass user friendly is simply our #1 priority. Folks can help directly (beyond PRs, bug reports and feature requests) by donating at https://patreon.com/matrixdotorg.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#195

Post by _asummers »

ericmj wrote: post #173
_asummers wrote: post #127

Earlier quoted context omitted.

I agree completely. I never said that it should be used for evaluation or that you shouldn't read the code or docs. But then the question is... how do you find high quality libraries without something like download count? The issue is signaling good libraries, with poor discovery. I subscribe to many "libraries worth looking at" mailing lists, the awesome lists, etc. but that is still a pretty coarse net. Searching f…

Search on hex.pm is not great and can definitely be improved, but we are currently at the limit of postgres full text search afaik, unless there are some magic things to tweak. This means we need to switch to other technologies to improve the search which is a bigger task. But regardless of how good search is it wouldn't find ryal_core when searching for "commerce" if that word or any similar is nowhere in the metada…

Absolutely, and I'm thankful for all the hard work you all put in. I sound more annoyed than I am, and I know it was worse years ago before hex came around. Why can't Hex mind read yet?! :P

I think indexing GH READMEs would go a long way to helping the discovery problem, perhaps utilizing the new GH tags thing in some capacity, even letting authors tag their own packages. That would help in this specific case though this is of course a pathological example I happened to find recently. Maybe also having a "people who have X as a dependency also tend to download Y", though I'm not sure if you have that information outside of direct dependency graphs on Hex packages.

I feel like many newer languages have this problem, but the weird thing about Elixir is the 20-30 years of Erlang libs that are out there that are great that people have issues finding. It's worse if the libs aren't on Hex, and anecdotally I think people have been moving to Hex more lately for Erlang libs.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#196

Post by eternalban »

majidazimi wrote: post #156
josevalim wrote: post #67

Earlier quoted context omitted.

People tend to forget that scalability is not a binary property. You always scale up to some users, up to some architecture, up to some amount of nodes. There is no system that will scale to infinity without requiring developer intervention once business needs and application patterns start to settle in. Distributed Erlang/Elixir has known limitations . For example, the network is fully meshed, which gives you about…

What difference does it make? Erlang/OTP distribution doesn't have pluggable architecture. Sooner or later you will reach a point that you have to modify it. Then you are diverging from the original branch which makes it even more difficult to maintain it. You have to merge your additions into every release (minor or major) and test it thoroughly. A better architecture for a distributed system has a strong composabil…

> A better architecture for a distributed system has a strong composability property.

Sure, you want a beautiful lover, but is your object of desire within your reach? One needs to be practical when playing engineer.

Let's try this: Composibility is a highly desired property for distributed systems however correctly scaling composible semantics is a hard problem.

This sort of duality props up all over the place in general engineering, and certainly in software systems engineering.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#197

Post by jlouis »

majidazimi wrote: post #175
josevalim wrote: post #161

Earlier quoted context omitted.

What do you mean by architecture here? If you mean the roles different nodes take and their topology, I actually doubt you can be decoupled from your architecture in a distributed system because they directly affect your design and capabilities. You can move away from fully meshed for topologies but how does this choice affect node ups and node downs? Rebalancing can affect how you store data in the cluster. How many…

I did also mention that for fast time to market it is indeed a good idea to use already available tools. By the way writing a distribution layer is not that difficult. Many companies/startups with scalable back-ends that are not using Erlang/OTP have already done it. The point is scalable software requires knowledge and Erlang/OTP is not going to magically solve it. But it seems many fans are trying to promote it suc…

Erlang (and by extension Elixir) definitely provides a set of tools which are good for building highly concurrent distributed systems. And your systems are likely to have few errors as well as being resilient, if you know what you are doing.

But indeed, you need to know your tools like every other part of computer science. Storing 9 billion elements in an array, doing linear search and then complaining your linear search is too slow and needs to run faster will be disastrous to your architecture. Likewise, assuming communication is free is equally disastrous.

The problem with building a distribution layer in another language is the effort involved. Most of the companies which pull that off and get stability in addition are usually large, multinational, and has ample amounts of engineering resources to throw at the effort required.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#198

Post by jlouis »

censhin wrote: post #129
iagooar wrote: post #19

This writeup make me even more convinced of Elixir becoming one of the large players when it comes to hugely scaling applications. If there is one thing I truly love about Elixir, it is the easiness of getting started, while standing on the shoulders of a giant that is the Erlang VM. You can start by building a simple, not very demanding application with it, yet once you hit a large scale, there is plenty of battle-p…

The challenge is going to be adoption, plain and simple. I worked two years on an Erlang project, and most of our frustration came from the lack of community to lean on when we needed help understanding best practices. Also, when you find only one library to solve your problem that has been abandoned for at least two years that is never fun. Few projects actually need the level of scalability that Erlang brings, and…

The key benefit of Erlang isn't the scalability in my experience. It is resilience. Once your system is deployed, it tend to run with 0 fatalities for months on end. We have systems where we've had 0 maintenance days on them for months and even years.

In turn, you can focus your development effort on new systems, new features, and scaling existing infrastructure since you are not bogged down in maintenance mode. Even better, you can often postpone errors in a system which are non-fatal. I often wait until I have a couple of bugs in a system and then I'll work on that system for a few days fixing them all. In systems where even the smallest amount of brittleness destroys them, this isn't really an option.

Another experience of mine is that Erlang systems tend to take a bit longer to write compared to cranking out a fast solution in another language. But the payoff is that your Erlang solution tend to have better robustness as load increases and people start using the system.

The community is small however, so you'll need to write more code yourself in-house as a result. For some problem spaces, this will hurt a lot because there are off-the-shelf solutions in other languages. On the other hand, larger projects which are more specialized can benefit from you knowing your own code better.

Re: How Discord Scaled Elixir to 5M Concurrent Users

#200

Post by agentgt »

Cieplak wrote: post #20

I know that the JVM is a modern marvel of software engineering, so I'm always surprised when my Erlang apps consume less than 10MB of RAM, start up nearly instantaneously, respond to HTTP requests in less than 10ms and run forever, while my Java apps take 2 minutes to start up, have several hundred millisecond HTTP response latency and horde memory. Granted, it's more an issue with Spring than with Java, and Parallel…

> while my Java apps take 2 minutes to start up

This is the classic case (and I'm/we are guilty of it as well) of Java apps that are probably traditional monolithic apps that use frameworks with extensive reflection and class loading (ie spring component scanning).

You can get ridiculously fast loading time with Java if you use Dart2, a reasonable web framework and no ORMs. I'm talking 500ms... sort of depends on your JVM settings. Sure not 10ms but still way faster than 2 minutes.

Post reply on HN