Live data from Hacker News

WhatsApp scaled to 1B users with only 50 engineers

quastor.org

121–130 of 462 posts

Re: WhatsApp scaled to 1B users with only 50 engineers

#121
post #31
post #23

Earlier quoted context omitted.

> Personally I want to build systems. Sure. And so do I. Specifically, I don't want to re build systems that have already been built for me which I can use as dependencies. And I don't want to spend all my time troubleshooting bugs, either - I'll take a well-maintained repository on GitHub with thousands of users bug-testing it for me over some janky thing some guy on another team threw together a few quarters ago be…

Having dependencies is not bad. But slapping a system together with a ton of dependencies you don't know about is bad. Simple is better in my opinion.

I agree, but this assumes you actually know what you are building. This isn't usually the case for a startup.

I'm sure whatsapp wanted a million users, but they didn't originally know that would happen. If whatsapp later turned out to be about serving fewer users with more features, this becomes a story about premature optimization.

Re: WhatsApp scaled to 1B users with only 50 engineers

#122
post #78

Remember the fast-cheap-secure triangle of product development? Now guess which one they skipped

The client is another story, but for the server they forked ejabberd which is a reliable, established XMPP server. So they probably had "fast-cheap-secure" on the server-side, because 6 years had been spent ironing out ejabberd bugs before the first version of Whatsapp came out.

My understanding is that vulns in Whatsapp so far were client-side, but i'm interested if i missed some on the server side.

Re: WhatsApp scaled to 1B users with only 50 engineers

#123
post #65

Earlier quoted context omitted.

A static React site, an Express backend, and a REST API is hardly exotic

Seems redundant to have a Express backend + REST API, why can't you just build your REST API on the Express backend? And also, React is complex, hardly anyone seems to know the internals. The API interface might be easy to learn, but "simple" is not something that you should label React with.

why can't you just build your REST API on the Express backend

Pretty sure he means to set up the REST api in Express with a few simple routes.

JS is not complex, and pretty vanilla choice. It’s what people add on top of it that gives it a bad name. I’ll link to this guy so you can understand:

https://kentcdodds.com/blog/how-i-built-a-modern-website-in-...

^ This is the problem, not JavaScript.

Re: WhatsApp scaled to 1B users with only 50 engineers

#125
post #84

Earlier quoted context omitted.

I don't see how this is related. Server-side you write and expose an API. Then you have 1-3 developers for each platform that write code that consumes that API. Those developers don't need to know anything about the internals of the server.

But this speaks to good architecture decisions, right? If they tried to make a shared framework that ran the same codebase everywhere and then port it onto different platforms ( cough facebook) then you don't get these nice isolated issues, for example.

Yes, if you don't write a native app for every platform the result will be rubbish. That's not a "good architecture decision", that is common sense. If they don't do it that way it's to save money, because as we all know Facebook is strapped for cash. (Talking about Facebook and Instagram here, because as far as I know WhatsApp is completely native on every platform)

Re: WhatsApp scaled to 1B users with only 50 engineers

#126
post #9

This article does very little to explain how they did that. It basically states the type of tech they used (Erlang, FreeBSD and SoftLayer), and something about not trying to over engineer stuff. The title also makes it sound like 500 engineers would make it easier to scale up WhatsApp, which does not make too much sense either.

The technology stack isn't even that relevant. There have been a few different articles about WhatsApp and their 1B users with only 50 engineers. The key in previous articles was that WhatsApp hired really smart people. Having extremely talented engineers and a rather small scope is what allowed them to grow til 1B users, with a minimum staff. Yet people seem to focus on the technology, because that's more easily rep…

Absolutely. A relatively small, highly efficient team that invests in their knowledge, skill and processes is the cause of correct technology choices and not the other way around.

Re: WhatsApp scaled to 1B users with only 50 engineers

#127
post #16

"The simpler product makes it much easier to maintain and scale." This is good news for the developers, the product owners and the customers. I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped. As a developer you need to support this in the future. It might be nice to build today but it will be a n…

"Personally I want to build systems. I don't want to spend my day debugging all kinds of weird problems."

Don't we all, I guess we all crave that snap, click, build -feeling Lego gave us. But in my experience building a system means debugging all kinds of weird problems. At least, when I start something, I usually for a long time feel like I'm just hopping from weird problem to weird problem. But, TBH, those problems don't seem weird anymore as experience grows.

Re: WhatsApp scaled to 1B users with only 50 engineers

#128
post #97

Earlier quoted context omitted.

I think the technology stack they choose is highly relevant to their success. The choice of Erlang was crucial: it's basically built for communications and has all pesky issues like version updating, resilience to failure and scalability already taken care of by nature of its architecture and framework. Of course, these 50 engineers were smart, most in that specialised space are, but I strongly doubt that they could…

Well true, but my point is that you're not going to be able to scale as successfully as WhatsApp, just by using the same technology stack, if your problem is different. You're right that Erlang was/is the right choice for WhatsApp, and it was most likely picked as the language of choice, because of the smart people working there. It's the same with FreeBSD, a failing startup isn't going to be able to layoff 50% of it…

Erlang was picked for a particular reason. WhatsApp uses a prebuilt open-source chat solution, an XMPP server written in Erlang, ejabberd. This thing was first released in 2003 (old). Already a smart move to get started.

I’m pretty sure they hired Erlang developers to dig into ejabberd internals and optimize certain things. They didn’t just decide to become an Erlang shop out of the blue.

It wasn’t Erlang that was the initial right choice, it was using xmpp and ejabberd that was the root reason. Erlang just happened to be a consequence of that.

https://www.ejabberd.im/

I will contest your attribution of ‘smart’ with Erlang. These types of correlations are generally bias fitting. It justifies ‘smart’ being correlated with any and all niche languages, eg ‘so and so likes Haskell, so they must be smart’. No good.

It’s better to attribute ‘smart’ with the pragmatic decision they made to simply use a pre-existing chat server solution that already has the capability to scale. Harder to assess this as smart since there’s no ‘signaling’ here, you have to objectively assess if it was the right tech (which it seems like it was). Way less vanity in this assessment as opposed to what I already pointed out, how your Haskell or Rust devs must be particularly smarter, as opposed to say PHP or JavaScript devs who are considered dumber. I don’t buy it, I need to see more than just your affiliations.

So, I reject your initial post contending ‘The technology stack isn't even that relevant.’ It was precisely the tech decisions that mattered, and the right people to make such decisions. Chicken and egg scenario, I’ll concede that.

In any case, one does not simply pick a old chat server written in Erlang out of the blue - this decision was critical. How many over-funded tech teams would try to do this from scratch in Go? Plenty, and that whole team would easily be full of ‘smart’ people.

Re: WhatsApp scaled to 1B users with only 50 engineers

#129

Earlier quoted context omitted.

Why is this being downvoted? Whatsapp is notoriously "just" a highly-optimized fork of ejabberd [0]. ejabberd is written in Erlang/OTP which enables quasi-horizontal scaling and amazing failure recovery, along with incredible uptime. What's different about Whatsapp compared to a traditional XMPP experience is: - it uses a single centralized server, without federating - it uses phone numbers as identifiers instead of…

Doesn't anybody remember how literally Whatsapp's founder wrote on one of the ejabberd mailing lists with something like "I just installed the server, please help me configure authentication" or the like? Retrospectively that has always colored my perception of how relevant technical aspects are for a successful startup (i.e. not much). Marketing (and viral, user-capturing & monopolist practices) are (sadly) much mor…

> how relevant technical aspects are for a successful startup (i.e. not much)

That's why i'm very interested in human and political aspects of free software. Successful FLOSS projects usually have a strong community backing them, and strong connections between developers and the community so that the technology isn't too disconnected from practical needs and UX concerns.

I think non-profits and cooperatives building upon free-software solutions are a better alternative model. For example, framasoft.org (french NGO for libre culture/software) has been instrumental in developing new solutions like Peertube, and in kickstarting a new wave of hosting coops (chatons.org federation).

In the IM world specifically, i'm very interested about Snikket.im project. It aims to build a complete client/server distribution of XMPP software tailored for specific user expectations.

Re: WhatsApp scaled to 1B users with only 50 engineers

#130
post #107
post #16

"The simpler product makes it much easier to maintain and scale." This is good news for the developers, the product owners and the customers. I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped. As a developer you need to support this in the future. It might be nice to build today but it will be a n…

WhatsApp was built on Erlang and FreeBSD, which feels way more exotic to me

FreeBSD has been released in 1993 and has been a well known OS in the hosting world.

Erlang has been released in 1986 and has been a well known language for message systems.

When I think about exotic I think about some new tech that sound really nice but is still very unproven.

Post reply on HN