Live data from Hacker News

LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

engineering.fb.com

221–230 of 333 posts

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#221

I feel like I had an aneurysm reading this. For 6 paragraphs the repeat that they shrunk the lines of code down, that the rewrote the whole thing and that it’s a massive undertaking and that a message app should be small. 6 paragraphs of that slightly reworded and I just gave up. I honestly think this was written by a bot.

Also just LOOK at this empty state. 12 buttons. And that’s apparently stripped down. Yikes. https://imgur.com/gallery/drOgDwF

That’s not the new version. You haven’t received the update yet.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#222

Earlier quoted context omitted.

It seems you are correct: > We also used quite a few of the OS libraries, including the JSON processing library, rather than building and storing our own libraries in the codebase.

Doesn't that mean they're not using third-party libraries?

The 1.7m was pre-rewrite and the quote about using os libraries was post-rewrite; it doesn't prove they counted the libraries in the 1.7m metric but it does imply they bundled more before.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#223
> And yet, for the people using the app, it won’t look or feel much different.

Yikes. Okay, I know I'm maybe being a bit unfair here, but I'm sorry to say that this is one of the weakest engineering blog posts I've ever read. This seems to be a massive "let's end the craziness and finally pay our tech debt" effort. Good for them, but not something that excites me about working at Facebook.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#224
post #85

So, they've reduced 1.7M lines of code to 360K. Let's pause here for a moment: - NumPy is 360K lines mostly C - Postgres is around 2.1M of C - Go 1.13 is around 1.5M of Go code - Rust 1.37 is around 1.2M of Rust code iirc - core llvm is around 3M of C++ iirc ... A chat app is 1.7M. A chat app... Makes me feel sorry for all the time spent/wasted by all those devs, many of whom are unquestionably brilliant and could've…

it seem you don't have experience on large scale apps... and the people that make these type of comments almost never have had to work on large applications..... Spotify had about 500k, lines of code in 2015, (not including external libraries). We did make it in a feature based architecture, where there was a main container app (and a major core playback library) and for every major feature they had their own sub-pro…

Thank you for saying this. The overwhelming majority of comments on this thread are pedestrian - as if FBs engineers have been making decisions out of naivety, incompetence or self interest.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#225
post #76

The loop of app building: 1. Build app with native framework. 2. Re-build app with some alternative framework which is slower, but let's you deploy single code on all platforms. 3. Re-build app with native framework, because it is faster. 4. GOTO 2

Better than plan plan plan. It is the customer or user that use it that count. Not the programmer.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#226

Earlier quoted context omitted.

I can see it. It's pretty easy to hit that much code when you include your own code for things as trivial as a JSON parser rather than leveraging what the native SDK provides (which was alluded to in the article). It sounds like they had a bit of Not Invented Here syndrome going on and this rewrite addressed a lot of that. It also sounds like they had a bunch of systems (probably different teams even) doing similar t…

> But, all that being said, it still has a ton of features. It's not your father's IRC client sending text around and failing to transfer files through NAT/firewalls. Meh. It displays text, pictures, and videos. That's about it. This is just Messenger we're talking about here, not the full-blown Facebook app. The only state to keep track of is "Have you received message UUID?". > And they also had their own UI framew…

Dont underestimate complexity.

For android you have to ship basically ffmpeg for correct mp4 encoding, on all platforms you have to carry some library on top of classical ui framework to make it just not suck at performance, you have to get your own json parser, yes, since built-in one will be 10x slower, you may be want to carry even low level QUIC library for networking and websockets, etc, etc.

All this also not increasing complexity since built in one stuff makes your code much harder to read/write.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#227
post #193

I'm truly shocked how many resources are dedicated to maintaining a chat app that essentially just talks to an api. Seems like you could easily build and maintain a simple chat system with <5 developers. You wouldn't have shiny new features, but it would by coincidence be faster and simpler. Networked chat is a standard project in university.

> Seems like you could easily build and maintain a simple chat system with Yes you could. Note however 'simple chat system'.

To build a global messaging app used by hundreds of millions of people is a whole different ball game.

By comparison, before acquisition by Facebook, Whatsapp had a team of ~50 people, and that was considered lean. Based on that, it would seem you're about an order of magnitude off.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#228

From: https://engineering.fb.com/data-infrastructure/messenger/ > We accomplished this by using the native OS wherever possible Does this mean this application does not use React Native?

Did the previous application use React Native?

When I interned at FB in 2018, there was no RN in the Messenger codebase. However, it did make extensive use of various FB UI abstractions like Yoga layout and ComponentKit.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#229
The technical meat of this post is below. It looks like they mostly eliminated functionality in their app code wherever it was redundant with either OS functionality, SQL Lite capabilities, or the Messenger backend servers at Facebook's datacenters.

>One of our main goals was to minimize code complexity and eliminate redundancies. ... To build this unified architecture, we established four principles: Use the OS, reuse the UI, leverage the SQLite database, and push to the server.

>We accomplished this by using the native OS wherever possible, reusing the UI with dynamic templates powered by SQLite, using SQLite as a universal system, and building a server broker to operate as a universal gateway between Messenger and its server features.

>... the existing OS often does much of what’s needed. Actions like rendering, transcoding, threading, and logging [and JSON processing] can all be handled by the OS.

>To simplify and remove redundancies, we constrained the design to force the reuse of the same [UI] structure for different [UI] views. So we needed only a few categories of basic [UI] views, and those could be driven by different SQLite tables.

>Now, ... All the caching, filtering, transactions, and queries are all done in SQLite. The UI merely reflects the tables in the database.

>We developed a single integrated schema for all features. We extended SQLite with the capability of stored procedures, allowing Messenger feature developers to write portable, database-oriented business logic, and finally, we built a platform (MSYS) to orchestrate all access to the database, including queued changes, deferred or retriable tasks, and for data sync support.

>MSYS is a cross-platform library built in C that operates all the primitives we need. ... With MSYS, we have a global view. We’re able to prioritize workloads. Say the task to load your message list should be a higher priority than the task to update whether somebody read a message in a thread from a few days ago; we can move the priority task up in the queue.

>With MSYS, it’s easier to track performance, spot regressions, and fix bugs across all these features at once. In addition, we made this important part of the system exceptionally robust by investing in automated tests, resulting in a (very rare in the industry) 100 percent line code coverage of MSYS logic.

>For anything that doesn’t fit into one of the categories above, we push it to the server instead. We had to build new server infrastructure to support the presence of MSYS’s single integrated data and sync layer on the client.

>Coordinating logic between client and server is very complex and can be error-prone — even more so as the number of features grows. ...

>Similar to MSYS on the client, we built a server broker to support all these scenarios while the actual server back-end infrastructure supports the features.

>[To minimize code-base growth] We also built a system that allows us to understand how much binary weight each feature is bringing in. We hold engineers accountable for hitting their budgets as part of feature acceptance criteria. Completing features on time is important, but hitting quality targets (including but not limited to binary size budgets) is even more important.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#230

Earlier quoted context omitted.

Ordering messages and providing reliability does not require 1.7M lines of code.

Code wins arguments. FB messenger beats everything, Whatsapp, Wire, Signal, WeChat, Telegram included, in the suckiest network connections.

If FB Messenger beat Whatsapp they wouldn’t have had to buy it.
Post reply on HN