Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

engineering.fb.com

Post reply on HN
Print view View original

261–270 of 333 posts

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#261

Post by ploxiln »

fulafel wrote: post #238
millstone wrote: post #12

From https://engineering.fb.com/data-infrastructure/messenger/ : Rather than reinventing the wheel, we used the UI framework available on the device’s native OS to support a wider variety of application feature needs. This reduced not only size, by avoiding the need to cache/load large custom-built frameworks, but also complexity. The native frameworks don’t have to be translated into sub-frameworks. We also used qui…

Well, they now have a single-platform application without all the platform-shared code. It's not like there's no tradeoff.

But look a bit further, it gets better ...

> For any cross-platform logic, we used an operating extension built in native C code, which is highly portable, efficient, and fast. We use this extension for anything OS-like that’s globally suboptimal, or anything that’s not covered by the OS. For example, all the Facebook-specific networking is done in C on our extension.

C was the universal donor 30 years ago, and it's the universal donor today!

If you look further down the article, it describes their extensive use of sqlite, and a layer around it for managing synchronization with FB servers, also written in C.

I write a lot of Python and a lot of Go, and I love C. So I enjoy imagining the subgroup of HN commenters who wail that Go is ignorant of the last 30 years of CS technology, or that no one should be allowed to write C these days, reading this.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#262

Post by hvidgaard »

tasogare wrote: post #257
hvidgaard wrote: post #246

Earlier quoted context omitted.

How in the name of everything did they manage to write 1.7M lines of code for a messenger app? That is an astonishingly large code base! We're talking about an instant messenger app with heaps of features, having a LoC footprint 3.5% of the size of modern Windows.

> Since then, we’ve added payments, camera effects, Stories, GIFs, and even video chat capabilities Simple: they added tons of crap nobody care. On this list, I only use video. Messenger is valuable because of the network effect, but feature wise 2003’s MSN was good enough.

None of that justifies 1.7 million LoC.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#263

Post by jcelerier »

ex3ndr wrote: post #226
Turing_Machine wrote: post #141

Earlier quoted context omitted.

> But, all that being said, it still has a ton of features. It's not your father's IRC client sending text around and failing to transfer files through NAT/firewalls. Meh. It displays text, pictures, and videos. That's about it. This is just Messenger we're talking about here, not the full-blown Facebook app. The only state to keep track of is "Have you received message UUID?". > And they also had their own UI framew…

Dont underestimate complexity. For android you have to ship basically ffmpeg for correct mp4 encoding, on all platforms you have to carry some library on top of classical ui framework to make it just not suck at performance, you have to get your own json parser, yes, since built-in one will be 10x slower, you may be want to carry even low level QUIC library for networking and websockets, etc, etc. All this also not i…

> For android you have to ship basically ffmpeg for correct mp4 encoding

hopefully they aren't counting ffmpeg in their LOC count, right ? else they may as well count the lines of code of the C and C++ standard library, V8, etc - this would be a very useless metric

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#264

Post by Cthulhu_ »

reaperducer wrote: post #66
millstone wrote: post #12

From https://engineering.fb.com/data-infrastructure/messenger/ : Rather than reinventing the wheel, we used the UI framework available on the device’s native OS to support a wider variety of application feature needs. This reduced not only size, by avoiding the need to cache/load large custom-built frameworks, but also complexity. The native frameworks don’t have to be translated into sub-frameworks. We also used qui…

They make it sound like the clouds parted and a ray of sunshine struck the programmers at Facebook and they realized that they didn't have to ride the framework-go-round to hell. These are supposed to be the smartest people in SV, but from the outside, it strikes me more as a "No shit, Sherlock" moment.

Oh yeah, this is a problem with a lot of developers, reinventing the wheel because the native JSON parser might be improved upon. I mean it's a clear case of NIH, and yet it's also defensible because I'm sure most native / builtin JSON parsers are not as fast as a custom solution. And for an app like Messenger, moving to something else like grpc would be very defensible.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#265

Post by Cthulhu_ »

lwb wrote: post #109
aldanor wrote: post #85

So, they've reduced 1.7M lines of code to 360K. Let's pause here for a moment: - NumPy is 360K lines mostly C - Postgres is around 2.1M of C - Go 1.13 is around 1.5M of Go code - Rust 1.37 is around 1.2M of Rust code iirc - core llvm is around 3M of C++ iirc ... A chat app is 1.7M. A chat app... Makes me feel sorry for all the time spent/wasted by all those devs, many of whom are unquestionably brilliant and could've…

Honestly, a chat app that 1 billion+ use per month that includes payments, camera effects, extensive social integrations, stories, GIFs, reactions, games, polls, voice recording, calling, video chat, and works across all mobile platforms, with nine years of technical debt, that thousands of developers build simultaneously seems WAY more complicated to me than any of your examples. Messenger may not have "HN clout" bu…

I'm on both sides of the fence on this one. On the one hand, it's people underestimating how much features are in the app that they've never seen, used or heard of. On the other, if that is the case then is their presence defensible?

I wonder if they could make apps more modular, and only load the binaries for e.g. payments on demand. I know Apple offers a system that does that for content chunks (e.g. levels and assets in games), and in a web-based application (e.g. react native) they could use bundling to remove it from the initial download.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#266

Post by Cthulhu_ »

jeswin wrote: post #224
ardit33 wrote: post #146

Earlier quoted context omitted.

it seem you don't have experience on large scale apps... and the people that make these type of comments almost never have had to work on large applications..... Spotify had about 500k, lines of code in 2015, (not including external libraries). We did make it in a feature based architecture, where there was a main container app (and a major core playback library) and for every major feature they had their own sub-pro…

Thank you for saying this. The overwhelming majority of comments on this thread are pedestrian - as if FBs engineers have been making decisions out of naivety, incompetence or self interest.

It might look that way in hindsight, but hindsight is easy. As is armchair theorizing about the competence of other developers.

Most people (including myself) will probably never see a >500K loc application or a project with >100 developers working on it. Biggest ones I've worked with had around 30-50 odd developers?

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#267

Post by blunte »

nneonneo wrote: post #171
butz wrote: post #76

The loop of app building: 1. Build app with native framework. 2. Re-build app with some alternative framework which is slower, but let's you deploy single code on all platforms. 3. Re-build app with native framework, because it is faster. 4. GOTO 2

The best part about this cycle is that developers can always point to some metric which shows improvement to justify their actions, resulting in an Escher-like loop where they think they’re always going up. These cycles pop up all the time in development as priorities shift (or as developers find more excuses to “fix” what isn’t broken). Real world coding always involves compromise - leadership is recognizing which c…

I don't know what world you live in, but in my world developers don't have to look for excuses to do work. Perhaps some companies running on endless VC overhire, but the real working world has fewer employees for most positions than it needs.

And in the case of Messenger, I highly doubt it was the developers who said, "Hey, let's add this completely unrelated thing to our codebase - payments!". More likely it was marketing or "product" that told the devs what to add.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#268

Post by dmitriid »

AWebOfBrown wrote: post #149

I thought Messenger was ReasonML's flagship use-case inside Facebook. If so, it will be interesting to know what impact this has on its development.

Web only. However, there haven't been any updates on that for over two years [1]

[1] https://reasonml.github.io/blog/2017/09/08/messenger-50-reas...

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#269

Post by eproxus »

reaperducer wrote: post #66
millstone wrote: post #12

From https://engineering.fb.com/data-infrastructure/messenger/ : Rather than reinventing the wheel, we used the UI framework available on the device’s native OS to support a wider variety of application feature needs. This reduced not only size, by avoiding the need to cache/load large custom-built frameworks, but also complexity. The native frameworks don’t have to be translated into sub-frameworks. We also used qui…

They make it sound like the clouds parted and a ray of sunshine struck the programmers at Facebook and they realized that they didn't have to ride the framework-go-round to hell. These are supposed to be the smartest people in SV, but from the outside, it strikes me more as a "No shit, Sherlock" moment.

But could it be that the OS functionality also evolved until today where it’s finally feasible to do this?

I’m guessing the oldest code in Messenger stems from a time when the OS-provided libraries were not up to par or simply didn’t exist yet. Partly because it’s not until now that the OS manufacturers finally see what there is demand for in apps.

Re: LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app

#270

Post by Jordanpomeroy »

Legogris wrote: post #165
SQLite wrote: post #93

Earlier quoted context omitted.

I did not. This is the first I've heard of it.

I have to ask, did they pay the 6k license fee yet?

I love the tongue-in-cheek licensing that SQLite offers.
Post reply on HN