341–350 of 458 posts
If you know you’ve got gigantic scale why would you not use a compiled language? Surely it makes massive difference to hardware utilization? I like Python and javascript but there’s perfectly fine tools around now for building systems that allow one server to do a lot more. They say you should not prematurely optimize. Equally you should not be prematurely unoptimised.
If you want the real answer, it’s because Instagram was acquired and the startup used Django. That’s it.
Earlier quoted context omitted.
If you have hundreds of millions of daily active users then needing to write a compiler isn't the end of the world.
Compiling PHP into machine code sounds pretty much impossible, are they using some subset of the language? Does the compiled code use a garbage collector? Or reference counting?
Earlier quoted context omitted.
Facebook scaled to ~1b daily actives on MySQL + InnoDB. There was lots of engineering work, like schema sharding (denormalization), automation, plenty of bug fixes and patches for MySQL (most or all contributed back to upstream, from what I remember), and of course a massive caching layer; plus throwing crazy hardware at the problem. Nonetheless the underlying engine was something any MySQL user or admin would have r…
> And we backed it all up, every day, in But… how? Considering all the transactions in flight, and everything? And did you ever Test disaster recovery with that setup? I’ve worked on relatively big projects, but FAANG engineering is like some entirely different field of software engineering. Fascinating.
Ironic that a post about Threads backend is hosted on twitter.
If you want real irony here, it's this: powering an app called Threads with Python that's got a global interpreter lock which makes threading hard. Get it? Threads, threading...? OK, I'll see myself out.
> It's running on Instagram's #Cinder fork that includes a JIT, lazy-loaded modules, pre-compiled static modules, and a bunch of other interesting tweaks against vanilla Python 3.10. So not entirely just python 3.1.
Mind that Python 3.10 is nine major versions ahead of 3.1
For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.
Yep, Vanilla Python isn't fast enough so they have to do more: "It's running on Instagram's #Cinder fork that includes a JIT, lazy-loaded modules, pre-compiled static modules, and a bunch of other interesting tweaks against vanilla Python 3.10" From the Tweet. Personally, I don't care what language people use. We all know that if you have money, you can just throw in more servers. Until your CFO and CTO decided that…
This is a huge loss for the world, and makes me a bit angry.
Edit: I had falsely assumed that because GP missed out on the JITing etc. described in the linked tweet, that Twitter was still inaccessible to the open web. However, trying again now, Twitter is again viewable without an account. So Twitter's communication about needing an account to view it was lacking, but so was my diligence.
Earlier quoted context omitted.
If you have hundreds of millions of daily active users then needing to write a compiler isn't the end of the world.
Compiling PHP into machine code sounds pretty much impossible, are they using some subset of the language? Does the compiled code use a garbage collector? Or reference counting?
They're using a derived language called Hack.
> Does the compiled code use a garbage collector? Or reference counting?
Not sure but it's open source so I'm sure you can dig up the answers one way or another: https://github.com/facebook/hhvm
> It's running on Instagram's #Cinder fork that includes a JIT, lazy-loaded modules, pre-compiled static modules, and a bunch of other interesting tweaks against vanilla Python 3.10. So not entirely just python 3.1.
Mind that Python 3.10 is nine major versions ahead of 3.1
Earlier quoted context omitted.
Facebook had hundreds of millions daily active users before writing a compiler.
98% of the people that initially thought (Python for Threads - OMG performance!) work at a company that will tell you straight up they expect 100M RPS in a year or two, will actually only get to 50k RPS in 5 years before the "Our incredible journey" letter is issued.
Being able to handle spikes and iterate quickly is probably more important.
Earlier quoted context omitted.
Mind that Python 3.10 is nine major versions ahead of 3.1
:s/major/minor
Python doesn't go by semver rules. Semver isn't a definition for all software project versioning systems everywhere.