Honestly, the only kind of re-writes I respect are when someone decides to rewrite something written in the hot new tech and uses 2005-era PHP instead. I recommend solid tools that have been proven to scale to billions of users.
Backend of Meta Threads is built with Python 3.10
391–400 of 458 posts
Re: Backend of Meta Threads is built with Python 3.10
#392Earlier quoted context omitted.
Guess I'll just write my next project in Bash then, after all, if it's a good idea they will come right? Execution can wait until you've got a few hundred mil.
>Guess I'll just write my next project in Bash then, after all, if it's a good idea they will come right? Execution can wait until you've got a few hundred mil. Yea actually. Bash/CGI could handle plenty. If thats your most comfortable language then go for it.
Bash is seriously underrated as a programming language these days.
About 80% of the code I now write is in bash, because usually the problem ones trying to solve has a trivial solution if you just use existing tooling in creative ways.
Re: Backend of Meta Threads is built with Python 3.10
#393Earlier quoted context omitted.
It's a heavily modified internal fork of Django. Source: I work on our Python Language Foundation team.
Are you using the ORM/auth/admin features or is Django just a lightweight router? I can't imagine the ORM being too useful at Facebook's scale.
I believe the django app would use a python version of the internal ent ORM. You can get a sense for the style of ent/node from this external fork that was written while the author had access to the original ent https://entgo.io/docs/getting-started
Re: Backend of Meta Threads is built with Python 3.10
#394Earlier quoted context omitted.
Facebook scaled to ~1b daily actives on MySQL + InnoDB. There was lots of engineering work, like schema sharding (denormalization), automation, plenty of bug fixes and patches for MySQL (most or all contributed back to upstream, from what I remember), and of course a massive caching layer; plus throwing crazy hardware at the problem. Nonetheless the underlying engine was something any MySQL user or admin would have r…
> And we backed it all up, every day, in But… how? Considering all the transactions in flight, and everything? And did you ever Test disaster recovery with that setup? I’ve worked on relatively big projects, but FAANG engineering is like some entirely different field of software engineering. Fascinating.
To expand on this question, I'm wondering how useful daily backups even are for a site like Facebook. I mean, of _course_ you need them, but also, something about reverting all of FB to a state 24 hours ago seems disastrous even if it works. I can't imagine that it's an acceptable thing in anything but an absolute emergency. Imagine every single facebook user got rewound in time to the previous day, every message sent over the past day was lost, etc.
It was a lifetime ago I ever did DB administration (postgres in my case), but the write-ahead-logs being replicated out independently was extremely important for point-in-time recovery, such that you could always take the latest backup, zip forward through the WAL, and recover to any arbitrary point in time you want, so long as the WALs were available. I wonder how much something like this would have been done at FB scale.
Re: Backend of Meta Threads is built with Python 3.10
#395Earlier quoted context omitted.
:s/major/minor
No, I meant major. Each of the 3.x releases brings in new features, deprecations, and even sometimes removed features. They are all major releases. Python doesn't go by semver rules. Semver isn't a definition for all software project versioning systems everywhere.
You are factually wrong.
"To clarify terminology, Python uses a major.minor.micro nomenclature for production-ready releases. So for Python 3.1.2 final, that is a major version of 3, a minor version of 1, and a micro version of 2.
* new major versions are exceptional; they only come when strongly incompatible changes are deemed necessary, and are planned very long in advance;
* new minor versions are feature releases; they get released annually, from the current in-development branch;" — https://devguide.python.org/developer-workflow/development-c...
Re: Backend of Meta Threads is built with Python 3.10
#396Threads is a repurposing of Instagram's backends / APIs. Instagram has been based on Python from the start. Therefore Threads happens to use Python. It doesn't mean that Python is the language that one should pick when building this sort of app given alternatives that don't require an in-house fork. The full post makes this reasonably easy to grok but I'm sure the hacker news "title-only" UI is misrepresenting.
Sheesh, relax, dude. Python is a perfectly good language.
Huh?
Re: Backend of Meta Threads is built with Python 3.10
#397Earlier quoted context omitted.
[flagged]
> This may be shocking to you, but _all_ code is effectively compiled to "machine code" eventually. I'm not sure you understood the point. It means nothing to claim that deep down it's all opcodes or electrons flowing. What matters is being able to transform the code targeting the high level interface to the lowest level interface in a way that remains usable and regression-free. Sometimes compilers for widely popula…
I don't think you understood the comments above, but I definitely don't understand your point...
Compilers introduce bugs, yes. Compilers are code, and code can have bugs and regressions.
But the comments above make no mention of bugs are regressions. They said compiling php into machine code is "impossible."
Going form high level interfaces to lower level is the easy part. It's what all interprets and compilers do already.
Having different intermediate forms is also a very common practice. There is nothing special about php here. Except for maybe it's underserved reputation of being an obtuse language.
Re: Backend of Meta Threads is built with Python 3.10
#398Earlier quoted context omitted.
So what? A company is trying to streamline their process and save costs? Is there any large company at all that hasn't invested in making their tool chain better? You talk about it like they are a failure. Lol
Costs that wouldn't have happened in first place if a scripting language hadn't been chosen in first place.
What we can’t know is whether they would have been fast enough to tweak their product to their current market without using a scripting language. Companies who do that aren’t sacrificing performance for nothing, scripting languages make a lot of things simpler and closer to the business domain, which cannot be ignored (anybody who’s in a huge project that requires a 30 min. compilation often enough to kill productivity knows what I’m talking about).
Besides, you can even use optimized C and get none of the benefits (but still all the drawbacks) because of the algorithms you’re using or your database or something else entirely such as microsservices etc. making differences in language speed negligible.
I guess the point is the language speed on backends don’t matter as much as many other things (network, database, architecture setup etc.) and usually the gains you get with your language being faster aren’t high unless you’re in a tight loop or you’re like Amazon where even a few milliseconds gain is more money.
Re: Backend of Meta Threads is built with Python 3.10
#399Earlier quoted context omitted.
Compiling PHP into machine code sounds pretty much impossible, are they using some subset of the language? Does the compiled code use a garbage collector? Or reference counting?
[flagged]
Such a pedantic response, the archetypical HN comment, I can sense the pretentious smirk on your face while writing it, thanks. Words still mean something, we don't compile a dynamically typed, interpreted scripting language such as PHP for a reason and as it turns out Facebook didn't "compile PHP" either but a dialect of the language.
Re: Backend of Meta Threads is built with Python 3.10
#400Earlier quoted context omitted.
I have ported code from Cinder to CPython . The fork has some optimizations that can be easily put in CPython and Facebook is open to port features. I’m not sure if Facebook wants to continually have a fork but CPython is open to have those features merged in if they make sense.
The Cinder team's longterm goal is to upstream as much as we can, and make the rest available as pip-installable extensions that anyone could theoretically install and use on CPython. The fewer internal changes and patches we need to maintain, the faster we can adopt upstream Python releases and all of the associated performance and tooling wins.
I've been following it with updates from Talk Python To Me and Python Bytes.
Great work, much appreciated.