Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backend of Meta Threads is built with Python 3.10

twitter.com

Post reply on HN
Print view View original

211–220 of 458 posts

Re: Backend of Meta Threads is built with Python 3.10

#211

Post by Tade0 »

citrin_ru wrote: post #108
lopkeny12ko wrote: post #80

Python is just the web server. All the truly performance-sensitive components--edge proxy, load balancers, backend services, databases, caches, and storage services--are mostly C++.

I would argue a web server which processes all (or most) requests is a performance sensitive part of the system. Straightforward rewriting from Python to Go in many cases gives at least 2x-3x performance improvement which on FAANG scale can mean huge difference in hardware cost.

It's probably cached to seven hells, which makes that 2-3x improvement much less relevant.

Wordpress wouldn't be this successful if this approach didn't work. After all, without caching it will happily take over 20 seconds to render the main page.

Re: Backend of Meta Threads is built with Python 3.10

#212

Post by amethyst »

PartiallyTyped wrote: post #195
amethyst wrote: post #109

Earlier quoted context omitted.

It's a heavily modified internal fork of Django. Source: I work on our Python Language Foundation team.

What exactly do you do? This sounds like a JIT/Interpreter/Language dev job. Is this correct? If so, do you have any recommendations or suggestions for someone getting their feet wet on this?

It's mostly foundational work around developer tooling and infrastructure that isn't already covered by other dedicated teams (eg, there is a dedicated Cinder team). My latest work has focused around formatting and linting, and includes open source work on µsort, our import sorter [1], and fixit, our custom linter [2] that makes it easy for teams to build and deploy their own custom lint rules with autofixes.

Some of my team mates work on driving internal adoption of Cinder, rolling out new versions of Python everywhere, improvements to our build and packaging systems, supporting other Python tooling teams, etc. There's a lot of cross-functional project work, and our primary goal is to improve the Python developer experience, both internally and externally wherever we can.

1: https://usort.rtfd.io

2: https://fixit.rtfd.io

Re: Backend of Meta Threads is built with Python 3.10

#213

Post by patrec »

patrec wrote: post #186
coding123 wrote: post #107

Earlier quoted context omitted.

Whether it's in C++ or Rust or Python, almost all of any slowness would be from database waiting anyway.

It wouldn't. Databases are fast, python is slow.

https://techspot.zzzeek.org/files/2015/pymysql_runsnake.png

Re: Backend of Meta Threads is built with Python 3.10

#214

Post by nikhizzle »

iknowSFR wrote: post #164
nikhizzle wrote: post #147

Earlier quoted context omitted.

Earlyish facebook engineer here. Early days FB php was nothing like the php used to template websites. All kinds of specialized libraries to enable a much more sophisticated programming style. Think functional helpers, and asynchronous execution on thousands of cores, spanning trees across data centers using ssh etc. As a tangent a lot of the good stuff I used was written by Evan Priestley, who also did Phabricator a…

Can you speak to the onboarding and learning curve in that environment? What was the training like? What kinds of timelines did you experience regarding expectations as a new hire? Asking because I’m curious about these early stage mega-tech companies that are working at a scale I can’t fathom.

Six week boot camp, you are pretty fluent by week 4. Notably though this was before mega tech days, we had under 100 engineers.

Re: Backend of Meta Threads is built with Python 3.10

#215

Post by nikhizzle »

supernikio2 wrote: post #203
nikhizzle wrote: post #147

Earlier quoted context omitted.

Earlyish facebook engineer here. Early days FB php was nothing like the php used to template websites. All kinds of specialized libraries to enable a much more sophisticated programming style. Think functional helpers, and asynchronous execution on thousands of cores, spanning trees across data centers using ssh etc. As a tangent a lot of the good stuff I used was written by Evan Priestley, who also did Phabricator a…

I'm of course speaking from an outsider's point of view, but do you think PHP was a good option, or was it a "fitting a square peg into a round hole" situation?

It was not a deliberate choice, just it was what the site was built in, so people like Evan and Marcel (Laverdet) made php beautiful for us.

Re: Backend of Meta Threads is built with Python 3.10

#216

Post by gen220 »

appplication wrote: post #99
sgt wrote: post #96

Earlier quoted context omitted.

Interesting and lovely to hear they decided to use Django. What is your source, though?

How is Django now? I’m a long time python dev (data eng space) but new-ish to web dev. I started a Flask project 2 years ago and found it to be pretty full of footguns, mixed messaging on best practices for scalable apps, and the ecosystem feels overbloated with vapor ware extensions. Is this just a Flask problem, or does Django have the same issues?

I've used Flask/SQLAlchemy and Django a bunch in my day, professionally and on the side.

It's a different strokes for different folks sort of deal.

Django is very opinionated, in a way that is "eventually" correct (i.e. they might have had some bad opinions many years ago, but they have generally drifted in a better direction). If the opinions don't align with what you're building, the escape hatches are not generally well-documented or without penalty.

Flask is minimalist and flexible. Once you find a "groove" to building with it that fits your sensibilities, it's quite, quite nice. That being said, the most recent versions of flask have excellent documentation, imo, and the tutorial is a bit more opinionated in a highly productive way. The "patterns" section of the docs is also super useful for productionizing your app.

Personally, I prefer the combo of Flask+SQLAlchemy, and eventually Alembic once you decide that's good for your app's maturity level. I respect Django a lot, I just enjoy the explicitly "less magical" aspects of a Flask stack, which is an opinion-based trade-off imo.

Re: Backend of Meta Threads is built with Python 3.10

#217

Post by DeathArrow »

mrweasel wrote: post #188
murkt wrote: post #176

Earlier quoted context omitted.

Ehh, Django ORM always was one of the worst parts of Django.

True, but it's the best ORM currently available for any language. It might not be the fastest, but it is the one that's has the highest level of developer comfort. Using Django is probably the reason why I can stand using SQLAlchemy, it's way to complicated for everything I do and it's just not a nice an experience.

>True, but it's the best ORM currently available for any language.

Better than Entity Framework? Consider me impressed.

Re: Backend of Meta Threads is built with Python 3.10

#218

Post by troupo »

supernikio2 wrote: post #203
nikhizzle wrote: post #147

Earlier quoted context omitted.

Earlyish facebook engineer here. Early days FB php was nothing like the php used to template websites. All kinds of specialized libraries to enable a much more sophisticated programming style. Think functional helpers, and asynchronous execution on thousands of cores, spanning trees across data centers using ssh etc. As a tangent a lot of the good stuff I used was written by Evan Priestley, who also did Phabricator a…

I'm of course speaking from an outsider's point of view, but do you think PHP was a good option, or was it a "fitting a square peg into a round hole" situation?

> but do you think PHP was a good option, or was it a "fitting a square peg into a round hole" situation?

Facebook was founded in 2004. Whatever fancy tech you're thinking that would fit the round hole, it didn't exist then. They also never thought it would serve a couple billion of users one day, so they built it in whatever they had and knew at the time.

Same for Instagram and Python (they started with Django).

Re: Backend of Meta Threads is built with Python 3.10

#220

Post by tredre3 »

Asooka wrote: post #16
TheAdamist wrote: post #9

Ironic that a post about Threads backend is hosted on twitter.

Threads is a mobile-only app, like Instagram. You can't actually use Threads on a desktop browser, so that's probably why this is shared as a Twitter post. Even if it were a Threads post, it would not be submittable to HN.

Instagram launched in 2010 (but really gained traction only in 2011). It has been available via the web since 2012 but it took until 2015 for the web version to reach feature parity with the app version (namely posting and discovery).

Based on that, I think it's safe to assume that a full featured web version of Threads should be up within a year. Threads already has some web presence (you can view threads and profiles).

Post reply on HN