Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backend of Meta Threads is built with Python 3.10

twitter.com

Post reply on HN
Print view View original

131–140 of 458 posts

Re: Backend of Meta Threads is built with Python 3.10

#131

Post by anayak »

seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

While that is technically correct, they haven't been running vanilla PHP for quite some time - it's jitted into x86 and run natively on the machine. It's also pretty extensively optimized. And the VM that does the jitting is all C++. Their MySQL deployment is also an internal fork that's been heavily modified for scalability (storage/compute separation, sharding, RocksDB based storage engine instead of InnoDB, Raft for synchronous replication, etc.). Lots of really fantastic engineering work has gone into making their backend infra scale.

Some worthwhile references:

[1] https://engineering.fb.com/2016/08/31/core-data/myrocks-a-sp...

[2] https://research.facebook.com/file/529018501538081/hhvm-jit-...

[3] https://research.facebook.com/file/700800348487709/HHVM_ICPE...

Re: Backend of Meta Threads is built with Python 3.10

#132

Post by KRAKRISMOTT »

amethyst wrote: post #109
sgt wrote: post #96

Earlier quoted context omitted.

Interesting and lovely to hear they decided to use Django. What is your source, though?

It's a heavily modified internal fork of Django. Source: I work on our Python Language Foundation team.

Are you using the ORM/auth/admin features or is Django just a lightweight router? I can't imagine the ORM being too useful at Facebook's scale.

Re: Backend of Meta Threads is built with Python 3.10

#134

Post by that_guy_iain »

brrrrrm wrote: post #121
citrin_ru wrote: post #108

Earlier quoted context omitted.

I would argue a web server which processes all (or most) requests is a performance sensitive part of the system. Straightforward rewriting from Python to Go in many cases gives at least 2x-3x performance improvement which on FAANG scale can mean huge difference in hardware cost.

Developer velocity is extremely important at FAANG scale.

Is it really? Don't they work really slowly compared to most of the industry? I really get the impression that it's slow progress for things at FAANG.

Re: Backend of Meta Threads is built with Python 3.10

#135

Post by icelancer »

chippiewill wrote: post #133
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

PHP is actually shockingly fast for a non-JIT'd language these days.

PHP8 has a JIT now but I get what you mean.

Re: Backend of Meta Threads is built with Python 3.10

#136

Post by thewataccount »

appplication wrote: post #99
sgt wrote: post #96

Earlier quoted context omitted.

Interesting and lovely to hear they decided to use Django. What is your source, though?

How is Django now? I’m a long time python dev (data eng space) but new-ish to web dev. I started a Flask project 2 years ago and found it to be pretty full of footguns, mixed messaging on best practices for scalable apps, and the ecosystem feels overbloated with vapor ware extensions. Is this just a Flask problem, or does Django have the same issues?

You simply can't beat Django's ORM for general stuff. It's too awesome. This alone makes it so hard to choose anything else.

I know django doesn't have that "shiny factor" to it these days - but it's very reliable.

> mixed messaging on best practices for scalable apps

The WSGI stuff can be kinda confusing and is used across a lot of python frameworks including django and I think flask?

My advice for "simple scaling" is to start with a separate Postgres instance, and then use gunicorn. Use celery immediately if you have any "long lived" tasks such as email. If you containerize your web layer, you'll be able to easily scale with that.

Finally - use redis caching, and most importantly - put Nginx in front! DO NOT serve static content with django!

> the ecosystem feels overbloated with vapor ware extensions.

This still exists to some degree for some more niche stuff, largely because of it's age. Although impressively they'll generally still work or work with minimal modifications. It's popular enough and old enough that most normal things you'd want to do have decent extensions or built in support already.

Re: Backend of Meta Threads is built with Python 3.10

#137

Post by melling »

ctur wrote: post #93
melling wrote: post #92

Earlier quoted context omitted.

Is the person from Meta? Doesn’t appear so. It’s still perfectly acceptable for Twitter users to discus other platforms on Twitter.

He was and worked on, among other things, Python while there. He's a trustworthy source.

So, no he doesn’t work at Meta. We could have stopped this thread an hour ago.

We need a social network that removes the noise from the conversation

Re: Backend of Meta Threads is built with Python 3.10

#139

Post by bjourne »

appplication wrote: post #99
sgt wrote: post #96

Earlier quoted context omitted.

Interesting and lovely to hear they decided to use Django. What is your source, though?

How is Django now? I’m a long time python dev (data eng space) but new-ish to web dev. I started a Flask project 2 years ago and found it to be pretty full of footguns, mixed messaging on best practices for scalable apps, and the ecosystem feels overbloated with vapor ware extensions. Is this just a Flask problem, or does Django have the same issues?

Django is the opposite. It holds your hand and tells you how to structure your "app", templates, database, and media assets. And django-admin is godsend. It's awesome when you are on the beaten path cause everything just works. But if you step of the beaten path you will have to tweak various obscure settings and create weird hooks to override Django's defaults. Flask is the opposite. Just a very well-written web server and it's up to you to decide things like project structure, template engine, orm, asset manager, etc. Personally I prefer Django since most of my web applications are quite mundane and don't require the flexibility Flask can provide.
Post reply on HN