Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backend of Meta Threads is built with Python 3.10

twitter.com

Post reply on HN
Print view View original

201–210 of 458 posts

Re: Backend of Meta Threads is built with Python 3.10

#201

Post by grishka »

anayak wrote: post #131
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

While that is technically correct, they haven't been running vanilla PHP for quite some time - it's jitted into x86 and run natively on the machine. It's also pretty extensively optimized. And the VM that does the jitting is all C++. Their MySQL deployment is also an internal fork that's been heavily modified for scalability (storage/compute separation, sharding, RocksDB based storage engine instead of InnoDB, Raft f…

Curiously, VKontakte also started from PHP+MySQL but went another way. PHP is compiled ahead of time with something called KittenPHP. It's open-source. For databases they switched some parts to their own bespoke solutions in C that talk with PHP over the memcached protocol. These are called KittenDB (or "engines") for the simple single-purpose ones, and there was also a more generic MySQL replacement in development when I left in 2016, MeowDB.

Re: Backend of Meta Threads is built with Python 3.10

#202

Post by jarjoura »

pjmlp wrote: post #179
DoesntMatter22 wrote: post #142

Earlier quoted context omitted.

Lol. Do you think you are just going to use go and everything is going to scale to FB level size? Facebook was already at a scale larger than 99.99999 percent of sites before they had HHVM

Remember when Twitter was written in Ruby?

LOL, at least Ruby inspired a new generation of languages that we have today.

Re: Backend of Meta Threads is built with Python 3.10

#203

Post by supernikio2 »

nikhizzle wrote: post #147
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

Earlyish facebook engineer here. Early days FB php was nothing like the php used to template websites. All kinds of specialized libraries to enable a much more sophisticated programming style. Think functional helpers, and asynchronous execution on thousands of cores, spanning trees across data centers using ssh etc. As a tangent a lot of the good stuff I used was written by Evan Priestley, who also did Phabricator a…

I'm of course speaking from an outsider's point of view, but do you think PHP was a good option, or was it a "fitting a square peg into a round hole" situation?

Re: Backend of Meta Threads is built with Python 3.10

#205

Post by DeathArrow »

coding123 wrote: post #107
taopai wrote: post #60

Earlier quoted context omitted.

Thanks for your comment. I'm really interested in this topic. How do you know the web server is Django? I searched but couldn't find this. Why would they use Django? I did some small projects in it but I assumed it wasn't very fast and wouldn't be suitable for a big app like this. I would like to know the pros and cons. Why didn't they build up something in C++ or Rust? Won't python limit the speed of responses despi…

Whether it's in C++ or Rust or Python, almost all of any slowness would be from database waiting anyway.

Not really. Look at Techempower web benchmarks.

Re: Backend of Meta Threads is built with Python 3.10

#206

Post by doteka »

winrid wrote: post #156
appplication wrote: post #99

Earlier quoted context omitted.

How is Django now? I’m a long time python dev (data eng space) but new-ish to web dev. I started a Flask project 2 years ago and found it to be pretty full of footguns, mixed messaging on best practices for scalable apps, and the ecosystem feels overbloated with vapor ware extensions. Is this just a Flask problem, or does Django have the same issues?

I can't give Django enough praise. I never thought I'd be writing Python. I've mostly used Node, Java, and Rust, with about 12yrs experience now. Only about one year with Python and Django recently. I am so much more productive than anything else I've tried. Using anything else feels like a mistake for web apps or CRUD apis. I also use type hints. I use Django + django-ninja + django-unicorn for dynamic UIs. Building…

You are also running sidewaysdata.com with DEBUG turned on in production, apparently ;)

Re: Backend of Meta Threads is built with Python 3.10

#207

Post by latexr »

DeathArrow wrote: post #196
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

Python is just fine for all the 100 users Threads will manage to acquire. :)

> all the 100 users

That’s off by multiple orders of magnitude, if you believe their reports of 30 million sign ups in under a day.

https://www.theverge.com/2023/7/6/23786108/threads-internal-...

Re: Backend of Meta Threads is built with Python 3.10

#208

Post by freeplay »

fleetfox wrote: post #75
kleton wrote: post #55

Really, all the performance intensive parts are in various c++ aggregator and recommendation type services. But the webserver is Django, yes.

Django is WSGI/ASGI framework not a webserver. What do they actually use to terminate HTTP?

Most likely a reverse proxy of some sort.

Re: Backend of Meta Threads is built with Python 3.10

#209

Post by seunosewa »

pookha wrote: post #177
ska wrote: post #122

Earlier quoted context omitted.

Is your problem that the c part of GDAL isn't concurrent and you are calling it multiple times, or are you blocking on something you shouldn't? Python calling a concurrent c thing is super common. For more complex things there is multiprocessing, but that essentially is simplification of IPC.

Had a massive amount of images that I was trying to process for a specific purpose. I wound up spinning up multiple python processes within a java wrapper using concurrent linked queues. Got the job done but it's not the proudest moment of my life.

Why did you need the Java wrapper? You can orchestrate stuff like that within Python. The multiprocessing module.

Re: Backend of Meta Threads is built with Python 3.10

#210

Post by Dah00n »

RamblingCTO wrote: post #15
mihaic wrote: post #13

Earlier quoted context omitted.

Why downvote this? It's a fair point.

Because it's not ironic. It's obvious. Twitter is just a medium to post stuff. Threads is as well. I can post this on mastodon even. Also: Europe still can't join in on the fun.

EU != Europe. Biggest country in Europe is Russia. Biggest population is Russians.
Post reply on HN