Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backend of Meta Threads is built with Python 3.10

twitter.com

Post reply on HN
Print view View original

301–310 of 458 posts

Re: Backend of Meta Threads is built with Python 3.10

#301

Post by smrtinsert »

Yoric wrote: post #18
blitzar wrote: post #11

Earlier quoted context omitted.

Yet here it is ... powering a global scale app. Too weak for someones hobby project, strong enough for global scale.

My few years of personal experience of using Python with even small-ish scale professional projects have been... underwhelming so far. I wonder how Meta (and Google, and all the others who write serious Python code) manage to keep things running – and I'm not talking (only) of performance, but largely of bugs. My personal suspicion is that the secret sauce is simply to have a sufficient budget to hire an army of code…

The secret sauce is Intellij and some community knowledge. There's a library for everything and a great IDE makes it pretty painless and fun. Not sure I'd always jump for it thanks to the GIL but I wouldn't mind a job where it's my primary instead of my secondary or third language.

The biggest pain with it is of course refactoring. It's tedious, but I think still comes out ahead in terms of productivity for high level stuff.

Re: Backend of Meta Threads is built with Python 3.10

#302

Post by czardoz »

feydaykyn wrote: post #229
czardoz wrote: post #222

I worked on Cinder, and on the web server. Happy to answer technical questions if any :)

Hi, Would you recommend using Cinder for stack made of - Django - Cython - Numpy Thanks!

If this stack is heavily skewed towards numerical computations, Cinder might require a lot of tuning to be effective.

Re: Backend of Meta Threads is built with Python 3.10

#303

Post by roflyear »

shaftoe444 wrote: post #191

> It's running on Instagram's #Cinder fork that includes a JIT, lazy-loaded modules, pre-compiled static modules, and a bunch of other interesting tweaks against vanilla Python 3.10. So not entirely just python 3.1.

Python is a spec not an implementation, for what it is worth.

Re: Backend of Meta Threads is built with Python 3.10

#304

Post by antod »

pookha wrote: post #117
ambivalence wrote: post #26

Earlier quoted context omitted.

If you want real irony here, it's this: powering an app called Threads with Python that's got a global interpreter lock which makes threading hard. Get it? Threads, threading...? OK, I'll see myself out.

Gotten burned by that. Have a python utility that calls something that's very CPU intensive (GDAL) and I literally have to wrap this python process in java for real concurrency. Not a python expert by any means but I'd love to know what other people do when this happens. I'm assuming it's something revolving around containers and messaging (blech)...

I thought Python calling a native lib was the one time you could multithread your way around the GIL if you were cpu limited? Are the GDAL bindings not releasing the lock or something?

Re: Backend of Meta Threads is built with Python 3.10

#305

Post by roflyear »

_fdsfs12123 wrote: post #296
thewataccount wrote: post #136

Earlier quoted context omitted.

You simply can't beat Django's ORM for general stuff. It's too awesome. This alone makes it so hard to choose anything else. I know django doesn't have that "shiny factor" to it these days - but it's very reliable. > mixed messaging on best practices for scalable apps The WSGI stuff can be kinda confusing and is used across a lot of python frameworks including django and I think flask? My advice for "simple scaling"…

> Use celery immediately if you have any "long lived" tasks such as email Hey, quick question from a relative newbie who is currently trying to solve this exact problem. Besides Celery, what are good options for handling long-running requests with Django? I see 3 options: - Use Celery or django Q to offload processing to worker nodes (how do you deliver results from the worker node back to the FE client?) - Use a lib…

Use anything except Celery, is my vote. Even if that "anything" is something you roll yourself.

Celery is mature, but has bitten me more than anything else.

For scheduling, there are many libraries, but it's good to keep this separate from Celery IMO.

For background tasks, I think rolling your own solution (using a communication channel and method tailored to your needs) is the way to go. I really do at this point.

It definitive is not using async, I think that will bite you and not be worth the effort.

Huey is worth a look.

Re: Backend of Meta Threads is built with Python 3.10

#306

Post by baq »

ineedasername wrote: post #259
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

Pfft. It would be 100x more performant if they'd written it in asm.

They also wouldn’t release this year, perhaps neither next

Re: Backend of Meta Threads is built with Python 3.10

#307

Post by justsocrateasin »

mrcwinn wrote: post #293

I’m building a tiny little product for trail runners and cyclists in Python 3.10 (Flask) at https://www.outandback.com . Python has been a lot of fun to pick up (I rely heavily on pandas and some ML libraries so it felt like the right stack).

Could you describe your tech stack more?

I'm a data engineer trying to teach myself React so I can build a full stack app using some APIs and it's certainly a good exercise, but I could also certainly build that in Python. Your app looks great and I'd like to explore that more!

P.S - fellow runner and cyclist and out and I'm checking out the app more now.

Re: Backend of Meta Threads is built with Python 3.10

#308

Post by roflyear »

mrweasel wrote: post #188
murkt wrote: post #176

Earlier quoted context omitted.

Ehh, Django ORM always was one of the worst parts of Django.

True, but it's the best ORM currently available for any language. It might not be the fastest, but it is the one that's has the highest level of developer comfort. Using Django is probably the reason why I can stand using SQLAlchemy, it's way to complicated for everything I do and it's just not a nice an experience.

Have you used SQLAlchemeny and PeeWee extensively? I have found these to be pretty fair when compared to Django.

Anyway, you can use the Django ORM without Django :)

Re: Backend of Meta Threads is built with Python 3.10

#309

Post by baq »

DeathArrow wrote: post #199
that_guy_iain wrote: post #130

Earlier quoted context omitted.

For me, it does seem really strange since PHP (currently) is faster than Python and PHP wasn't fast enough for Facebook. But it's interesting to see how it'll play out.

Probably Python is deemed as more appealing to hipsters than good old PHP.

Nothing good about old php. Been doing php in 2004 and Django in 2009. Both were shit by todays standards but php was exceptionally so.

Re: Backend of Meta Threads is built with Python 3.10

#310

Post by DrBenCarson »

callamdelaney wrote: post #264
andrewstuart wrote: post #258

If you know you’ve got gigantic scale why would you not use a compiled language? Surely it makes massive difference to hardware utilization? I like Python and javascript but there’s perfectly fine tools around now for building systems that allow one server to do a lot more. They say you should not prematurely optimize. Equally you should not be prematurely unoptimised.

Because Zuck probably told his team to make this a few weeks ago to annoy Musk. Weird ego thing going on. Python is likely faster to push out an app like this than say rust or even go. Also why spend time optimising something that will be dead in a few months?

Dead in a few months? Lol this isn’t a Google product

Instagram stories crushed Snap’s DAU, Reels is gobbling up TikTok user-minutes…bleeding other services is what Meta excels at

Post reply on HN