Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backend of Meta Threads is built with Python 3.10

twitter.com

Post reply on HN
Print view View original

431–440 of 458 posts

Re: Backend of Meta Threads is built with Python 3.10

#431

Post by ReflectedImage »

pdimitar wrote: post #419
balder1991 wrote: post #398

Earlier quoted context omitted.

What we can’t know is whether they would have been fast enough to tweak their product to their current market without using a scripting language. Companies who do that aren’t sacrificing performance for nothing, scripting languages make a lot of things simpler and closer to the business domain, which cannot be ignored (anybody who’s in a huge project that requires a 30 min. compilation often enough to kill productivi…

> What we can’t know is whether they would have been fast enough to tweak their product to their current market without using a scripting language. Exactly. And I am saying the following as a fan of dynamic languages but let's be realistic: this is practically their ONLY true benefit -- quicker time to market. They lose out on pretty much any other metric, with the exception of also slightly quicker iteration in the…

In a startup, time to market is everything.

Re: Backend of Meta Threads is built with Python 3.10

#432

Post by ddorian43 »

anayak wrote: post #131
seanp2k2 wrote: post #110

For the “Python isn’t fast enough for production backend” crowd from the same company that brought you the largest social network built on PHP + MySQL.

While that is technically correct, they haven't been running vanilla PHP for quite some time - it's jitted into x86 and run natively on the machine. It's also pretty extensively optimized. And the VM that does the jitting is all C++. Their MySQL deployment is also an internal fork that's been heavily modified for scalability (storage/compute separation, sharding, RocksDB based storage engine instead of InnoDB, Raft f…

> storage/compute separation

Have they really done this for OLTP workloads running in mysql? I know they do it for OLAP though.

Re: Backend of Meta Threads is built with Python 3.10

#433

Post by Dah00n »

RamblingCTO wrote: post #374
Dah00n wrote: post #210

Earlier quoted context omitted.

EU != Europe. Biggest country in Europe is Russia. Biggest population is Russians.

Yeah, that's up for debate. It's not as clear as you make it out to be. As a (EU) European, Russia is not. The western most part of it maybe, yeah. Same for Turkey.

No, it really isn't at all.

European Russia is 40% of the total European landmass. 15% of its total population. That's 110 million Russians in only the part that is inside Europe! Germany is at a low 80 million. Total population of EU is 445 million. Besides, numbers doesn't change because you are from the EU and disagree. I'm also in the EU.

Re: Backend of Meta Threads is built with Python 3.10

#434

Post by pdimitar »

DoesntMatter22 wrote: post #429
pdimitar wrote: post #423

Earlier quoted context omitted.

Yet it's still super behind tech like Elixir or Golang, which are both faster and give you more compile-time guarantees.

Behind how? There are no good web frameworks for Golang at all, and relatively few sites being built with it. Elixir? Okay, it's a nice language that is also dynamically typed. It has some advantages but good luck finding anyone with knowledge of how to program in it or has even used a functional language before. I don't personally use Ruby these days but to say it's super behind is just silly. Golang is as similar t…

People are starting to recognize benefits of even gradual typing en masse these days, while some of us knew it for 10+ years. Not to be an elitist, I've made plenty of other mistakes and I am trying to not look down on anyone, but to discount types is not a well-informed take IMO.

That's what I mostly meant by saying Ruby is behind Elixir and Golang. And mind you, Elixir is strongly but dynamically typed and I still find it much better than Ruby.

Re: Backend of Meta Threads is built with Python 3.10

#435

Post by pdimitar »

DoesntMatter22 wrote: post #428
pdimitar wrote: post #421

Earlier quoted context omitted.

Why resort to a straw man? There are plenty of options between "a language that does not even have strong types" and "machine code"? E.g. Elixir is strongly but dynamically typed and you can take it quite far while reaping most of the benefits of typing and not losing velocity as you would with many statically strongly typed languages. Nuance matters. I wouldn't pick Ruby for anything except scripts nowadays. You get…

It's not a straw man, I'm asking him where he stands. How far does he want to take it? You mentioned Elixir. You literally suggested a language that was created 6 years after Shopify began. You wouldn't pick Ruby, thats your choice. People are still making billions of dollars with it. So, since you took up the mantle, what language should they have used 17 years ago when they started the company?

> You wouldn't pick Ruby, thats your choice. People are still making billions of dollars with it.

Not arguing with their results here. I am saying that they made it work despite how terrible it is. You have no guarantees about anything and you have to have much more tests than you would have in other languages just so you have basic stuff ensured. Not cool at all and I am saying this as a guy who made good money with Rails for 6.5 years. Not impressed to this day and I am happy I left it behind. It's good for scripting I'll admit, though nowadays I just learned bash/zsh better and use Golang for the same task occasionally.

> what language should they have used 17 years ago when they started the company?

Whoops, you are 100% correct that I ignored the historical timeline (Elixir didn't exist back then, yes), sorry about that, I derped big time.

That being said, all of PHP, Ruby, Python, Java and C# were indeed valid choices back then. But I've been part of successful rewriting efforts (biggest one was about 270k coding lines which I'll admit is much smaller than what Facebook and other big corps are dealing with) and to me the downsides of rewriting are overplayed:

- You need extensive tests? Well, you need them even if you never rewrite.

- You will now have two systems? So what, you already have a load balancer, you will just put a few more rules in it.

- Hard to find engineers for $new_language? That's true if you do it in its first 5 years of life but I've been part of growing ecosystems twice (Elixir and Rust) and it only gets easier with time. Solution: don't be a super early adopter, that's obviously too risky. E.g. both Elixir and Rust are beyond 10 years old at this point and are now a safer choice.

- Have to duplicate the old system's bugs? Tough nut to crack and I partially agree with this one but what my previous teams did was write down these bugs in the docs and made sure to start fixing them after the rewrite was completed. And in all cases fixing the bugs in the original language was planned anyway but was eternally kicked down the road.

--

My main point here is that it's OK to admit at one point that the tech you used for a while has outgrown its usefulness. I understand and recognize some of the downsides but to me they are overblown.

Re: Backend of Meta Threads is built with Python 3.10

#436

Post by pdimitar »

DoesntMatter22 wrote: post #430
pdimitar wrote: post #420

Earlier quoted context omitted.

Elixir, Golang, maybe even Rust (that one is a hard sell though, there could be a lot of friction at the start and part of it persists even when the project matures).

Literally none of those languages existed when Shopify was launched. Did you expect them to get into a time machine or make that happen or what?

True, I lost track of the original problem, sorry.

Still, at one point it's OK to admit the original tech is no longer as useful and migrate to something else IMO.

Re: Backend of Meta Threads is built with Python 3.10

#437

Post by pdimitar »

ReflectedImage wrote: post #431
pdimitar wrote: post #419

Earlier quoted context omitted.

> What we can’t know is whether they would have been fast enough to tweak their product to their current market without using a scripting language. Exactly. And I am saying the following as a fan of dynamic languages but let's be realistic: this is practically their ONLY true benefit -- quicker time to market. They lose out on pretty much any other metric, with the exception of also slightly quicker iteration in the…

In a startup, time to market is everything.

True, though I've measured my time to prototype with different languages some months ago and the advantages of PHP / Ruby / Python are oversold.

- Ruby on Rails prototype: 2.5 days.

- Golang's Gofiber: 4.5 days.

- Elixir's Phoenix: 7.

- Rust: don't ask (too long lol).

- Python's Django (made by a friend): 3.5 days.

Not to split hairs here but if you are in a situation where 5 days more for a prototype matter then I am not very sure you would have succeeded even short-term after. Putting your foot in the door can be extremely important, absolutely, but after whipping out a quick MVP you'd likely immediately stumble upon the next obstacle which is not guaranteed to be overcome.

As mentioned in my previous comment, the only argument in favor of the fastest-to-prototype languages could be that they allow slightly higher velocity day-to-day as well. But I've worked with PHP and Ruby for a long time, and I've worked with Elixir, Golang and Rust for quite a while now as well and again, that particular advantage of PHP / Ruby / Python (a) does exist, yes, but (b) is not as big as people make it out to be.

Re: Backend of Meta Threads is built with Python 3.10

#438

Post by DoesntMatter22 »

pdimitar wrote: post #436
DoesntMatter22 wrote: post #430

Earlier quoted context omitted.

Literally none of those languages existed when Shopify was launched. Did you expect them to get into a time machine or make that happen or what?

True, I lost track of the original problem, sorry. Still, at one point it's OK to admit the original tech is no longer as useful and migrate to something else IMO.

I appreciate your admission here.

We can disagree about usefulness I guess. Ultimately Ruby has worked just fine. They didn't report massive issues, and the DB is going to be the bottle neck way before Ruby, so I just don't see an issue.

If I knew I was building a site for a massive load would I use Ruby? No. But I don't think it's a terrible choice either.

Re: Backend of Meta Threads is built with Python 3.10

#439

Post by DoesntMatter22 »

pdimitar wrote: post #434
DoesntMatter22 wrote: post #429

Earlier quoted context omitted.

Behind how? There are no good web frameworks for Golang at all, and relatively few sites being built with it. Elixir? Okay, it's a nice language that is also dynamically typed. It has some advantages but good luck finding anyone with knowledge of how to program in it or has even used a functional language before. I don't personally use Ruby these days but to say it's super behind is just silly. Golang is as similar t…

People are starting to recognize benefits of even gradual typing en masse these days, while some of us knew it for 10+ years. Not to be an elitist, I've made plenty of other mistakes and I am trying to not look down on anyone, but to discount types is not a well-informed take IMO. That's what I mostly meant by saying Ruby is behind Elixir and Golang. And mind you, Elixir is strongly but dynamically typed and I still…

I'm not discounting types at all. I started in strongly typed languages and only gradually moved to dynamic languages. For massive apps generally dynamic languages are not good.

However, Ruby also has a lot of creature comforts that Elixir and GO don't have. Maybe there wouldn't be a product at all with GO or Elixir. It's hard to say.

Re: Backend of Meta Threads is built with Python 3.10

#440

Post by DoesntMatter22 »

pdimitar wrote: post #437
ReflectedImage wrote: post #431

Earlier quoted context omitted.

In a startup, time to market is everything.

True, though I've measured my time to prototype with different languages some months ago and the advantages of PHP / Ruby / Python are oversold. - Ruby on Rails prototype: 2.5 days. - Golang's Gofiber: 4.5 days. - Elixir's Phoenix: 7. - Rust: don't ask (too long lol). - Python's Django (made by a friend): 3.5 days. Not to split hairs here but if you are in a situation where 5 days more for a prototype matter then I a…

So, wait, all apps are trivial in nature and can be built in less than a week?

By your own admission here GO takes nearly twice as long (which is about what I've seen). So if you aren't building a trivial app but instead building something that takes 6 months in Ruby you'd be spending nearly 12 months in go. Sounds like a real significant amount of time if you are rushing to get to market.

Post reply on HN