Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Instagram scaled to 14 million users with only 3 engineers

engineercodex.substack.com

Post reply on HN
Print view View original

161–170 of 202 posts

Re: How Instagram scaled to 14 million users with only 3 engineers

#161

Post by JimDabell »

djtango wrote: post #111
rapnie wrote: post #107

Here's recent insiders experience on developing Meta's Threads app on top of Instagram infra: https://newsletter.pragmaticengineer.com/p/building-metas-th... > In July of this year, Meta launched its latest mobile app, Threads, a microblogging service and new rival to X, formerly Twitter. In the first five days following its launch, the app achieved 100M downloads – a new record for the company by some margin. Meta’s…

I tthink the reality of modern development, especially one plugged into the Meta ecosystem is that there is probably a tonne of integration work to be done. - monetization - finance - analytics - ad placement - ad bidding - android client - apple client - browser client Just off the top of my head in 2 mins that's a few of the extra concerns I can come up with... I remember meeting someone who was responsible for wri…

Remember Instagram was iOS-only for a good long while. So no need for Android developers or front-end web developers.

Re: How Instagram scaled to 14 million users with only 3 engineers

#162

Post by JimDabell »

tiffanyh wrote: post #108

Threads I find it interesting that Meta choice largely this same tech stack for their newly created Threads service.

It’s not that they made the same choices again so much as they just re-used Instagram code and infrastructure.

Re: How Instagram scaled to 14 million users with only 3 engineers

#163

Post by notacoward »

djtango wrote: post #111
rapnie wrote: post #107

Here's recent insiders experience on developing Meta's Threads app on top of Instagram infra: https://newsletter.pragmaticengineer.com/p/building-metas-th... > In July of this year, Meta launched its latest mobile app, Threads, a microblogging service and new rival to X, formerly Twitter. In the first five days following its launch, the app achieved 100M downloads – a new record for the company by some margin. Meta’s…

I tthink the reality of modern development, especially one plugged into the Meta ecosystem is that there is probably a tonne of integration work to be done. - monetization - finance - analytics - ad placement - ad bidding - android client - apple client - browser client Just off the top of my head in 2 mins that's a few of the extra concerns I can come up with... I remember meeting someone who was responsible for wri…

You hit the nail on the head. I'd add moderation to that list. Also, when you're in revenue mode and scaling from 14M users to billions it's no longer sustainable to outsource most of your infrastructure - OS, storage, other basic services - to a cloud provider. So you're going to need people to develop and manage those, even if you tell yourself you don't have to count them because they're elsewhere in a large company (as I was once). Then you have things like observability tools, non-trivial deployment tools, etc. that are also Other People's Effort.

Don't get me wrong, I think it's great that we live in a world where there's infrastructure lying around that makes it possible for a three-person team to achieve so much so quickly, and that team deserves a ton of credit for doing it. But it's not the "these companies are so bloated" lesson that some people are going to try to take away from it. That's just Tall Poppy Syndrome.

Re: How Instagram scaled to 14 million users with only 3 engineers

#164

Post by rich_sasha »

jakjak123 wrote: post #152
rich_sasha wrote: post #25

Stories like this kind of make sense to me. 3 people is very few but I guess they really knew what they are doing. Meanwhile, all these orgs with essentially a CRUD app, with 1,000s of engineers..? That I never understood.

They stayed true to a single product purpose, did not add frivolous features and experiments, and didn't deal with payments/billing, eventually being "rescued" by VC money? You seem to gloss over that AWS also helps them out a lot already here, by being the load balancer entry points and ordering VMs from a api.

There's no glossing, I just don't understand. It's also far from my corner of technology.

Still, between 3 devs and thousands, that's 3 orders of magnitude. Let's take the core team and 30x them for a broader project. Add 50 (?) for billing. Another 50 for web apps. Another 30 for general devops? That takes us to a "mere" 220 devs.

Then I read that Uber had around 2,000 SEs in 2020, and Airbnb at some point was 1,200. I don't know how to grok that.

You mention AWS, my impression was that even many very large orgs run their their hardware in the cloud (surely not all but still).

Re: How Instagram scaled to 14 million users with only 3 engineers

#165

Post by djtango »

JimDabell wrote: post #161
djtango wrote: post #111

Earlier quoted context omitted.

I tthink the reality of modern development, especially one plugged into the Meta ecosystem is that there is probably a tonne of integration work to be done. - monetization - finance - analytics - ad placement - ad bidding - android client - apple client - browser client Just off the top of my head in 2 mins that's a few of the extra concerns I can come up with... I remember meeting someone who was responsible for wri…

Remember Instagram was iOS-only for a good long while. So no need for Android developers or front-end web developers.

Didn't know that, I still don't use Instagram lol. So yeah exactly - it's easy to run a simple app with a few devs.

Iirc in Singapore, Netflix has a reasonably sized team just dealing with payments.

Presumably for a global business, payments is a pretty gnarly long tail of integrations (never worked with payments so pretty ignorant on the subject) not to mention I wonder whether they also are onboarding content distributors onto some kind of automated payment platform so they can very quickly churn content libraries or attribute some kind of viewing related payment...

I guess tldr doing business is complex and needs lots of engineers vs making a small and focused app.

Re: How Instagram scaled to 14 million users with only 3 engineers

#166

Post by nielsole »

_xivi wrote: post #7

> we can assume that Instagram was written using Objective-C and a combination of other things like UIKit. How can they know the internal infrastructure but have to assume the app language? Edit: So the entire piece is taken almost verbatim as-is from a couple old articles on instagram engineering blog. It might as well just redirect to: https://instagram-engineering.com/what-powers-instagram-hund... This is against…

I wonder how they horizontally scaled shards. If they had 2k logical shards they probably had much fewer real/physical shards. So a single database was holding many sharding keys. So when a new physical shard gets added, the data needs to somehow be replicated. That is only true if w reading is the problem. If only a relatively short time period of data is hot you can probably just move over the logical shard IDs to the new physical shard without moving existing data. This requires keeping track of when which physical shard became active.

Re: How Instagram scaled to 14 million users with only 3 engineers

#167

Post by alphabettsy »

rich_sasha wrote: post #164
jakjak123 wrote: post #152

Earlier quoted context omitted.

They stayed true to a single product purpose, did not add frivolous features and experiments, and didn't deal with payments/billing, eventually being "rescued" by VC money? You seem to gloss over that AWS also helps them out a lot already here, by being the load balancer entry points and ordering VMs from a api.

There's no glossing, I just don't understand. It's also far from my corner of technology. Still, between 3 devs and thousands, that's 3 orders of magnitude. Let's take the core team and 30x them for a broader project. Add 50 (?) for billing. Another 50 for web apps. Another 30 for general devops? That takes us to a "mere" 220 devs. Then I read that Uber had around 2,000 SEs in 2020, and Airbnb at some point was 1,200…

There seems to be a pretty big difference in business logic and requirements between a photo sharing app and one that moves or houses people.

Dealing with payments globally to hosts/drivers and specifically compliance is complex in itself.

Re: How Instagram scaled to 14 million users with only 3 engineers

#168

Post by 3np »

drno123 wrote: post #36
Sparkyte wrote: post #31

Earlier quoted context omitted.

Back when he acquired twitter he fire 2/3rds the company.

No, what is the consequence, how has quality of Twitter diminished? As a “normal” user I can not observe any degradation of Twitter service.

Experiences vary wildly between and IP addresses and browser profiles. The variance increased drastically post-takeover.

Re: How Instagram scaled to 14 million users with only 3 engineers

#169

Post by bornfreddy »

wild_egg wrote: post #145
BoorishBears wrote: post #80

Earlier quoted context omitted.

Which makes things like Vercel so depressing to me. You've got a company paying off influential people in a space where people are looking for guidance, convincing them that they too have hard problems that cannot settle for simple solutions. Selling the narrative that developers need to be all in on the most irrelevant aspects of building a product, and ignoring the fact that if you instead focus on building simple,…

The fun part is often all that bullshit hyper optimized complexity can still be beaten on all the web core vitals by a humble LAMP server So not only are you wasting your time on all the wrong things, you're also gaining nearly no benefit from it I'm pretty convinced that all these shiny new hotness tools and frameworks of the past decade are actually just meant to sabotage competition and small companies

I agree - for most of them. One exception would be React/Vue/other frontend frameworks that split updating the state and visualizing it. It sounds like a small thing, but it makes a world of difference in non-trivial projects, compared to native js / jquery. Then again, it's the idea itself that matters, overoptimizing the implementation is not beneficial. I.e., React classes are just as good (or better) as hooks as far as I'm concerned, it would be better if they would let it be at that stage.

Re: How Instagram scaled to 14 million users with only 3 engineers

#170

Post by bornfreddy »

kyriee wrote: post #153
bornfreddy wrote: post #133

Earlier quoted context omitted.

> When you cut the company to 1/3 and keep foreigners because of their visa status, ... that says a lot about you! That... you care about people regardless if they are foreigners and that you try to help those that would have the most problems if they were let go, especially as these problems are a consequence of your hiring of them? I'm not familiar with the story, but from the way you presented it it sounds like a…

Or… heart me out, maybe it’s because Elon knows they have the most to loose and are thus more likely to do whatever he wants to not get kicked out of the country. Elon made some statements that made his disdain for the “laptop class” quite clear. He is no saint, or genius. He’s mostly just an extremely ruthless smart man.

Sure, that is also a possible (even likely) explanation. As I said, I don't defend Musk - my opinion of him is quite low and is getting lower and lower. Just wanted to point out the trouble with the argument. Anyway.
Post reply on HN