Live data from Hacker News

How Instagram scaled to 14 million users with only 3 engineers

engineercodex.substack.com

131–140 of 202 posts

Re: How Instagram scaled to 14 million users with only 3 engineers

#131

Earlier quoted context omitted.

I think this idea hurts the industry. Don't get me wrong. I think it is important to hire the right people, but if you don't hire enough people this 3 man job is 80-100 hours a week per for months to years. Look how X has diminished in quality as Elon started slashing team sizes. Then when you design more features, security and other various systems to serve the customers it will creep in complexity. You can not esca…

I have noticed absolutely no difference with Twitter/X. I am a casual user, sure, but for me it seems to work well.

We don't know how that relates to the traffic. If the load caved, you need less capacity to handle it. The DeSantis launch campaign was a complete disaster, and only recovered mid-stream when 2/3 of the users left.

There is a case to be made that Twitter would have been profitable if it didn't torch money on unnecessary complexity, but the crashing ad revenue suggests Musk is not the business genius that you might expect.

Re: How Instagram scaled to 14 million users with only 3 engineers

#132

"When you choose the right people you'll need only a few. When you don't, you'll need them all"

I think this idea hurts the industry. Don't get me wrong. I think it is important to hire the right people, but if you don't hire enough people this 3 man job is 80-100 hours a week per for months to years. Look how X has diminished in quality as Elon started slashing team sizes. Then when you design more features, security and other various systems to serve the customers it will creep in complexity. You can not esca…

Considering the hostile takeover of Twitter, the layoffs, lack of leadership and Elon pulling cables out of servers, the platform has been working exceptionally well, at least in my experience. Either it was built like a tank or some of the people working there are extremely competent. To be fair is not like Twitter has never been prone to issues. I'm an old enough user to remember the fail whale and many of the great outages of Twitter's first decade. It didn't became stable until maybe 5 years ago and even recently just prior to Elon coming in they had another hours-long downtime.

Re: How Instagram scaled to 14 million users with only 3 engineers

#133
post #96

Earlier quoted context omitted.

> Look how X has diminished in quality as Elon started slashing team sizes. He had a point there though: He said that there seem to be "3 managers 'managing' one engineer", and I believe this is a common problem in the industry. VC-funded startups are terribly overstaffed and over-inflated.

When you own a company and fire someone to cut costs nobody is gonna say anything, of course. It's the sad reality. When you cut the company to 1/3 and keep foreigners because of their visa status, nobody is gonna say anything of course, but that says a lot about you! I do not believe that half the company was just "overstaff". I have been in situations where 1 manager had 1 reporter/reported, but they were single ca…

> When you cut the company to 1/3 and keep foreigners because of their visa status, ... that says a lot about you!

That... you care about people regardless if they are foreigners and that you try to help those that would have the most problems if they were let go, especially as these problems are a consequence of your hiring of them?

I'm not familiar with the story, but from the way you presented it it sounds like a proper thing to do.

EDIT: not taking Musk's side, just pointing out the issue with the parent's argument.

Re: How Instagram scaled to 14 million users with only 3 engineers

#134
post #58

Earlier quoted context omitted.

> Look how X has diminished in quality as Elon started slashing team sizes. He had a point there though: He said that there seem to be "3 managers 'managing' one engineer", and I believe this is a common problem in the industry. VC-funded startups are terribly overstaffed and over-inflated.

Musk's actions have certainly damaged Twitter and its engineering. When you set the house on fire of course some termites will get killed but then you might also kill some babies and the family dog. A lot of tech companies have bloat in the form of AI ethics people, DEI people and so on. They need to go. But Musk probably hurt twitter a lot in short term by firing a lot of engineers and making it a place that made pe…

Our company has a position like a DEI director, and honestly, it baffles me. All they seem to do is push what some might call 'leftist woke propaganda' in their never-ending meetings, and I question whether our company's resources should be invested in that.

Re: How Instagram scaled to 14 million users with only 3 engineers

#135

I find the simplicity of the stack brilliant. It makes me to think now, to which extent, the industry nowadays simply suffers from a mix of lack of knowledge, CV driving design and big players in the game trying to sell you overkilling solutions and approaches for their own economical benefit. Have we perhaps, fell into a big enchant and, at the end of the day, 99% of companies out there could just use a classic LAMP…

Agree. The state of Kubernetes increasingly reminds me of the complexity hell that is Java J2EE. It’s an ecosystem of vendors with a self interest in selling complex solutions to sell high margin consulting services on top. Thus was J2EE.

Re: How Instagram scaled to 14 million users with only 3 engineers

#136
post #94

Earlier quoted context omitted.

I'm not even a casual user, I just go there when a friend sends me a funny tweet or an article point to one, and even I have noticed major bugs like comments not loading or other functionalities not working.

That's not a bug. If you're not logged in or your account looks like a bot, you can't see comments

> your account looks like a bot

That's not a bot if you are not a bot? I know it's 'normal', but I still it as a bug. AI classifying me as something I not is a bug.

Re: How Instagram scaled to 14 million users with only 3 engineers

#137
post #58

Earlier quoted context omitted.

Musk's actions have certainly damaged Twitter and its engineering. When you set the house on fire of course some termites will get killed but then you might also kill some babies and the family dog. A lot of tech companies have bloat in the form of AI ethics people, DEI people and so on. They need to go. But Musk probably hurt twitter a lot in short term by firing a lot of engineers and making it a place that made pe…

Our company has a position like a DEI director, and honestly, it baffles me. All they seem to do is push what some might call 'leftist woke propaganda' in their never-ending meetings, and I question whether our company's resources should be invested in that.

TIL: DEI == Diversity, Equity, & Inclusion (if anyone else didn't know).

Re: How Instagram scaled to 14 million users with only 3 engineers

#138
post #58

Earlier quoted context omitted.

Musk's actions have certainly damaged Twitter and its engineering. When you set the house on fire of course some termites will get killed but then you might also kill some babies and the family dog. A lot of tech companies have bloat in the form of AI ethics people, DEI people and so on. They need to go. But Musk probably hurt twitter a lot in short term by firing a lot of engineers and making it a place that made pe…

Our company has a position like a DEI director, and honestly, it baffles me. All they seem to do is push what some might call 'leftist woke propaganda' in their never-ending meetings, and I question whether our company's resources should be invested in that.

Could you give some concrete examples from this DEI director of the kind of things you're calling 'leftist woke propaganda'?

Re: How Instagram scaled to 14 million users with only 3 engineers

#139
post #79

Earlier quoted context omitted.

These CRUD apps need complex business rules, requiring expertise in the domain and making them configurable on the application level for customer while trying to keep the app not bloated. Scaling is not the only challenge engineers face, but somehow it's the one that is mostly praised.

They also need to respond to customer requirements, which IG never needed to do while they had no actual customers. And as soon as fun was up and IG had actual customers (spoiler alert, advertisers) what a surprise 3 devs was not enough. They also need to quickly respond to downtime, because unlike IG if some of those CRUD apps go down in B2B world you are often losing customers actual money not just ad views

Ad views translate to actual money??

Re: How Instagram scaled to 14 million users with only 3 engineers

#140
The article says nothing about how they instantaneously updated millions of user feeds. It was the most challenging task, as it's way easier to scale reads than writes in distributed systems. Rumor has it early Twitter had a target of 5 sec to update everyone of 50M fan feeds when Justin Bieber touched a screen. I would love to hear some technical details on how they did it.
Post reply on HN