Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Instagram scaled to 14 million users with only 3 engineers

engineercodex.substack.com

Post reply on HN
Print view View original

171–180 of 202 posts

Re: How Instagram scaled to 14 million users with only 3 engineers

#171

Post by BoorishBears »

jabradoodle wrote: post #158
BoorishBears wrote: post #82

Earlier quoted context omitted.

How does it make the backend more resilient than a monolith? Do you not realize you have multiple instances of a monolith or something?

There are some reasons it may lead to resiliency; another teams features slow dB queries not being on your db, teams not mutating data in a shared db, memory leaks in someone elses feature not taking your app down. Being able to choose language/libraries and tune the runtime to your requirements. Of course when you replace function calls with network calls, make everything asynchronous and eventually consistent, ther…

A monolith doesn't force a single database.

A monolith doesn't force a single process. IPC is still simpler and cheaper than network calls.

A monolith doesn't force never having an external service for a specialized use case, or FFI.

Re: How Instagram scaled to 14 million users with only 3 engineers

#172

Post by BoorishBears »

ahoka wrote: post #114
BoorishBears wrote: post #82

Earlier quoted context omitted.

How does it make the backend more resilient than a monolith? Do you not realize you have multiple instances of a monolith or something?

Bad input crashes app, monolith fails over, other instance crashes. Full outage. Assuming proper vertical separation, this risk can be reduced by microservices.

This is not what microservices solve.

There'll always be critical microservices that keep your app running. It doesn't matter if all your other services are running if the one serving up core functionality goes down.

If your engineering rigor is so poor that you can't get reliable failovers with a monolith, god help you keeping microservices running.

Re: How Instagram scaled to 14 million users with only 3 engineers

#173

Post by weird-eye-issue »

Sparkyte wrote: post #31
weird-eye-issue wrote: post #20

Earlier quoted context omitted.

> Look how X has diminished in quality as Elon started slashing team sizes. what happened?

Back when he acquired twitter he fire 2/3rds the company.

This response is about what I expected

Re: How Instagram scaled to 14 million users with only 3 engineers

#174

Post by nulltype »

coldtea wrote: post #87
_xivi wrote: post #7

> we can assume that Instagram was written using Objective-C and a combination of other things like UIKit. How can they know the internal infrastructure but have to assume the app language? Edit: So the entire piece is taken almost verbatim as-is from a couple old articles on instagram engineering blog. It might as well just redirect to: https://instagram-engineering.com/what-powers-instagram-hund... This is against…

It IS an original source. It's not just a repost or a report on another older article. It's a reworking of those articles.

I suspect they mean it’s a secondary source not a primary one

Re: How Instagram scaled to 14 million users with only 3 engineers

#175

Post by kermatt »

Octabrain wrote: post #123
ahoka wrote: post #109

Earlier quoted context omitted.

I just imagined trying to sell this architecture for a new product in an imaginary company, an amalgamation of every place I have ever worked: You have to change to Azure, because we are Microsoft partners and we have free credit. The credit is not too much though and we have to spend the same money on useless trainings so we keep being partners. 4 core and 8GB should be plenty for your dev VMs, that’s the largest we…

> What did I miss? From the top of my head... * A bunch of opinionated devs pushing for microservices because they heard that's what cool kids do now. No technical reason behind at all, just pure bias flamed by a couple of blog posts/YouTube talks they watched one evening. * A bunch of devops with intentions of implementing "industry best practices and modern tooling ©" that will end up creating a house of cards in T…

Experience and cynicism are closely related.

Re: How Instagram scaled to 14 million users with only 3 engineers

#176

Post by BoorishBears »

bornfreddy wrote: post #169
wild_egg wrote: post #145

Earlier quoted context omitted.

The fun part is often all that bullshit hyper optimized complexity can still be beaten on all the web core vitals by a humble LAMP server So not only are you wasting your time on all the wrong things, you're also gaining nearly no benefit from it I'm pretty convinced that all these shiny new hotness tools and frameworks of the past decade are actually just meant to sabotage competition and small companies

I agree - for most of them. One exception would be React/Vue/other frontend frameworks that split updating the state and visualizing it. It sounds like a small thing, but it makes a world of difference in non-trivial projects, compared to native js / jquery. Then again, it's the idea itself that matters, overoptimizing the implementation is not beneficial. I.e., React classes are just as good (or better) as hooks as…

Vercel and co managed to mangle the one benefit you mentioned.

They introduced the concept of a "server component": one that is rendered once and cannot update state.

And instead of making that opt-in, they made it the default.

—

That is to say, the default of React is to no longer allow updating state in components.

No value proposition is safe from the forces of financially incentivized thought leaders.

Re: How Instagram scaled to 14 million users with only 3 engineers

#177

Post by jabradoodle »

BoorishBears wrote: post #171
jabradoodle wrote: post #158

Earlier quoted context omitted.

There are some reasons it may lead to resiliency; another teams features slow dB queries not being on your db, teams not mutating data in a shared db, memory leaks in someone elses feature not taking your app down. Being able to choose language/libraries and tune the runtime to your requirements. Of course when you replace function calls with network calls, make everything asynchronous and eventually consistent, ther…

A monolith doesn't force a single database. A monolith doesn't force a single process. IPC is still simpler and cheaper than network calls. A monolith doesn't force never having an external service for a specialized use case, or FFI.

> IPC is still simpler and cheaper than network calls.

I specifically called out the extra complexity of network calls in microservices, not sure if you read the full comment.

> A monolith doesn't force a single process

I'm not convinced; if my small/specific code has it's own process, I would say it's a microservice. Sure, we can have replicas for redundancy, that doesn't mean I won't have reliability issues when my process is crashed.

> A monolith doesn't force a single database.

> A monolith doesn't force never having an external service for a specialized use case, or FFI

True, sadly it doesn't usually work this way. People take the path of least resistance.

Also once you add multiple DBs you start to get into eventual consistency; which is one of the harder parts of microservices.

Re: How Instagram scaled to 14 million users with only 3 engineers

#178

Post by mk89 »

bornfreddy wrote: post #133
mk89 wrote: post #96

Earlier quoted context omitted.

When you own a company and fire someone to cut costs nobody is gonna say anything, of course. It's the sad reality. When you cut the company to 1/3 and keep foreigners because of their visa status, nobody is gonna say anything of course, but that says a lot about you! I do not believe that half the company was just "overstaff". I have been in situations where 1 manager had 1 reporter/reported, but they were single ca…

> When you cut the company to 1/3 and keep foreigners because of their visa status, ... that says a lot about you! That... you care about people regardless if they are foreigners and that you try to help those that would have the most problems if they were let go, especially as these problems are a consequence of your hiring of them? I'm not familiar with the story, but from the way you presented it it sounds like a…

Sorry, you're right. I was try to point to different direction: you keep foreigners because you can exploit their visa status. :)

Re: How Instagram scaled to 14 million users with only 3 engineers

#179

Post by mk89 »

kyriee wrote: post #153
bornfreddy wrote: post #133

Earlier quoted context omitted.

> When you cut the company to 1/3 and keep foreigners because of their visa status, ... that says a lot about you! That... you care about people regardless if they are foreigners and that you try to help those that would have the most problems if they were let go, especially as these problems are a consequence of your hiring of them? I'm not familiar with the story, but from the way you presented it it sounds like a…

Or… heart me out, maybe it’s because Elon knows they have the most to loose and are thus more likely to do whatever he wants to not get kicked out of the country. Elon made some statements that made his disdain for the “laptop class” quite clear. He is no saint, or genius. He’s mostly just an extremely ruthless smart man.

He has finally showed the world his true self. I am happy about that - I don't want another capitalist sanctified for being a genius, which he is not. The more this becomes clearer the better for everyone.

Re: How Instagram scaled to 14 million users with only 3 engineers

#180

Post by 59nadir »

Octabrain wrote: post #123
ahoka wrote: post #109

Earlier quoted context omitted.

I just imagined trying to sell this architecture for a new product in an imaginary company, an amalgamation of every place I have ever worked: You have to change to Azure, because we are Microsoft partners and we have free credit. The credit is not too much though and we have to spend the same money on useless trainings so we keep being partners. 4 core and 8GB should be plenty for your dev VMs, that’s the largest we…

> What did I miss? From the top of my head... * A bunch of opinionated devs pushing for microservices because they heard that's what cool kids do now. No technical reason behind at all, just pure bias flamed by a couple of blog posts/YouTube talks they watched one evening. * A bunch of devops with intentions of implementing "industry best practices and modern tooling ©" that will end up creating a house of cards in T…

Funnily enough I've never worked with a dev that thought microservices were actually a good idea as a standard solution. My last brush with it is by proxy of one of my former teammates who has a client contract where the client insisted they do microservices for their app that fits in a t2small with 90% resources to spare and will for the foreseeable future. So far it's slowed down development to probably 25% or so of the speed it was before and 10% of what it should be.
Post reply on HN