891–900 of 1001 posts
Earlier quoted context omitted.
> removing engineers won't instantly crash the product. It'll happen slowly It's amazing to me how many people following the Twitter saga, some familiar with or actually working in technology, thought that Twitter would crash within days of the engineers being fired. And because it didn't, the job cuts are justified.
Quoted post unavailable.
> This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I was partly expecting the rest of the article to explain to me why exactly it wasn't just bloat. But it goes on talking about this 1~3-person cache SRE team that built solid infra automation that's really resilient to both hardware and software failures. If anything, the article might actually…
Once thing I've seen that I think is overlooked in the whole "is Twitter's tech stack inefficient" chat is the nature of the service. A lot of the "Twitter could be run off my laptop" style comments seem to come from people who run, effectively, Read-Only services. They might serve data at thousands of queries per second but the data itself is _slow_. It is video files or music streams or other data that updates infr…
I've heard this elsewhere, and when Twitter was originally founded in 2006 it wouldn't have been a particularly hard problem. Twitter processes roughly 10K tweets per second. Even if you bloat out the text quite a lot to account for encoding overheads, metadata, etc, etc... and assume that each one is 10KB, then this is just 1 Gbps. A single NIC on an old server. Okay, I get it, Twitter needs a lot of data too. Lists…
Twitter is both sides. Tweets are replicated out in milliseconds. Twitter search is lightning fast, comprehensiveband powerful. o Old tweets are not just glaciered into S3!
Earlier quoted context omitted.
> thought that Twitter would crash within days of the engineers being fired Because they work for companies where the product would fail within days of them being fired themselves.
That's what people keep telling themselves. Truth is, everybody is replaceable. And usually, loss of institutional knowledge takes a while to show real effects.
That said you can replace people and build back that institutional knowledge -- both loss and gain take significant amount of time.
Earlier quoted context omitted.
> the article might actually persuade me that it was all bloat First of all, how does it persuade you of that? The article touches a really small (though incredibly important for up-time) subject. Secondly, in any large company, the majority is 'bloat'. It's security engineers, code reviews, data architecture, HR, internal audit teams, content moderators, ccrum masters and I can keep going. In a start-up many of thes…
How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.
"20 with cloud, 40 without. So much overlap between iOS, Android, and the web, three people can do all three. More for the backend." https://twitter.com/realGeorgeHotz/status/159371372367535718...
Earlier quoted context omitted.
A lot of people who don't understand why Twitter owns two datacenters point to "complexity" as an argument and completely disregard scale. It turns out that massive scale adds a lot of complexity to a system, particularly around many-to-many pubsub systems (like social media). It also means a lot of features, like compliance with government regulations around the world.
WhatsApp had a Billion users and 50 employees. You can say that Twitter is incrementally more complex due to wider broadcasting, but it’s also 1/5 the user count. Everybody stating this number of employees is necessary to maintain a Twitter scale system is simply wrong. And as technology progresses, fewer and fewer people are needed to maintain the same size system Many just don’t want to believe that the leverage in…
Chat apps are well known to rely on infrastructure that scales up exceedingly well. That is why there are so many of them, and why they all have 100 tech employees or less.
Instagram at the time of the acquisition could have run on 20 servers plus S3. Today's Instagram, along with today's Twitter, does a lot of work that is super-linear in user count, and has something like 2000 engineers. Timeline building is reportedly O(n^2) in user count. The scale difference has a huge effect.
Notice that chat apps have no equivalent to "timeline building." The worst scale factor a chat app has is linear.
> This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I was partly expecting the rest of the article to explain to me why exactly it wasn't just bloat. But it goes on talking about this 1~3-person cache SRE team that built solid infra automation that's really resilient to both hardware and software failures. If anything, the article might actually…
Earlier quoted context omitted.
> the article might actually persuade me that it was all bloat First of all, how does it persuade you of that? The article touches a really small (though incredibly important for up-time) subject. Secondly, in any large company, the majority is 'bloat'. It's security engineers, code reviews, data architecture, HR, internal audit teams, content moderators, ccrum masters and I can keep going. In a start-up many of thes…
> In a start-up many of these roles can be ignored, becaus growth > stability. In a large organization, part of the bloat helps insure a certain amount of stability that's necessary to keep an organization alive. It also (a) increases the bus factor, [1] and (b) allows people to take vacations and time off without having to watch their phones like hawk. [1] https://en.wikipedia.org/wiki/Bus_factor
I haven’t worked on Twitter-scale products. But I do wonder if it’s safe to draw a straight line between services that have middling and large scales. Things don’t necessarily work the same at the extremes.
Earlier quoted context omitted.
Quoted post unavailable.
Weird to hear the work culture that involves being nice to people being called the toxic one.
Breathtaking niceness going on from lovely stable people:
apartheid profiteer, dictator's asskisser, insecure colonizer, cruel hoarder, mediocre manchild, petty racist, supreme parasite, petulant pimple
The projections were the work of Alan Marling, a Bay Area activist who declined to give his age or comment on his profession. He stood nearby — donning a Captain America face mask and a Sunrise Movement beanie — as passersby, many of whom were tech workers getting off work from nearby offices, stopped to take videos and photos.