Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What New Twitter Employees Do For The First Week

quora.com

Post reply on HN
Print view View original

21–30 of 69 posts

Re: What New Twitter Employees Do For The First Week

#21

Post by x5315 »

ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

One thing that CEO Dick Costolo has said is the amount of technical debt we have (http://kara.allthingsd.com/20110117/full-dces-interview-vide...). The scale is simple tremendous, and you've got to remember: Twitter isn't just a website; it's an SMS service, clients and an API.

Re: What New Twitter Employees Do For The First Week

#22

Post by ra »

ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Twitter.com is probably one of the great engineering feats of todays social Internet.

You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline.

The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter.com, or via the APIs) and is also copied onto those timelines.

Also, you have followers. So Twitter also copies the tweet onto the inbox timeline of each of your followers.

Not only is all this information published in real time on Twitter.com, but it is also made available via JSON API, Streaming API and Firehose API.

All tweets (on all timelines) are stored forever.

Twitter's scale is hard to fathom - all of this processing is way beyond what your Rails / Django app could process with a MySQL backend. MySQL replication wouldn'e even come close to keeping up with the sheer volume of events to be processed.

To make matters more complicated - Twitter is expected to scale to meet the demand of emerging world events (eg: Egypt, Iran, snowstorms, hurricanes, earthquakes, bushfires). These events don't evenly spread traffic across Twitter's network, but instead provide "storm surges" of localised intense traffic.

Oh, and Twitter haven't just launched an analytics product?

I think for sheer engineering at scale, there are maybe only about half a dozen other companies in the same league.

(Edit: grammer changes for readability)

Re: What New Twitter Employees Do For The First Week

#23

Post by __Joker »

ck2 wrote: post #12
tzs wrote: post #11

Earlier quoted context omitted.

Handling on average something like 76000 status updates per second (much higher when something interesting happens in the world), each of which must quickly become available to anywhere from 1 to a million clients. Wordpress.com is trivial compared to Twitter.

Yes but that's handled by hardware, not by people, and I am sure only a few critical coders handle the software performace. WordPress.com is not as trivial as you might think http://en.wordpress.com/stats/ Certainly it's just a matter of scaling once you hit a certain level of volume, you just have to be able to bring more servers online into the grid. Scaling from 10,000 users to 1 million is probably very hard. Sca…

http://www.quora.com/Twitter-Inc-company/Why-does-Twitter-ne...

Re: What New Twitter Employees Do For The First Week

#24

Post by klbarry »

phaedrus wrote: post #14

My "new hire" treatment as government contractor was so bad I almost quit. Even though it's a huge organization there's no process at all in place; you're just thrown to the wolves. Basic things like where and how to get my security badge and car tags were not covered. I was not given any input about what was expected of me, but I found out about things I was supposed to do only when I was getting yelled at for not h…

Wow, how strange.

Re: What New Twitter Employees Do For The First Week

#25

Post by js2 »

scarmig wrote: post #10
ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Quora to the rescue: http://www.quora.com/Twitter-Inc-company/Why-does-Twitter-ne...

If you ever wonder about why a company needs X developers, please come back to this list.

Re: What New Twitter Employees Do For The First Week

#26

Post by jacques_chester »

ck2 wrote: post #12
tzs wrote: post #11

Earlier quoted context omitted.

Handling on average something like 76000 status updates per second (much higher when something interesting happens in the world), each of which must quickly become available to anywhere from 1 to a million clients. Wordpress.com is trivial compared to Twitter.

Yes but that's handled by hardware, not by people, and I am sure only a few critical coders handle the software performace. WordPress.com is not as trivial as you might think http://en.wordpress.com/stats/ Certainly it's just a matter of scaling once you hit a certain level of volume, you just have to be able to bring more servers online into the grid. Scaling from 10,000 users to 1 million is probably very hard. Sca…

Based on the stats on the site, I calculate that Wordpress.com is averaging about 10 writes and around 8800 reads per second.

edit: Correction, ~880 reads per second.

Re: What New Twitter Employees Do For The First Week

#27

Post by ck2 »

ra wrote: post #22
ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Twitter.com is probably one of the great engineering feats of todays social Internet. You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline. The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter…

That's a good breakdown of the processes but I am positive all that is done in stages and queues.

So with a good design you have groups of servers doing the different stages in the queue.

One you've got the pattern down for 100 tweets per second, the pattern should be reproducible by scaling servers in each queue to 1000 tweets per second, and eventually 10,000 tweets per second.

The database requirements may explain why it's all done in one datacenter instead of trying to do replication across the country/world.

Re: What New Twitter Employees Do For The First Week

#28

Post by jacques_chester »

ra wrote: post #22
ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Twitter.com is probably one of the great engineering feats of todays social Internet. You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline. The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter…

A lot of companies deal with traffic of this magnitude. UPS perform 12 billion transactions per day through their systems - DB2 on Sysplex'd mainframes. Finance companies are currently dealing with tens of thousands of messages per second with much more complex handling and forwarding rules than Twitter. And, of course, in terms of page reads, Google and Facebook leave them well behind.

Twitter has become impressive, but let's not overstate its achievements.

Re: What New Twitter Employees Do For The First Week

#29

Post by jacques_chester »

ck2 wrote: post #27
ra wrote: post #22

Earlier quoted context omitted.

Twitter.com is probably one of the great engineering feats of todays social Internet. You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline. The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter…

That's a good breakdown of the processes but I am positive all that is done in stages and queues. So with a good design you have groups of servers doing the different stages in the queue. One you've got the pattern down for 100 tweets per second, the pattern should be reproducible by scaling servers in each queue to 1000 tweets per second, and eventually 10,000 tweets per second. The database requirements may explain…

In the research literature this is called SEDA: http://en.wikipedia.org/wiki/Staged_event-driven_architectur...

Re: What New Twitter Employees Do For The First Week

#30

Post by ry0ohki »

phaedrus wrote: post #14

My "new hire" treatment as government contractor was so bad I almost quit. Even though it's a huge organization there's no process at all in place; you're just thrown to the wolves. Basic things like where and how to get my security badge and car tags were not covered. I was not given any input about what was expected of me, but I found out about things I was supposed to do only when I was getting yelled at for not h…

Contractors are the lowest scum in the pond in the Government (they all know your making twice what they do). "Real Employees" get treated a little better, but you'll often find the same indifference.
Post reply on HN