Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What New Twitter Employees Do For The First Week

quora.com

Post reply on HN
Print view View original

31–40 of 69 posts

Re: What New Twitter Employees Do For The First Week

#31

Post by ck2 »

jacques_chester wrote: post #26
ck2 wrote: post #12

Earlier quoted context omitted.

Yes but that's handled by hardware, not by people, and I am sure only a few critical coders handle the software performace. WordPress.com is not as trivial as you might think http://en.wordpress.com/stats/ Certainly it's just a matter of scaling once you hit a certain level of volume, you just have to be able to bring more servers online into the grid. Scaling from 10,000 users to 1 million is probably very hard. Sca…

Based on the stats on the site, I calculate that Wordpress.com is averaging about 10 writes and around 8800 reads per second. edit: Correction, ~880 reads per second.

Looking at the posts per day detail I think it's more like 30 per second with bursts that probably go to twice that, while nighttime is idle depending on timezones.

But I agree it's a fraction of 7k/sec peak for twitter.

However, twitter does not have to parse html, has no plugins to execute or templates, and has a max string length of 140 characters which is trivial.

Each post/comment published on wp.com takes many, many more cpu cycles than twitter.

Re: What New Twitter Employees Do For The First Week

#32

Post by ry0ohki »

ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Once you hire over 15 people, you need to start hiring people to manage the HR issues that it brings, and then once you have 20 why you need middle management, and at 40 you need someone to write internal tools and manage hardware etc... then for every 40.... you get the point. Past a small team, employee count snowballs.

Re: What New Twitter Employees Do For The First Week

#33

Post by zohebv »

benologist wrote: post #18
tzs wrote: post #11

Earlier quoted context omitted.

Handling on average something like 76000 status updates per second (much higher when something interesting happens in the world), each of which must quickly become available to anywhere from 1 to a million clients. Wordpress.com is trivial compared to Twitter.

You're an order of magnitude past their peak load - New Years Eve they set a new record of almost 7000/second. http://thenextweb.com/twitter/2011/01/06/new-years-eve-set-a... I don't believe Twitter is trivial, but I think they're perceived as more complex than they are - WordPress + Reddit + Heroku only have like 10% that number combined . Apples to oranges, but those 3 companies would be doing more everythings per…

Handling updates to 7000 user accounts per second isn't what makes it hard for twitter. It is the propagation of these updates to a gazillion followers that taxes the system. Their primary overhead is the messaging update to followers. Ashton Kutcher has 6 million followers. Sending instantaneous notifications to all those million twitter clients is what makes it a killer.

TLDR; It is not the vertices in the twitter graph but the number of edges in it that is unprecedented.

Re: What New Twitter Employees Do For The First Week

#34

Post by jacques_chester »

ck2 wrote: post #31
jacques_chester wrote: post #26

Earlier quoted context omitted.

Based on the stats on the site, I calculate that Wordpress.com is averaging about 10 writes and around 8800 reads per second. edit: Correction, ~880 reads per second.

Looking at the posts per day detail I think it's more like 30 per second with bursts that probably go to twice that, while nighttime is idle depending on timezones. But I agree it's a fraction of 7k/sec peak for twitter. However, twitter does not have to parse html, has no plugins to execute or templates, and has a max string length of 140 characters which is trivial. Each post/comment published on wp.com takes many,…

I agree that Wordpress requires far more CPU time (I run a network of Wordpress blogs).

My figures: 900,000 transactions (500,000 posts + 400,000 comments) / 86,400 seconds per day:

~= 10 transactions per second on average, though like twitter, activity probably has a power law distribution corresponding with US daylight hours.

Reads: 2.3 billion / month Suppose a 30-day month, that's 86,400*30 or 2,592,000 seconds per month.

2,300,000,000 / 2,592,000 ~= 887 pageviews per second.

I was incorrect by an order of magnitude, though the same uneven traffic patterns caveat applies.

Re: What New Twitter Employees Do For The First Week

#35

Post by ra »

jacques_chester wrote: post #28
ra wrote: post #22

Earlier quoted context omitted.

Twitter.com is probably one of the great engineering feats of todays social Internet. You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline. The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter…

A lot of companies deal with traffic of this magnitude. UPS perform 12 billion transactions per day through their systems - DB2 on Sysplex'd mainframes. Finance companies are currently dealing with tens of thousands of messages per second with much more complex handling and forwarding rules than Twitter. And, of course, in terms of page reads, Google and Facebook leave them well behind. Twitter has become impressive,…

Sure but UPS, Google and Facebook have a lot more than 350 employees.

Re: What New Twitter Employees Do For The First Week

#36

Post by nostrademons »

benologist wrote: post #18
tzs wrote: post #11

Earlier quoted context omitted.

Handling on average something like 76000 status updates per second (much higher when something interesting happens in the world), each of which must quickly become available to anywhere from 1 to a million clients. Wordpress.com is trivial compared to Twitter.

You're an order of magnitude past their peak load - New Years Eve they set a new record of almost 7000/second. http://thenextweb.com/twitter/2011/01/06/new-years-eve-set-a... I don't believe Twitter is trivial, but I think they're perceived as more complex than they are - WordPress + Reddit + Heroku only have like 10% that number combined . Apples to oranges, but those 3 companies would be doing more everythings per…

As has been pointed out elsewhere on the thread, complexity doesn't scale linearly. It's far easier to write ten sites that each gets 30 qps than it is to write one site that gets 300 qps, and it's easier to write ten sites that get 300 qps than one site that gets 3000 qps.

Twitter's fundamental problem also is a harder one to scale than something like Heroku or Wordpress. For those hosted sites, you can shard easily by host, so that each of the 100,000 Heroku-hosted sites can get its own EC2 instance(s) and behave pretty much independently. You can't do that when the point of your site is that any action might instantly be broadcast to thousands of followers. High-fanout writes are not an easy problem to solve.

Re: What New Twitter Employees Do For The First Week

#37

Post by nostrademons »

ck2 wrote: post #12
tzs wrote: post #11

Earlier quoted context omitted.

Handling on average something like 76000 status updates per second (much higher when something interesting happens in the world), each of which must quickly become available to anywhere from 1 to a million clients. Wordpress.com is trivial compared to Twitter.

Yes but that's handled by hardware, not by people, and I am sure only a few critical coders handle the software performace. WordPress.com is not as trivial as you might think http://en.wordpress.com/stats/ Certainly it's just a matter of scaling once you hit a certain level of volume, you just have to be able to bring more servers online into the grid. Scaling from 10,000 users to 1 million is probably very hard. Sca…

Understand that Google is continuously rewriting their infrastructure to handle increased scale. That's what they have 25k employees for.

Jeff Dean's rule of thumb is that you should build a growth factor of 10 into the design, but any more than that and you will probably have to re-architect anyway. So going from 10,000 to 1 million and 1 million to 100 million are probably roughly equivalent in difficulty.

Re: What New Twitter Employees Do For The First Week

#38

Post by aditya »

Quite a contrast from the purported Honda "onboarding" process (or lack thereof). They basically let you figure out where you'd be most useful, and no one really tells you what to do. It seems to scale well, I suppose.

A funny thing happened on the way to the solution. Try and picture this. On my first day of work, no one told me what to do. On the second day, the same thing happened, and on the third. That’s as much as I could take. I decided to meet with everyone I was coming in contact with to find out more about their individual talents and personalities, and to find out what was going on. Before I knew it, I was developing a picture of how things really were, and who needed what, and I became creatively involved in defining my own participation in relation to the skills I could bring to the table. In the process of doing this, I had complete access to everyone in the company, from other newlings to the President. Nothing but open cubicles no higher than 3 1/2 feet. I was allowed to learn, interact, and find solutions to every problem and need I recognized. I always found something important to do, and it became natural to provide effective solutions as needed. I am not a very unique individual, but I am effective because I am allowed to be. I also know it may be different for some people, experiencing this kind of freedom. I know that some people are petrified by this kind of freedom, and equate it to abandonment, and it drives them crazy not knowing what to do. I also know that even under the best of circumstances, people become sedate sometimes and settle in to patterns of repetition for false comfort. The answer, then, is to have them all switch places every few years, no matter how well they may be doing their job, because it is just as important to let everyone see their own position from someone else’s position. It also allows for the surprise of finding how much fun change can be when your creativity meets a new challenge. See what you end up with. It’s either this, or that.

More here: http://aditya.sublucid.com/2008/11/20/let-your-employees-fig...

Re: What New Twitter Employees Do For The First Week

#39

Post by GBond »

ra wrote: post #22
ck2 wrote: post #4

How can over 100 people be needed for basically a scaled blog system that consists only of 140 character titles (not even any article content) and RSS feeds? It doesn't even have a search feature (past the previous week) or html, or tags, etc. Doesn't WordPress.com do it with like a dozen people and their system is way, way, way more complex? update: no, wait, in December 2010 they announced it's over 350 people now…

Twitter.com is probably one of the great engineering feats of todays social Internet. You tweet from your mobile phone app (maintained by twitter) or twitter.com, and that tweet gets posted on your public timeline. The tweet is parsed for mentions, and it is also copied on to the mentionee's incoming timeline and their mentions timeline. The tweet is parsed for hashtags, and active searches (i.e. ones open on twitter…

Great "social internet" Engineering feat, eh maybe. Compared with other real-time & data intensive platforms, not even close. Electronic finance assets exchange platforms are a order of magnitudes more complex.

Re: What New Twitter Employees Do For The First Week

#40

Post by jacques_chester »

ra wrote: post #35
jacques_chester wrote: post #28

Earlier quoted context omitted.

A lot of companies deal with traffic of this magnitude. UPS perform 12 billion transactions per day through their systems - DB2 on Sysplex'd mainframes. Finance companies are currently dealing with tens of thousands of messages per second with much more complex handling and forwarding rules than Twitter. And, of course, in terms of page reads, Google and Facebook leave them well behind. Twitter has become impressive,…

Sure but UPS, Google and Facebook have a lot more than 350 employees.

Not an apples-to-apples comparison. UPS, Google and Facebook have much less concentrated product/service offerings (ie, they each have >1).
Post reply on HN