Live data from Hacker News

Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

nytimes.com

271–280 of 414 posts

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#271
post #206

I think the general market could learn a thing or two from WhatsApp: 55 people powering a platform used by 800 million _active_ users across many different platforms. And WhatsApp has innovated, e.g. the new video and calling system works very well. I have often wondered what thousands of Twitter employees are doing all day long? Their platform and apps have been stale for a long time.

I mostly agree, but the scale of what Twitter is doing technically is very different. Network size is not as important as how messages are distributed within it. Whereas most WhatsApp messages are one to one, on Twitter they're almost always one-to-very-many. If 10000 Twitter users with 1000 followers each send a tweet, that's 10000000 messages hitting queues. 10000 WhatsApp users sending a single message each are li…

> I mostly agree, but the scale of what Twitter is doing technically is very different.

If anything, Twitter's operation is significantly easier.

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#272
post #232

Earlier quoted context omitted.

Perhaps that is the lesson to be learned? That technology with limits... scales? If you want to "do social", do it for small groups. Nobody really has 40 000 friends, so why bother with the use-case that is literally only used by advertising companies?

I don't think the final question is realistic. I have 20K followers (not huge, but far beyond who I could realistically ever "know") and get huge value from it as an individual, not as a business. It would be so boring (to me) to only be able to talk to people one knows closely. There are numerous non-company non-celebs-in-the-Hollywood-sense-of-the-word people who have huge followings on Twitter who get a lot of val…

Soooo... RSS? Isn't that what you've just described? You could run that on a pretty small web server....

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#273

Earlier quoted context omitted.

I mostly agree, but the scale of what Twitter is doing technically is very different. Network size is not as important as how messages are distributed within it. Whereas most WhatsApp messages are one to one, on Twitter they're almost always one-to-very-many. If 10000 Twitter users with 1000 followers each send a tweet, that's 10000000 messages hitting queues. 10000 WhatsApp users sending a single message each are li…

In a certain sense, twitter's one-to-many messaging is simpler, because it is a smaller volume of messages that have to be served.

Not at all. Have you ever thought about caching Twitter? It's an archetypical example of a site that's extremely difficult to cache well, which is the source of so many of its scaling problems.

Every person's new feed is uniquely generated based on their followers. That means you can only at best cache at the individual tweet level -- you still need to uniquely generate a news feed for each user. This means that for every user you need to individually grab information simultaneously from the same high velocity tweets, while still being able to present information to each user uniquely based on conditions regarding how twitter should display tweets for that specific user individually. That's incredibly demanding data wise because it makes horizontal distribution of load much more difficult --- a huge number of people are all making demands on the same set of data resources, and you can't easily cache that data because of how real time and quickly updating it is.

In comparison WhatsApp on Erlang is basically just an example of extremely dramatic real time horizontal scaling. Hosting hundreds of millions of simultaneous chats is relatively trivial in Erlang, given each one is operating under its own separately scalable chat process. Since very few chats share data with one another, horizontal scaling is easy because you don't have the issue of the same resource getting called simultaneously by millions of unique users at the same time, who all need that resource represented back to their own news feed in differently categorized ways.

tl;dr: WhatsApp with the right technology is WAY easier to scale than Twitter.

---------------

(Speaking of the magic of Erlang for chat programs, this funny video "for Ruby/Node.js hipsters" explains how Erlang does this magic quite well: https://www.youtube.com/watch?v=rRbY3TMUcgQ) )

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#274
post #206

I think the general market could learn a thing or two from WhatsApp: 55 people powering a platform used by 800 million _active_ users across many different platforms. And WhatsApp has innovated, e.g. the new video and calling system works very well. I have often wondered what thousands of Twitter employees are doing all day long? Their platform and apps have been stale for a long time.

I mostly agree, but the scale of what Twitter is doing technically is very different. Network size is not as important as how messages are distributed within it. Whereas most WhatsApp messages are one to one, on Twitter they're almost always one-to-very-many. If 10000 Twitter users with 1000 followers each send a tweet, that's 10000000 messages hitting queues. 10000 WhatsApp users sending a single message each are li…

WhatsApp also appears, at the moment, to have a much simpler business model - you buy it. No advertising technology, no audience/data teams and, probably most significantly, no sales team to go out and get people to use the platform. And no big agency handholding people, no data scientists, no customer success account managers. I'd be curious to see what the breakdown is between tech and sales at Twitter.

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#275
post #271

Earlier quoted context omitted.

I mostly agree, but the scale of what Twitter is doing technically is very different. Network size is not as important as how messages are distributed within it. Whereas most WhatsApp messages are one to one, on Twitter they're almost always one-to-very-many. If 10000 Twitter users with 1000 followers each send a tweet, that's 10000000 messages hitting queues. 10000 WhatsApp users sending a single message each are li…

> I mostly agree, but the scale of what Twitter is doing technically is very different. If anything, Twitter's operation is significantly easier.

In what sense? Technically? Business? Why is easier?

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#276
post #232

Earlier quoted context omitted.

Perhaps that is the lesson to be learned? That technology with limits... scales? If you want to "do social", do it for small groups. Nobody really has 40 000 friends, so why bother with the use-case that is literally only used by advertising companies?

Perhaps because providing ad companies with the tools like these is the way to monetize the service?

Well, it seems that idea has failed now. Why else would they be cutting costs?

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#277
post #206

I think the general market could learn a thing or two from WhatsApp: 55 people powering a platform used by 800 million _active_ users across many different platforms. And WhatsApp has innovated, e.g. the new video and calling system works very well. I have often wondered what thousands of Twitter employees are doing all day long? Their platform and apps have been stale for a long time.

I agree @amix. More often than not it's asign of poor vision and leadership.

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#278

Earlier quoted context omitted.

Twitter is doing something vastly more technically challenging that WhatsApp. WhatsApp just delivers messages. It doesn't store them, and it certainly doesn't, say, try to make them available in a searchable away to everyone in the world. This isn't a knock on WhatsApp -- I use WhatsApp and don't use Twitter -- but Twitter's scale challenges are vastly more complex than WhatsApp's, which doesn't have many features th…

> It doesn't store them, and it certainly doesn't, say, try to make them available in a searchable away to everyone in the world. Back of the envelope calculation: If your messages are 256 char messages, on a 128GB ram server you can fit 500 billion of them in memory; incidentally, that's one day's worth of tweets these days. I recently got a quote for a beefy 256GB ram server from Dell for $6,000. To make things eas…

500M tweets per day, not 500B: http://www.internetlivestats.com/twitter-statistics/ Assuming 100 Bytes per message gives 50GB of messages per day.

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#279
post #218

Earlier quoted context omitted.

Note that it is "if", and not "if and only if". Basically, if you can be tracked or advertised to ("advertracked"), you ARE the product whether you pay or not. Money has to come from somewhere; if it is not from you, it is by monetizing you. But ... why not monetize the user even when they did pay? It's a whole new revenue stream, that's often orthogonal to the payment stream. Paid Hulu has ads. Windows Solitaire has…

And now, thanks to the hobby of one developer, my phone blocks ads in /etc/hosts, preventing even in-app ads from appearing. My web browser on my laptop does just as well with a plugin. All that's left in my browsing experience to hint that sites are trying to maintain themselves on advertising are reels of clickbait stories that come at the side and bottom of every traditional media article. I can't help but hope th…

How do you block in app adds?

Re: Twitter Expected to Begin Layoffs and Stop Headquarters Expansion

#280
post #269
post #247

Earlier quoted context omitted.

There are many well understood approaches to scaling this. There are operational challenges to it, but it's not that hard. E.g. decompose large follower lists by using reflectors, and cap the size in the messaging system to a suitably low number. Now you've reduced the issue to one of sharding it over enough machines.

What are reflectors?

A piece of software that takes one input and fans it out to many consumers.
Post reply on HN