Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

infoq.com

Post reply on HN
Print view View original

91–100 of 114 posts

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#91

Post by robotmay »

jedberg wrote: post #80
dap wrote: post #77

I was confused about the memcached problem after moving to the cloud. I understand why network latency may have gone from submillisecond to milliseconds, but how could you improve latency by batching requests? Shouldn't that improve efficiency, not latency, at the possible expense of latency (since some requests will wait on the client as they get batched)? And while maybe efficiency is valuable, why would that be an…

Sorry that wasn't clear. The latency didn't get better, but what happened is that instead of having to make a lot of calls to memcache it was just one (well, just a few), so while that one took longer, the total time was much less.

I actually did some (simplistic) examples of this in a small presentation to illustrate the performance improvements of batching memcached requests, if anyone's interested: https://speakerdeck.com/robotmay/a-simple-introduction-to-ef... (slides 11 to 14)

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#92

Post by gargoiler00 »

300bps wrote: post #88
gargoiler00 wrote: post #65

Earlier quoted context omitted.

For me personally, it ends up being far more annoying. I really wish wikipedia would just put small unobtrusive text adverts on each page rather than the massive intrusive banners begging for money.

I really wish wikipedia would just put small unobtrusive text adverts on each page rather than the massive intrusive banners begging for money. Hi, welcome to your first day on the Internet. Since you're new, let me tell you how things work around here. There are probably dozens of web sites similar to Wikipedia. But Wikipedia is on the first page of search engine results for just about anything you search for. Why i…

Welcome to the internet. You seem new.

Everyone goes to Google.com to search for things. You know that when you do a search, you're going to see helpful related adverts.

That level of conflict of interests and possible abuse, privacy concerns etc, means that the entire world uses google as their search engine. Oh and they make billions in profit.

Your hypothesis about an advertiser asking wikipedia to alter content surely applies to google search results.

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#93

Post by heed »

russelluresti wrote: post #70

Disappointment in the HN community has reached a new high today. So, instead of discussing the topics in the video, the majority of commenters here are discussing the flaws of the website it's hosted on or debating whether or not reddit is profitable. Neither of which has ANYTHING to do with scaling. I expected better, people. Seriously.

I'm not trying to be snarky, but your comment is off topic as well.. When does it end?

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#94

Post by 300bps »

gargoiler00 wrote: post #92
300bps wrote: post #88

Earlier quoted context omitted.

I really wish wikipedia would just put small unobtrusive text adverts on each page rather than the massive intrusive banners begging for money. Hi, welcome to your first day on the Internet. Since you're new, let me tell you how things work around here. There are probably dozens of web sites similar to Wikipedia. But Wikipedia is on the first page of search engine results for just about anything you search for. Why i…

Welcome to the internet. You seem new. Everyone goes to Google.com to search for things. You know that when you do a search, you're going to see helpful related adverts. That level of conflict of interests and possible abuse, privacy concerns etc, means that the entire world uses google as their search engine. Oh and they make billions in profit. Your hypothesis about an advertiser asking wikipedia to alter content s…

Your hypothesis about an advertiser asking wikipedia to alter content surely applies to google search results.

Google indexes other people's content. All Google has to say is, "Sorry, we're not in control of the content others make, our automated systems follow an algorithm we're unable to make one-off tweaks to." It could conceivably cost Google $1MM to make a one-off tweak to their algorithm in terms of programming and testing time.

Wikipedia on the other hand is all content. They have no plausible response other than, "Yeah, it would take 5 minutes to update that but we won't do that for you." Hell, all they'd really have to do is let the advertiser update it as they want and then instruct editors to do nothing.

It really is just different for this and a number of other reasons.

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#96

Post by jonursenbach »

conradfr wrote: post #95

I wonder if the demise of Digg three years ago and the (supposedly) inflow of new users have been problematic at the time.

Most likely as Reddit had continued downtime around that time with EC2/EBS scaling issues.

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#97

Post by jonursenbach »

RyanZAG wrote: post #38
OGC wrote: post #11

Reddit's not profitable though..

Profitable is not that big a deal with something on the size and important of Reddit though. Firstly, as with Wikipedia, if Reddit were forced to close because of money issues, Reddit could simply post a 'donate now or reddit shuts down' post and they would likely be rolling in millions of dollars. Second, simply because reddit itself is not profitable does not mean people are not making a lot of money off reddit. Th…

> Firstly, as with Wikipedia, if Reddit were forced to close because of money issues, Reddit could simply post a 'donate now or reddit shuts down' post and they would likely be rolling in millions of dollars.

From what I remember, this is kind of why they started Reddit Gold.

http://blog.reddit.com/2010/07/reddit-needs-help.html

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#98

Post by kerkeslager »

diminish wrote: post #76
russelluresti wrote: post #70

Disappointment in the HN community has reached a new high today. So, instead of discussing the topics in the video, the majority of commenters here are discussing the flaws of the website it's hosted on or debating whether or not reddit is profitable. Neither of which has ANYTHING to do with scaling. I expected better, people. Seriously.

The dominance of non-relevance is interesting for HN. Was writing exact the same thing, and just saw yours. In this and some other technical topics, people end up discussing their personal tastes with web site's design, their individual UI frustrations with some button on the web site, the font, the color, and other random non-relevant topic; like now the profitability.

It's no excuse, but I see a reason for this: most people feel they understand these irrelevant topics better than they understand the scalability features of the infrastructure Reddit has built. People talk about their comfort zone.

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#99

Post by gargoiler00 »

300bps wrote: post #94
gargoiler00 wrote: post #92

Earlier quoted context omitted.

Welcome to the internet. You seem new. Everyone goes to Google.com to search for things. You know that when you do a search, you're going to see helpful related adverts. That level of conflict of interests and possible abuse, privacy concerns etc, means that the entire world uses google as their search engine. Oh and they make billions in profit. Your hypothesis about an advertiser asking wikipedia to alter content s…

Your hypothesis about an advertiser asking wikipedia to alter content surely applies to google search results. Google indexes other people's content. All Google has to say is, "Sorry, we're not in control of the content others make, our automated systems follow an algorithm we're unable to make one-off tweaks to." It could conceivably cost Google $1MM to make a one-off tweak to their algorithm in terms of programming…

The problem is, if wikipedia did alter articles based on advertisers demands (Which seems pretty far fetched to me), the public would just alter them back. Or see the edits wikipedia is making and put 2 and 2 together.

>and then instruct editors to do nothing.

Yeah good luck with getting wikipedia editors to comply with that request!

A site like wikipedia would likely have thousands upon thousands of advertisers. They wouldn't be dependent on a few big advertisers. If an advertiser came to wikipedia and asked them to change a page, wikipedia would just say "no", publish the details to make the advertiser look like a douche (cue internet witch hunt, boycot naming shaming etc), and not care about the 0.000% temporary drop in revenue.

Re: Scaling Reddit from 1 Million to 1 Billion – Pitfalls and Lessons [video]

#100

Post by vegasje »

Does anyone know how the different storage systems are utilized, and why each system is utilized for that purpose? The presenter mentions using memcached, Cassandra, and PostgreSQL, and mentions the same type of data when discussing each (votes, for instance). I would definitely benefit from a more in-depth understanding of how each system is utilized, and why.
Post reply on HN