Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reddit: 2012 State of the Servers

blog.reddit.com

Post reply on HN
Print view View original

21–30 of 62 posts

Re: Reddit: 2012 State of the Servers

#21

Post by dangrossman »

plasma wrote: post #16
joering1 wrote: post #10

Earlier quoted context omitted.

and also -- any idea how much they can bring id ad revenues?

Usually 1,000 ad impressions is around $1.00 (varies greatly though, can be lower like $0.15c and higher like $4.00+). Assuming $1.00 per 1,000 impressions, and taking their 2.07billion impressions/month figure: Roughly 2,070,000,000 / 1,000 = $2,070,000 in ad revenue per month? Wild guess.

Many/most page views on Reddit don't have any ads. Promoted stories only appear on story lists, not individual story pages, and don't appear 100% of the time (the space is also used to promote random new submissions). The graphical slot in the sidebar is almost 100% non-paid in-house ads.

Re: Reddit: 2012 State of the Servers

#22

Post by tibbon »

joevandyk wrote: post #13

They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.

By local storage, do they mean they are running their DB disks on physical hardware, but the database servers still on EC2?

Re: Reddit: 2012 State of the Servers

#23

Post by cheald »

jarcoal wrote: post #17
joevandyk wrote: post #13

They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.

What a bummer. Amazon advertises EBS as being both faster and more reliable, but it sounds like they are delivering neither.

EBS is fine until it isn't. The problem isn't general EBS suckage, it's unpredictable and sporadic suckage. When your DB server is blocking while it tries to write to a disk that isn't responding, things get really hairy really quickly.

Re: Reddit: 2012 State of the Servers

#24

Post by x3c »

joevandyk wrote: post #13

They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.

But how do you achieve data persistence in case of server crash? Snapshots are not reliable for that, slave db servers aren't foolproof either.

Re: Reddit: 2012 State of the Servers

#25

Post by cheald »

davej wrote: post #20
cluda01 wrote: post #2

I'm unfamiliar with hosting costs or really any costs running a site as popular as reddit. Anyone with experience in this area have a ballpark figure for how much it would cost per month to run this sort of setup?

My back of the envelope estimate. These are based on the figures from last year and the fact that they currently have 240 EC2 instances, some are large (guessed 70), more are x-large (guessed 170). 8760 is the number of hours in a year. (8760 * $0.24 * 170) + (8760 * $0.12 * 70) = $430,992/yr in hourly fees ($1,820 * 170) + ($910 * 70) = $373,100/yr in reservation fees 373,100 + 430,992 = 804,092 / 12 months = $67,00…

Nearly $1million/year in infrastructure costs so that I can laugh at GIFs of cats.

The internet is truly a wonderous thing.

Re: Reddit: 2012 State of the Servers

#26

Post by cheald »

x3c wrote: post #24
joevandyk wrote: post #13

They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.

But how do you achieve data persistence in case of server crash? Snapshots are not reliable for that, slave db servers aren't foolproof either.

Lots and lots of replication, more or less. I don't know how it works in Postgres, but with something like Mongo, you set up a replication cluster and presume that up to (half - 1) of the nodes can fail and still maintain uptime. Postgres, being a relational database rather than a document store, likely has an additional set of challenges to overcome there, but it's very possible to do.

Re: Reddit: 2012 State of the Servers

#27

Post by cluda01 »

dangrossman wrote: post #21
plasma wrote: post #16

Earlier quoted context omitted.

Usually 1,000 ad impressions is around $1.00 (varies greatly though, can be lower like $0.15c and higher like $4.00+). Assuming $1.00 per 1,000 impressions, and taking their 2.07billion impressions/month figure: Roughly 2,070,000,000 / 1,000 = $2,070,000 in ad revenue per month? Wild guess.

Many/most page views on Reddit don't have any ads. Promoted stories only appear on story lists, not individual story pages, and don't appear 100% of the time (the space is also used to promote random new submissions). The graphical slot in the sidebar is almost 100% non-paid in-house ads.

I've always wondered why they don't contract out with other ad networks when they cannot fill the ad content themselves. Say for example their self serve ad can't fill the page request why not put in a google text ad link on the right side where the banner is? That to me seems like a straightforward way to massively increase revenues.

Re: Reddit: 2012 State of the Servers

#28

Post by rogerbinns »

tibbon wrote: post #22
joevandyk wrote: post #13

They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.

By local storage, do they mean they are running their DB disks on physical hardware, but the database servers still on EC2?

The Amazon servers have local disks physically attached. They are wiped between customers, on machine failure etc hence "ephemeral". The EBS (elastic block storage) is accessed as a disk but is over the other end of a network connection. Amazon does more to ensure the contents are available and durable (eg replication, backup to S3). The problem with EBS is that performance especially latency is highly variable and unpredictable.

Re: Reddit: 2012 State of the Servers

#29

Post by redslazer »

cluda01 wrote: post #27
dangrossman wrote: post #21

Earlier quoted context omitted.

Many/most page views on Reddit don't have any ads. Promoted stories only appear on story lists, not individual story pages, and don't appear 100% of the time (the space is also used to promote random new submissions). The graphical slot in the sidebar is almost 100% non-paid in-house ads.

I've always wondered why they don't contract out with other ad networks when they cannot fill the ad content themselves. Say for example their self serve ad can't fill the page request why not put in a google text ad link on the right side where the banner is? That to me seems like a straightforward way to massively increase revenues.

They dont do it because they really care about the user. Just sticking up random google ads isnt going to make anybody happier and with an internet savy crowd like reddit ad clicks are likley to be low. Sure very targeted ads like the ones that self-serve currently delivers work because its redditors advetising to redditors.
Post reply on HN