Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a MongoDB Clone in Postgres

legitimatesounding.com

Post reply on HN
Print view View original

51–60 of 99 posts

Re: Building a MongoDB Clone in Postgres

#51

Post by nsanch »

taligent wrote: post #45
jeltz wrote: post #43

Earlier quoted context omitted.

I would say most sites will do just fine without sharding. You can get very far by just scaling up with a more expensive database server and caching the most common read operations. Some of the largest websites in the world do not need to do more than this.

Like ? I don't know of ANY decent sized website that uses a single database server.

An example is stack overflow. They've been able to scale up instead of scaling out.

http://highscalability.com/blog/2009/8/5/stack-overflow-arch... is the link I can find right now.

Re: Building a MongoDB Clone in Postgres

#52

Post by leothekim »

jeffdavis wrote: post #24
nsanch wrote: post #16

Earlier quoted context omitted.

Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.

"That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad." I think you're being too absolute. For instance, Instagram used sharding in postgres, and they didn't have to throw anything away or dedicate any huge engineering team to solve it.

They had to put engineering effort into it. With Mongo, you don't.

Re: Building a MongoDB Clone in Postgres

#53

Post by leothekim »

jshen wrote: post #39
nsanch wrote: post #34

Earlier quoted context omitted.

Sorry, those were meant to be examples, not an exhaustive list.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

What do you mean by "serious"? That's broadly stated as well.

Re: Building a MongoDB Clone in Postgres

#54

Post by jshen »

taligent wrote: post #47
jshen wrote: post #39

Earlier quoted context omitted.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

You're joking, right ? Every large site on the internet has talked about strategies around sharding. At some point you are surely going to hit the physical limitations of one database on one server.

He said "serious site". He did not say "large site". For example, techcrunch is most certainly a serious site. It is NOT large scale and in need of sharding.

Re: Building a MongoDB Clone in Postgres

#55

Post by jshen »

nsanch wrote: post #50
jshen wrote: post #39

Earlier quoted context omitted.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

My definition of seriousness includes some relatively large scale. Your definition of seriousness appears to mean any site that is important to the person or business running it. Is that a fair assessment? I think your definition is better, and I should've said "a potentially-large site" or something like that.

Yes, this. Thanks for listening. It's a pet peeve of mine because most people work on serious sites, and a minority of them have large scale data, but people often imply that everyone needs to solve massive scaling problems even if they will never have them.

Re: Building a MongoDB Clone in Postgres

#56

Post by jshen »

leothekim wrote: post #53
jshen wrote: post #39

Earlier quoted context omitted.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

What do you mean by "serious"? That's broadly stated as well.

techcrunch is a serious site. They don't need sharding. They have thousands of posts, not millions, and certainly not billions or trillions.

Re: Building a MongoDB Clone in Postgres

#57

Post by jshen »

taligent wrote: post #45
jeltz wrote: post #43

Earlier quoted context omitted.

I would say most sites will do just fine without sharding. You can get very far by just scaling up with a more expensive database server and caching the most common read operations. Some of the largest websites in the world do not need to do more than this.

Like ? I don't know of ANY decent sized website that uses a single database server.

I'm not sure why you said "a single database server". No one said that. We're talking about sharding across databases, and the vast majority of sites don't need that.

Having a master/slave setup is more than a single server, but it's NOT the same as sharding.

Re: Building a MongoDB Clone in Postgres

#58

Post by jeltz »

taligent wrote: post #45
jeltz wrote: post #43

Earlier quoted context omitted.

I would say most sites will do just fine without sharding. You can get very far by just scaling up with a more expensive database server and caching the most common read operations. Some of the largest websites in the world do not need to do more than this.

Like ? I don't know of ANY decent sized website that uses a single database server.

On top of my head I can think of two: Wikipedia and leboncoin.fr (one of the largest websites in france).

EDIT: Both run read-only slaves. I was only talking about no sharding.

Post reply on HN