Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a MongoDB Clone in Postgres

legitimatesounding.com

Post reply on HN
Print view View original

41–50 of 99 posts

Re: Building a MongoDB Clone in Postgres

#41

Post by autarch »

jeffdavis wrote: post #14

Relational databases have a history of absorbing the advantages of other systems when they come along, particularly changes to the model (cf. object databases and XML databases). As the author shows, a similar thing will happen quite quickly for document models. Architectural changes are slower, but you can also start to see this happening in postgres with features like unlogged tables (i.e. don't write to the recove…

I'd phrase this differently. Once you implement Mongo in Postgres, what you have isn't a relational database.

What you are taking advantage of is the incredibly solid underlying infrastructure of Pg, which provides reliability, scalability, transactions, replication, etc.

But the data model itself is no longer relational from the application's perspective.

Re: Building a MongoDB Clone in Postgres

#43

Post by jeltz »

nsanch wrote: post #16
jshen wrote: post #10

Earlier quoted context omitted.

"but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story." There are many serious sites that don't need sharding.

Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.

I would say most sites will do just fine without sharding. You can get very far by just scaling up with a more expensive database server and caching the most common read operations. Some of the largest websites in the world do not need to do more than this.

Re: Building a MongoDB Clone in Postgres

#44

Post by taligent »

bad_user wrote: post #33
mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way. However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs. Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or a…

I guess if only I "woke up" I would realise that PostgreSQL is the answer to all my problems. Guess what. PostgreSQL is just another SQL database. It's definitely one of the best ones but it still stuffers from all the same issues, limitations and frustrations. Many of which stem not from the database itself but from the relational modelling and tools to support it.

I use Java + MongoDB and life is significantly better now that I don't have to worry about the domain model so much. I can have lists, maps etc and can add classes or make changes seamlessly. It's worth the tradeoffs for me.

Re: Building a MongoDB Clone in Postgres

#45

Post by taligent »

jeltz wrote: post #43
nsanch wrote: post #16

Earlier quoted context omitted.

Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.

I would say most sites will do just fine without sharding. You can get very far by just scaling up with a more expensive database server and caching the most common read operations. Some of the largest websites in the world do not need to do more than this.

Like ? I don't know of ANY decent sized website that uses a single database server.

Re: Building a MongoDB Clone in Postgres

#46

Post by semiquaver »

mark_l_watson wrote: post #38
mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

I agree: Mongoid is great. I haven't done this in a few years, but Datamapper had the nice capability to map a singe model classes to multiple backends (e.g., use PostgreSQL for some attributes, MongoDB for others). PostgreSQL and MongoDB cover most of my data store requirements and are a pleasure to use.

I use datamapper pretty extensively to join mysql tables with mongodb collections, but I've always used associations between several models to do this. Do you have any links talking about this capability?

Re: Building a MongoDB Clone in Postgres

#47

Post by taligent »

jshen wrote: post #39
nsanch wrote: post #34

Earlier quoted context omitted.

Sorry, those were meant to be examples, not an exhaustive list.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

You're joking, right ?

Every large site on the internet has talked about strategies around sharding. At some point you are surely going to hit the physical limitations of one database on one server.

Re: Building a MongoDB Clone in Postgres

#48

Post by k_bx »

bad_user wrote: post #33
mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way. However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs. Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or a…

> My main problem with most NoSQL solutions is that I have to tweak the problems I have to fit the solution, instead of the other way around. Technologies that can be tweaked simply rock.

Well, I would disagree. While it's great that sql databases let you do lots of joins, I would lose imagination of my query complexity very fast (also, you can't review all sql-queries stopping people from doing joins and "where tablename.status=active" in every joined table). While MongoDB would restrict you from that, but gives you "documents". I'd say it's a lot of power of complexity-clarity (having no joins, but power-enough documents).

p.s.: and yes, I agree with you that developers are ok with postgres on most of cases. more of that, it's better to have transactions and other stuff until it's performance-urgent to disable it).

Re: Building a MongoDB Clone in Postgres

#49

Post by masklinn »

taligent wrote: post #44
bad_user wrote: post #33

Earlier quoted context omitted.

Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way. However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs. Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or a…

I guess if only I "woke up" I would realise that PostgreSQL is the answer to all my problems. Guess what. PostgreSQL is just another SQL database. It's definitely one of the best ones but it still stuffers from all the same issues, limitations and frustrations. Many of which stem not from the database itself but from the relational modelling and tools to support it. I use Java + MongoDB and life is significantly bett…

> Guess what. PostgreSQL is just another SQL database.

Uh... hstore says "I'm not quite sure what the bloody hell you're talking about". So does ltree.

Re: Building a MongoDB Clone in Postgres

#50

Post by nsanch »

jshen wrote: post #39
nsanch wrote: post #34

Earlier quoted context omitted.

Sorry, those were meant to be examples, not an exhaustive list.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.

My definition of seriousness includes some relatively large scale. Your definition of seriousness appears to mean any site that is important to the person or business running it. Is that a fair assessment?

I think your definition is better, and I should've said "a potentially-large site" or something like that.

Post reply on HN