Building a MongoDB Clone in Postgres • Hacker News Board

Re: Building a MongoDB Clone in Postgres

#31

Post by ibotty » Mon, Jun 04, 2012, 7:19 PM UTC

nsanch wrote: post #9

jerrysievert wrote: post #4

Earlier quoted context omitted.

while sharding is an important aspect of mongodb, i don't consider it the most important feature.

I don't know if it's the _most_ important feature, but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story. With postgres you have to roll your own. If you want to bridge the gap from postgres to mongo, I think that's where you have to start.

"With postgres you have to roll your own [sharding]"

but it will be trivial if you do not do joins.

Re: Building a MongoDB Clone in Postgres

#32

Post by roncohen » Mon, Jun 04, 2012, 7:21 PM UTC

If you just need a simple key/value, a good way to go is HSTORE in PostgreSQL. It allows only string key/values, and not complex structures. Use the JSON datatype if you need lists, nested objects etc.

With HSTORE you get indexes on keys.

Re: Building a MongoDB Clone in Postgres

#33

Post by bad_user » Mon, Jun 04, 2012, 7:27 PM UTC

mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way.

However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs.

Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or about how the modeling tools and architecture exposed by PostgreSQL are insufficient for certain problems ... not until you're operating at Google's scale, and MongoDB won't save you there ;)

     give it a try before buying that you can do it all 
     with Postgres tweaks

My main problem with most NoSQL solutions is that I have to tweak the problems I have to fit the solution, instead of the other way around. Technologies that can be tweaked simply rock.

Re: Building a MongoDB Clone in Postgres

#34

Post by nsanch » Mon, Jun 04, 2012, 7:28 PM UTC

jshen wrote: post #27

nsanch wrote: post #16

Earlier quoted context omitted.

Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.

" Sites that are read-only or store blob data in something like S3 can often avoid sharding" Still too broad. Sorry, this is a pet peeve of mine, where tech people assume that everyone else has the same issues as them. For example, I worked on an ecommerce site that made over a mil a year. They had less than 10k products, and will never need sharding. They are not read-only, they have people updating their products o…

Sorry, those were meant to be examples, not an exhaustive list.

Re: Building a MongoDB Clone in Postgres

#35

Post by dickeytk » Mon, Jun 04, 2012, 7:47 PM UTC

mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

I really wish I could agree with you, but in my experience, ActiveRecord is just way easier to deal with than Mongoid.

Re: Building a MongoDB Clone in Postgres

#36

Post by dickeytk » Mon, Jun 04, 2012, 7:48 PM UTC

jerrysievert wrote: post #4

nsanch wrote: post #3

Unless he's planning to build sharding on postgres too, I think he's missing the point.

while sharding is an important aspect of mongodb, i don't consider it the most important feature.

It also seems to be the flakiest feature

Re: Building a MongoDB Clone in Postgres

#37

Post by DEinspanjer » Mon, Jun 04, 2012, 7:49 PM UTC

rhizome wrote: post #15

DEinspanjer wrote: post #8

It is interesting, but what I really need in the JSON functionality of PG is some internal representation that will allow fast and efficient exploration of the JSON blob within the query. i.e. being able to refer to a single attribute within the select/where/groupby clauses without having to pay the toll of serde every time.

Isn't this a category error? If you need to query within serialized data, you don't want to serialize. Normalize your database for this.

Well, what I need is expressive and flexible querying over document data that doesn't have a rigid schema. There are several document stores that provide filtering and retrieval matching that, but they all have various limitations when it comes to aggregation.

Re: Building a MongoDB Clone in Postgres

#38

Post by mark_l_watson » Mon, Jun 04, 2012, 8:09 PM UTC

mtkd wrote: post #25

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (…

I agree: Mongoid is great.

I haven't done this in a few years, but Datamapper had the nice capability to map a singe model classes to multiple backends (e.g., use PostgreSQL for some attributes, MongoDB for others).

PostgreSQL and MongoDB cover most of my data store requirements and are a pleasure to use.

Re: Building a MongoDB Clone in Postgres

#39

Post by jshen » Mon, Jun 04, 2012, 8:10 PM UTC

nsanch wrote: post #34

jshen wrote: post #27

Earlier quoted context omitted.

" Sites that are read-only or store blob data in something like S3 can often avoid sharding" Still too broad. Sorry, this is a pet peeve of mine, where tech people assume that everyone else has the same issues as them. For example, I worked on an ecommerce site that made over a mil a year. They had less than 10k products, and will never need sharding. They are not read-only, they have people updating their products o…

Sorry, those were meant to be examples, not an exhaustive list.

Let me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story".

I think the opposite. Most serious sites will never need sharding.

Re: Building a MongoDB Clone in Postgres

#40

Post by mark_l_watson » Mon, Jun 04, 2012, 8:15 PM UTC

nsanch wrote: post #3

Unless he's planning to build sharding on postgres too, I think he's missing the point.

You have a good point. I must admit that after many years of using and loving PostgreSQL I have never had to scale it out: that is outside of my experiences.

On the other hand, old fashion MongoDB master slave configurations and now replica sets have been easy for me to set up when on just a few occasions I had to use a scaled out MongoDB setup.