Live data from Hacker News

A proof of concept MongoDB clone built on Postgres

github.com

41–50 of 51 posts

Re: A proof of concept MongoDB clone built on Postgres

#41

Earlier quoted context omitted.

Write a function in PLV8?

currently, that is exactly how they are handled. the JSON structure is brought into memory, parsed, modified, and then written again to the database. this is non-optimal, but currently the only method available.

That wouldn't change even if there was a specialized syntax, because of MVCC. So the only problem is that syntactic sugar is lacking, not that the performance isn't as good as it could be.

Re: A proof of concept MongoDB clone built on Postgres

#42
I thought MongoDb:s main selling point was the simple scale-out model? At least that's what stuck with me. I'm well aware about the non-robustness properties of Mongo, but to me it seems like calling this a clone without the scale-out capabilities would be missing the point.

Re: A proof of concept MongoDB clone built on Postgres

#44
Álvaro here from ToroDB.

Check out ToroDB (github.com/torodb/torodb). It is a Mongo implementation based on PostgreSQL that transforms JSON documents into relational tables. This has many advantages like significant storage reduction, less I/O required and it allows for updates (a concern raised on some comments below). Please check it out! :)

Re: A proof of concept MongoDB clone built on Postgres

#45

I thought MongoDb:s main selling point was the simple scale-out model? At least that's what stuck with me. I'm well aware about the non-robustness properties of Mongo, but to me it seems like calling this a clone without the scale-out capabilities would be missing the point.

It's a proof of concept. The author's essentially saying, "here, it can be done, this is one way of doing it." It would need more bits and pieces and more tuning to be useful in a production case, but this shows that it at least can work.

Re: A proof of concept MongoDB clone built on Postgres

#46
post #27

Earlier quoted context omitted.

Or Python, or any language that has a PL in PostgreSQL. Unfortunately a lot of people consider that a hack, and they're not wrong. If we can address fields with operators (->, ->>, etc.) then why can't those operators modify, too? It works for everything else, after all. It's fairly counter-intuitive from a dev perspective. That, and PLV8 still doesn't support JSONB. With JSONB being so much more efficient, faster, a…

> If we can address fields with operators (->, ->>, etc.) then why can't those operators modify, too? It works for everything else, after all. It's fairly counter-intuitive from a dev perspective. It certainly doesn't work for "everything else". You can update individual "parts" of arrays and record types, that's it. And those exception are hard-coded, so someone would either have to hard-code similar exceptions for…

Why is special-casting json or jsonb worse than special-casing arrays?

Re: A proof of concept MongoDB clone built on Postgres

#47
post #12

DB2 also implements the MongoDB query language: http://www.ibm.com/developerworks/data/library/techarticle/d... So does CouchDB: https://cloudant.com/blog/couchdb-and-mongodb-let-our-query-...

FWIW, I've recently been implementing the MongoDB query language as well: https://github.com/zumero/Elmo In F#. Not even remotely close to usable or production-ready. The approach here is somewhat different, as this implementation is built on SQLite, which it treats as a simple key-value storage layer.

Thanks for posting that, it's interesting and instructive to see examples of f# code 'out in the wild'. Much appreciated!

Re: A proof of concept MongoDB clone built on Postgres

#48
post #6

How are JSON updates handled? For those who aren't aware, Postgres currently lacks support for doing updates to JSON fields via SQL[1]. For many this isn't a problem, but I'd imagine that people expecting a MongoDB clone would need it. [1] But even though you can individually address the various fields within the JSON document, you can’t update a single field. Well, actually you can, but by extracting the entire JSON…

Write a function in PLV8?

You can also write an update function in plain SQL. See my example in this comment on a recent plant postgresql post: http://bonesmoses.org/2015/04/10/pg-phriday-functions-and-ad...

Re: A proof of concept MongoDB clone built on Postgres

#50
post #28

Earlier quoted context omitted.

problem : meteor relies on a specific mongodb feature,ie polling the db for any change. I heard there is experimental support for PG but it seems to rely on a complex hack using triggers ... So the issue isn't really about mongo queries, but whether it is possible to track db edits from third parties or not.

There's infrastructure for that in postgres, since 9.4. See http://www.postgresql.org/docs/devel/static/logicaldecoding.... for the (somewhat low level) description of the feature. You'd have to write an output plugin that formats the output as json, but that should be pretty easy. Disclaimer: I'm the author of the feature ;)

That specific MongoDB feature is "tailing the oplog", where the oplog is the capped collection with the idempotent commands that represent the changes in the database. This is what Meteor uses to receive (be pushed) changes (not polling).

Certainly, logical decoding provides the necessary infrastructure to implement it (and also normal tailable cursors): thank you Andres, really nice work! :) But some work is also needed to transform the representation you are using in PostgreSQL into MongoDB's oplog entries.

This is definitely what we are using in ToroDB to emulate it (currently, under development).

Post reply on HN