Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Mongita is to MongoDB as SQLite is to SQL

github.com

Post reply on HN
Print view View original

61–70 of 95 posts

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#61

Post by scottrogowski »

hardwaresofton wrote: post #43
scottrogowski wrote: post #23

Earlier quoted context omitted.

Thank you! I actually did consider doing exactly what you said. Early on, I decided one my goals for the project would be to make it easy to swap back and forth between Mongita and PyMongo/MongoDB. For me, that meant getting as close as possible to their implementation and using things like BSON, ObjectIds, etc. For that, I sacrificed some performance but I think most people who would use something like Mongita would…

Oh thanks for the insight, that makes a lot of sense -- pretty sure mongita is one of the first projects I've seen even attempt this, and it makes sense to target the platform that fits your use case (and obviously everyone who uses PyMongo) the tightest. Your project looks very high quality -- benchmarks, tests, and comparisons are basically an indicator in my mind. Looking through the code you've also already left…

Thanks for the kind words :)

I like your idea. As I understand it, since SQLite performs better on most benchmarks anyway, why not have an engine subclass that utilizes SQLite itself?

This would be a little trickier than your envisioning. The engine class is the lowest layer but doesn't handle the slow bits like the finds, managing the indicies, etc. So to really get the benefit of SQLite you would have to pull those slow bits in.

I do really like the idea of offering a third option so that you have - Memory (fastest) - SQLite (almost as fast but compromises on perfect MongoDB reproduction) - Disk (slower but is faithful to MongoDB)

Happy to discuss it more if you want to email me.

> How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see)

I used type annotations for a while when they came out but didn't find them to offer advantages over docstrings. The world might have moved over to type annotations though and I've not been aware.

> PyPy? I wonder if you'd get a ~free speedup

I'm getting a lot of suspicion right now from other commenters who are convinced I have my thumb on the benchmarks and I don't want to give them any more excuses :tears-of-joy:. But joking aside, you're probably right and it's not something I had considered.

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#62

Post by deknos »

scottrogowski wrote: post #58
deknos wrote: post #56

what i always think about stuff like this: if stuff like mongita and sqlite would exist for all kinds of databases (graph,kv,xml; as for document and sql it already exists), couldn't we "just make distributed versions" if we put stuff ontop of it? like with dqlite/rqlite with sqlite? or does there have to be some inherent mechanisms withIN the database to support distributed versions?

I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases. This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have…

> I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases.

Exactly!

> This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks.

well, that would be also the case with traditional db services, the question can they have more granular mechanisms for more granular locking than embedded databases. but perhaps they even can have only less granular locking?

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#63

Post by deknos »

ezrast wrote: post #60
deknos wrote: post #56

what i always think about stuff like this: if stuff like mongita and sqlite would exist for all kinds of databases (graph,kv,xml; as for document and sql it already exists), couldn't we "just make distributed versions" if we put stuff ontop of it? like with dqlite/rqlite with sqlite? or does there have to be some inherent mechanisms withIN the database to support distributed versions?

There are generic distributed consensus algorithms out there. The most famous are Paxos and Raft. In theory, you can jam those on top of any system you like, as long as it has well-defined state transitions. Making it fast - or usable at all in the presence of heavy contention - is another story. Distributing a write-heavy workload over a cluster is useless if the cluster ends up rejecting most updates because they g…

thank you!

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#64

Post by chx »

pbreit wrote: post #47
saurik wrote: post #39

Earlier quoted context omitted.

I am not even sure what is going on now, as I thought the only reason people had ever cared about MongoDB in the first place was in an attempt to get performance they thought was impossible using a relational database (notably without realizing all of the tradeoffs... some inexcusable, such as simply coming with defaults that didn't call fsync, leading to a ton of memes about "database administrators running with sci…

The correct comparison is "Mongita is to MongoDB as SQLite is to Postgres/Mysql". "performance or scalability" are not benefits of SQLite. The benefit of SQLite is that it is a "small, fast, self-contained, high-reliability, full-featured, SQL database engine".

> "performance or scalability" are not benefits of SQLite.

Where scalability is indeed nonsensical for an embedded database, performance very much is a feature of it.

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#66

Post by jbverschoor »

afaict, the disk_engine does not lock the file when reading / writing. Also, the storage engine is bson and relies on cached offsets. There's also this non-atomic defrag method.

I dunno.. wouldn't touch it with a pole wearing a hazmat suit, sorry.

Using sqlite for storage and querying would've been better. Heck, that would be pretty great for moving a few smalller (server) applications off of mongodb. Although they're using ruby

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#67

Post by jbverschoor »

abrookewood wrote: post #12
saurik wrote: post #10

So, by their own benchmarks, unless you are doing totally random lookups of documents by identifier--and mostly reads, with very few writes--you should absolutely use SQLite with JSON values, which absolutely destroy this project in performance?...

The project isn't about performance, it's about providing an embedded version of MongoDB - it clearly states that if you grow too large, then you can easily migrate to the full MongoDB. It also clearly states not to use it if you want a relational database.

Then why not just create a library which uses sqlite as a proven storage and query engine?

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#68

Post by giorgioz »

Very cool, I'm looking for MongoDB to be able to run on AWS Lambda and be more serverless. This seem like a step in the good direction. Personally I think JS would have been a better choice to implement this in than Python given all the mongo queries are in javascript db.collection('users').find({_id:'AN_ID'}) Also having it made in javascript would have open the option to embed it in the browser later on.

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#69

Post by necovek »

kabes wrote: post #42
saurik wrote: post #39

Earlier quoted context omitted.

I am not even sure what is going on now, as I thought the only reason people had ever cared about MongoDB in the first place was in an attempt to get performance they thought was impossible using a relational database (notably without realizing all of the tradeoffs... some inexcusable, such as simply coming with defaults that didn't call fsync, leading to a ton of memes about "database administrators running with sci…

Having the best performance isn't necessary for a lot of use cases. Sometimes you just want to store and search a bunch of json objects and for that the mongo api is way more convenient then the postgres/sqlite json options

I am not too familiar with MongoDB, but how does the API compare to an ancient key-value store library of Berkeley DB (db.get/put/delete/...)?

(Asking in the context of Mongita, or rather, a file-based key-value store)

Re: Show HN: Mongita is to MongoDB as SQLite is to SQL

#70

Post by krageon »

pbreit wrote: post #47
saurik wrote: post #39

Earlier quoted context omitted.

I am not even sure what is going on now, as I thought the only reason people had ever cared about MongoDB in the first place was in an attempt to get performance they thought was impossible using a relational database (notably without realizing all of the tradeoffs... some inexcusable, such as simply coming with defaults that didn't call fsync, leading to a ton of memes about "database administrators running with sci…

The correct comparison is "Mongita is to MongoDB as SQLite is to Postgres/Mysql". "performance or scalability" are not benefits of SQLite. The benefit of SQLite is that it is a "small, fast, self-contained, high-reliability, full-featured, SQL database engine".

SQLite performs very well and it scales vertically to millions of ops pretty easily. Then you can say "well, we'll worry about more performance later". In this case, perhaps not so much.
Post reply on HN