Live data from Hacker News

I’ll Give MongoDB Another Try In Ten Years

diegobasch.com

81–90 of 194 posts

Re: I’ll Give MongoDB Another Try In Ten Years

#81
I think blaming the user here is partially valid (he didn't read the docs), but that's not the whole story.

There is a discontinuity between the ease-of-use story and the blame-the-user story, regardless of how well documented the async insert behavior is.

And it doesn't have to be this way. There are ways of designing interfaces, APIs, and even naming that go a long way to prevent your users from shooting themselves in the foot.

Take postgres. It also supports at least a couple kinds of async insert, one of which is a part of libpq (postgres C client library). It's called "sendQuery" and it's documented under the "Asynchronous Command Processing" section. It's hard to imagine a user trying to use that and expecting it to return an error code or exception. Even if the user doesn't read the docs, or reads some fragment from a blog post, they will still see that the name suggests async and that it returns an int rather than a PGResult (which means it obviously doesn't fit into the normal sync pattern).

There is no reason mongo couldn't be clear about this distinction -- say, rename "insert" to "async_insert" and have "insert" be a wrapper around async_insert and getLastError. But instead, it's the user's fault because they didn't read the docs.

Careful API design is important to reduce the frequency of these kinds of errors. In postgres, it's relatively hard to shoot yourself in the foot this badly in such a simple case. I'm sure there are gotchas, but there is a conscious effort to prevent surprises of this sort.

Re: I’ll Give MongoDB Another Try In Ten Years

#82

Earlier quoted context omitted.

> Nowhere in the documentation does it mention that it will silently discard your data. Demonstrably false. http://www.mongodb.org/display/DOCS/getLastError+Command "MongoDB does not wait for a response by default when writing to the database. Use the getLastError command to ensure that operations have succeeded."

It blows my mind that you'd have to post this at all. Who's writing to mongo without making sure the write succeeded?

People who think reading is overrated.

Re: I’ll Give MongoDB Another Try In Ten Years

#83

It's almost like the commenters who are bashing the author of the post did not read the bit in bold, which is his main point: If you tell a database to store something, and it doesn’t complain, you should safely assume that it was stored. This has nothing to do with the 2Gb limitation. Nowhere in the documentation does it mention that it will silently discard your data. What will happen with the 64-bit version if you…

"I'm just struggling to imagine being willing to lose some amount of data purely for the sake of performance...".

A foursquare check-in database could be an example where performance is actually way more valuable than consistency. (I have no idea what database they use)

Re: I’ll Give MongoDB Another Try In Ten Years

#85
I ran into this same nasty surprise building a prototype to store requests in Mongo instead of Postgres. It was enough to scare me away, too. Glad I noticed it while it was still just a script+Makefile simulation.

Another problem with Mongo I never heard anyone else raise is that there are no namespaces. If I install Mongo, all the tables/collections live in the same namespace. What if I want to use it for multiple projects? How do other people solve this problem?

Re: I’ll Give MongoDB Another Try In Ten Years

#86
I had a very similar experience about a year ago. Except instead of running out of RAM on a 32-bit instance, I was running out of disk on a 64-bit instance. That's right, my database ran out of disk, and the driver didn't throw an exception.

Yes, I realize that there's a "safe=True" option to my python driver. But I'm writing to a database. As others have said here and elsewhere, the default behavior of a database and its drivers should be to complain loudly when a write fails. It is ridiculous that safe!=True by default. If I want to turn off this feature to improve performance, I will.

Re: I’ll Give MongoDB Another Try In Ten Years

#87
post #13
post #4

Earlier quoted context omitted.

Except that it was hidden deeply in the documentation. It's like making a car that has the gas and brake pedals switched, and then blaming accidents on people not reading section 5 of the owner's manual. I'm hardly an inexperienced programmer. I've used Cassandra, SimpleDB, Voldemort, etc. I wrote part of the Inktomi Search Engine in the 90s, and plenty of (what today would be called) NoSQL stores over the years. A d…

As the OP points out the limit is pretty clearly specified on the download page and there's a "note" linking to the limit right next to where it says "32-bit". Sometimes you just have to admit you screwed up and didn't read the documentation. Everyone does it, we're hackers, we'd much rather play with technology than read docs.

Being limited to 2GB, and failing silently after 2GB are different things. I know about the 2GB limit but I also would have expected an error. (Though I think I managed to enable safe mode for my internal app.)

Re: I’ll Give MongoDB Another Try In Ten Years

#88
post #60

Earlier quoted context omitted.

Well, any time you'd rather have speed over completeness. Maybe you're aggregating tweets from the Twitter API and if the occasional one goes missing, it's not a big deal, or perhaps you can grab it on the next update. Maybe you're generating a real-time stats dashboard for your site and if one pageview gets lost every million, it's not a big deal. Look, I agree that in most cases you probably want to do everything y…

This wasn't the occasional write though, this was every write after 2GB. That should set off alarm bells somewhere.

The way MongoDB is designed, this would be outside the scope of a driver. A storage library that's based on the ruby MongoDB driver could certainly do it, though. That's what the OP ideally should have been using. In fact MongoDB would be a good choice for his use case, if he would switch to a 64-bit VM and handle error conditions (heh).

Re: I’ll Give MongoDB Another Try In Ten Years

#89
post #79

Earlier quoted context omitted.

This may be a case when using the package manager is not always the best option. If you're talking about Ubuntu, I can attest that the default PM there is several versions out of date for a lot of things, and thus to get the version you'd expect, you're forced to install by hand. Also, even using the PM version, didn't you get a warning when you started the server? I thought Mongo threw up a warning at start time abo…

Second every word here. The version in the Ubuntu's repositories is not the latest(which is 2.2) and they can't be more explicit about it than pointing it out on the download page and giving a message upon the database startup.

The version in the Ubuntu's repositories is not the latest(which is 2.2)

What does the author's complaint have to do with the version Ubuntu is distributing? Are the 32-bit limitations present in Ubuntu's version not present in the most recent version? If they are, than who cares which of them he installed?

they can't be more explicit about it than pointing it out on the download page and giving a message upon the database startup

Uhh, yeah they can. On Debian-derived systems like Ubuntu you can make your .deb packages throw up dialogs that the user has to read and agree to before installation via debconf (http://www.fifi.org/cgi-bin/man2html/usr/share/man/man8/debc...). There's probably a way to do the same thing in RPM-based systems as well. If the warning is something that every user of the software needs to see, putting up a warning dialog and requiring the user to confirm having seen it before installation starts would probably be appropriate.

They could also write an error to the database's error log whenever data is discarded due to the 32-bit limitation. Someone mentioned above that it puts a message at the start of the log, but if that's the case IMO it's insufficient; most of the time people interact with logs by looking in them for a particular moment in time, not by reading them from the first line on. Logging the error on or near the moment the data loss happens would make the issue visible to people using logs in this manner.

Re: I’ll Give MongoDB Another Try In Ten Years

#90

Earlier quoted context omitted.

> Nowhere in the documentation does it mention that it will silently discard your data. Demonstrably false. http://www.mongodb.org/display/DOCS/getLastError+Command "MongoDB does not wait for a response by default when writing to the database. Use the getLastError command to ensure that operations have succeeded."

I fail to understand how and why silent failure is considered a reasonable default.

It actually happens in some scenarios.

Let's say, outside of the tech world: When you send a post card (cheap ones) to a friend, you won't receive any delivery confirmation. You just send it and go do whatever you please, believing the post card will be there. If the envelope don't get there, no biggies, you will send another on your next trip anyways. No hurt feelings.

But, let's say you need to send me a check. You want to know if I received it or not, specially because sometimes I don't cash checks right away. Without confirmation it would be difficult to you to decide if you cancel the previous check and send another, or do nothing, because I could be at that very time trying to cash the check or it could be lost somewhere. The delivery confirmation is an add-on where you receive a confirmation that the envelope got there, but see, it will take time for that confirmation to arrive. It's expensive. If you are sending a 0.01 check, you can just send another if the recipient asks.

Post reply on HN