Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The genius and folly of MongoDB

nyeggen.com

Post reply on HN
Print view View original

261–270 of 280 posts

Re: The genius and folly of MongoDB

#261

Post by pg »

byroot wrote: post #176
pg wrote: post #101

Earlier quoted context omitted.

That's caused by using closures to create dynamically generated "callbacks" on the server, not keeping data structures in RAM. If you ask for some old item not in memory, it just gets lazily loaded.

Sure you have full permalink support, but why do you have to rely on closure to do pagination ? My guess: because by relying on in-memory data-structures you can't do what any half assed php forum do, ad hoc queries.

Because it's much cleaner and more powerful if the thing that generates the next page is a closure rather than just an index. Among other things it lets you show each user a different set of items (depending on whether they have showdead turned on for example).

Re: The genius and folly of MongoDB

#262

Post by nostrademons »

swah wrote: post #241
nostrademons wrote: post #219

Earlier quoted context omitted.

I suspect he doesn't have to rely on closures to do pagination: they're a programming convenience that means you don't have to do things like think about what state persists between pages. Anything you can do with SQL you can do with in-memory data structures. If you're interested, I'll be happy to take any SQL query and convert it to some Python list comprehensions on arrays of dicts.

Some statements like self-joins become relatively compact in SQL though... BTW, do you miss Java's more advanced structures (say MultiSet) when programming in Python/Go?

Pretty rarely, at least in Python. I don't miss MultiSet, because Python has that (collections.Counter). Ditto LinkedHashMap (collections.OrderedDict). Those are the two "extended" collections that I most often use. I do miss the absence of balanced binary trees occasionally, since sometimes it's useful to have an associative container with a defined iteration order, but sorted(dict) is usually good enough where performance is critical. And Python's heapq module is a bit harder to use than Java's PriorityQueues, but all the functionality is there.

I think I'd miss these a bit more in Go because the built-in datatypes are privileges in some of the language statements, but I haven't written enough Go code to really feel their absence.

Re: The genius and folly of MongoDB

#263

Post by adron »

rdtsc wrote: post #106

The problem with MongoDB is their shadiness. The shipped with unacknowledged writes up until not too long ago. In other words you would write to it and there wouldn't be an ok or fail response, you'd just sort of hoped it would go in. They fixed that problem but it was too late. In my eyes they proved they are not to be trusted with data. Had they called themselves MangoCache or MongoProbabilisticStorage, fine, can s…

Mongo was sort of great at first, it's a not bad solution for low traffic sites. It's also great for prototyping. But when anything grows up into high availability, high traffic or anything high.

It's kind of funny, albeit interesting that it took this long for much of the industry to start knocking down the house of cards that is built around the database. However many of the databases that are coming out these days have a lot of the same cultural issue of "hide the issues" and "talk about the cool parts".

I'm expecting any minute now for the anti-schema-less pro-schema movement to rise up...

IMHO, it's all about what you're doing at the time, and making the right decision... albeit it helps if the decisions for a database isn't glossing over the issues as insignificant. :-/

btw - "MongoProbabilisticStorage" is a great name for a product!

Re: The genius and folly of MongoDB

#264

Post by Zak »

grmarcil wrote: post #254
Zak wrote: post #249

Earlier quoted context omitted.

That's fairly unusual; modern cars have a fairly standardized user experience such that there aren't many ways for a car to do something surprising and dangerous that's covered in the manual. I did experience one once though: I discovered that a vehicle had traction control when the system activated during a skid. The computer and I disagreed about the best way to respond, and the surprise did make the situation more…

I'm curious about your desired response during the skid and what the traction control system did differently. Would you mind expanding on that if you remember it clearly?

Sure. The vehicle involved was rear wheel drive and, as loaded probably had a rear-biased mass distribution. It began to oversteer in a corner on a patch of ice. Standard protocol for these situations is to apply a moderate amount of throttle to shift weight to the rear and increase traction there while reducing or reversing steering input. The traction control system counteracted my attempt to increase power, requiring vastly more reverse steering input and interfering with my ability to position the vehicle on the road.

I suspect some people will believe the results wouldn't have been what I expected without the traction control. I can't prove they would have been, but I did grow up and learn to drive in Alaska. Based on my experience, I think I would have done better than the computer did.

Here's somebody describing the basic idea involved while discussing the joys of mildly irresponsible driving on cloverleafs: http://www.scottgood.com/jsg/blog.nsf/d6plinks/SGOD-66RJ2Y

Re: The genius and folly of MongoDB

#265

Post by nostrademons »

philwelch wrote: post #252
nostrademons wrote: post #216

Earlier quoted context omitted.

Isn't this thread about non-serious use? Pretty much everything I see here is about how MongoDB is only suitable for prototypes, how it doesn't even guarantee writes, how they just want something quick & dirty to build a MVP with. The parent poster asked for something to replace MongoDB with - if the use-case is prototypes and "web scale" startups that don't have users or a product yet, I think a single server with i…

I would say that a MVP should be written in such a way that you don't have to waste time rewriting from scratch once the concept is validated. If you're writing a MVP to be disposable you aren't necessarily launching it on a real server with persistent storage anyway, more likely Heroku or at the very least AWS, but in either case you're well equipped to do the right thing from the outset rather than being forced to…

You will have to rewrite anyway. Multiple times. If you pick a RDBMS you will have to rewrite it to scale, if you pick MongoDB you will have to rewrite it for reliability, if you pick Heroku or AppEngine you will have to rewrite it to avoid paying them a good chunk of your profits.

That's probably the biggest surprise I learned from working in a fast-growing, well-functioning engineering organization. The half-life of code in a market that's actively growing and changing is roughly 1 year, i.e. 50% of the code you write now will have been removed within a year from now. And attempts to optimize for problems you're going to have in a year, rather than the ones you have now, actively make things worse because you inevitably have a different product direction in a year, and baking in last year's speculative assumptions just means there's more code you have to work around.

Re: The genius and folly of MongoDB

#266

Post by revetkn »

functional_test wrote: post #243
revetkn wrote: post #199

Earlier quoted context omitted.

> What's impractical about spending a little time to read? It's impractical because people live a finite amount of time and this is a terrible use of it

If understanding your production database is a bad use of your time, then I really don't understand your priorities, but I'm glad you're not on my team. You don't need to read every word of everything, but some things are worth it. Do you sign contracts without reading them too since it's a "terrible use of" your time?

> You don't need to read every word of everything, but some things are worth it.

Yes - I am saying in this particular car example, the benefit derived reading the entire manual before purchasing/driving a car is not worth the cost unless your time is worth very little. As others have pointed out, no one is flipping through the manual to check whether the brake pedal actually applies the brakes

But it is a good engineering decision to thoroughly read the docs before jumping into a new datastore like Mongo, I agree. Learning there are things like gigantic global locks and unsafe writes are normally enough to make you say, "hey, I probably shouldn't use this to store production data I actually care about"

Re: The genius and folly of MongoDB

#267

Post by philwelch »

nostrademons wrote: post #265
philwelch wrote: post #252

Earlier quoted context omitted.

I would say that a MVP should be written in such a way that you don't have to waste time rewriting from scratch once the concept is validated. If you're writing a MVP to be disposable you aren't necessarily launching it on a real server with persistent storage anyway, more likely Heroku or at the very least AWS, but in either case you're well equipped to do the right thing from the outset rather than being forced to…

You will have to rewrite anyway. Multiple times. If you pick a RDBMS you will have to rewrite it to scale, if you pick MongoDB you will have to rewrite it for reliability, if you pick Heroku or AppEngine you will have to rewrite it to avoid paying them a good chunk of your profits. That's probably the biggest surprise I learned from working in a fast-growing, well-functioning engineering organization. The half-life o…

If it takes you a year from persisting serialized data on the hard drive of your one server to using a real data store, you're fucked either way. As low as that half-life might be, that's no reason to make deliberately short-sighted engineering decisions to make it even worse, especially when all the quick and easy ways of shipping an MVP effectively preclude that strategy. You're gonna go through all the effort of shipping your MVP to a real server but you're not going to go through the effort of setting up a database? Are you kidding me? Setting up Heroku with shared Postgres is not only much quicker to ship to, but it gives you a software and data architecture that you can much more easily improve in the future.

Re: The genius and folly of MongoDB

#268

Post by nostrademons »

philwelch wrote: post #267
nostrademons wrote: post #265

Earlier quoted context omitted.

You will have to rewrite anyway. Multiple times. If you pick a RDBMS you will have to rewrite it to scale, if you pick MongoDB you will have to rewrite it for reliability, if you pick Heroku or AppEngine you will have to rewrite it to avoid paying them a good chunk of your profits. That's probably the biggest surprise I learned from working in a fast-growing, well-functioning engineering organization. The half-life o…

If it takes you a year from persisting serialized data on the hard drive of your one server to using a real data store, you're fucked either way. As low as that half-life might be, that's no reason to make deliberately short-sighted engineering decisions to make it even worse, especially when all the quick and easy ways of shipping an MVP effectively preclude that strategy. You're gonna go through all the effort of s…

You understand that Hacker News uses precisely this persistence strategy (in-memory data structures with persistent state written to the filesystem on the hard disk of the server), and has been going on 6 years now?

You also understand that most of the advice easily accessible on the Internet comes from people trying to sell you something, and so they have a vested interest in you adding many layers into your software stack that you don't need?

If you work in an actual engineering organization that has a clue what they're doing, mmap() is your best friend, and the more layers you can cut out of the stack, the better off you are.

Re: The genius and folly of MongoDB

#269

Post by twic »

asperous wrote: post #66
twic wrote: post #58

Earlier quoted context omitted.

No check constraints. Spotty transaction isolation. Silent data corruption if you happen to make certain kinds of updates while using statement-based replication. No on-line schema updates (is that still true?). Complete inability to execute joins of any size in reasonable time due to the lack of merge or hash join strategies. Corresponding inability to handle subqueries of any complexity. Readers block writers (at t…

Besides the corruption, that just sounds like it's missing features. Missing features is ridicule worthy?

When the "feature" is a basic property of a relational database - as check constraints and efficient joins and subqueries are - then yes.

Re: The genius and folly of MongoDB

#270

Post by twic »

nostrademons wrote: post #211
rsynnott wrote: post #128

Earlier quoted context omitted.

> Mongo's really great for rapid prototyping. Has any phrase ever struck more fear into the heart of a programmer? Rapid prototypes have a nasty tendency of accidentally becoming products... > You can always switch databases later shudder

That's why you use a language or technology that's politically unfeasible for your rapid prototypes, like Clojure or Haskell, or...for that matter...MongoDB. ;-)

Someone tried that at my company. We now have a Clojure app in production. Brilliant.
Post reply on HN