Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Year with MongoDB

blog.engineering.kiip.me

Post reply on HN
Print view View original

131–140 of 153 posts

Re: A Year with MongoDB

#131

Post by jeffdavis »

scubaguy wrote: post #57
judofyr wrote: post #40

Earlier quoted context omitted.

So it's actually 100% the same as you would do with on-the-fly migrations in SQL: (1) Add column and add code moves the old data when you access it. Deploy. (2) Let it run for a while. Run a background job that migrates the rest (this might be done days or months later). (3) Remove the column and the custom code. The more I hear about "schemaless" the more I realize that it doesn't make any difference at all.

For a large table, adding a new column can be non-trivial, each record is updated when the new column is added.

If you choose to update all of the old records.

As other posts have already mentioned, adding a new column to a table in postgres does so instantly unless you set a default (telling postgres that you want it to update all the old records too).

If you add the default separately from adding the column, you get the default but don't rewrite all the old rows (which remain NULL).

Re: A Year with MongoDB

#132

Post by leif »

charlieok wrote: post #117
rdtsc wrote: post #10

> Safe off by default I think that is fixed now. But this the single most appalling design decision they could have made while also claiming their product was a "database". (And this has been discussed here before so just do a search if you will). This wasn't a bug, it was a deliberate design decision. Ok, that that would have been alright if they put a bright red warning on their front page. "We disabled durability…

I think they just assumed that people would run the database in clusters, not single instances. If you've done enough research to choose a relatively off-the-beaten-path DBMS such as MongoDB, the assumption is that you've carefully weighed the tradeoffs made by the various alternatives in the space, and learned the best practices for deploying the one you chose to use.

That's a reckless assumption.

Re: A Year with MongoDB

#133

Post by charlieok »

leif wrote: post #132
charlieok wrote: post #117

Earlier quoted context omitted.

I think they just assumed that people would run the database in clusters, not single instances. If you've done enough research to choose a relatively off-the-beaten-path DBMS such as MongoDB, the assumption is that you've carefully weighed the tradeoffs made by the various alternatives in the space, and learned the best practices for deploying the one you chose to use.

That's a reckless assumption.

Maybe, but as I understand it, the value proposition of MongoDB is closely tied to clustered deployments.

Re: A Year with MongoDB

#134

Post by aschobel »

strlen wrote: post #64
aschobel wrote: post #38

We love MongoDB at Catch, it's been our primary backing store for all user data for over 20 months now. > Catch.com > Data Size: 50GB > Total Documents 27,000,000 > Operations per second: 450 (Create, reads, updates, etc.) > Lock % average 0% > CPU load average 0% Global Lock isn't ideal, but Mongo is so fast it hasn't been an issue for us. You need to keep on slow queries and design your schema and indexes correctly…

> Data Size: 50GB I am sorry, to sound blunt, but that's an irrelevant data point. With a data set that fits comfortably into RAM (much less SSDs in RAID!), most any data store will work (including MySQL or Postgres). > Operations per second: 450 Again, not a relevant data point. With a 10 ms seek time on a SATA disk, this is (again) well within the IOPS capacity of a single commodity machine (with RAID, a SAS drive,…

Agreed, this is a trivial amount of data.

Context is the article which made is seem like even 220GB is an ok amount, still can fit in memory.

Re: A Year with MongoDB

#135

Post by aschobel »

siavosh wrote: post #41
bretthoerner wrote: post #39

Earlier quoted context omitted.

> I don't get the safety issue, 20 months and we haven't lost any user data. shrug Nobody loses any user data until they do.

This should be a deal breaker for any serious app. Does the performance hit of safe mode negate all other advantages of MongoDB?

Do you have a citation to back up the claim that you shouldn't use MongoDB for serious apps?

We have done billions of ops with Mongo and have never lost any data.

Re: A Year with MongoDB

#136

Post by aschobel »

siavosh wrote: post #41
bretthoerner wrote: post #39

Earlier quoted context omitted.

> I don't get the safety issue, 20 months and we haven't lost any user data. shrug Nobody loses any user data until they do.

This should be a deal breaker for any serious app. Does the performance hit of safe mode negate all other advantages of MongoDB?

Do you have a citation to back up the claim that you shouldn't use MongoDB for serious apps?

We have done billions of ops with Mongo and have never lost any data.

Re: A Year with MongoDB

#137

Post by leif »

charlieok wrote: post #133
leif wrote: post #132

Earlier quoted context omitted.

That's a reckless assumption.

Maybe, but as I understand it, the value proposition of MongoDB is closely tied to clustered deployments.

Sorry, I meant this assumption:

you've carefully weighed the tradeoffs made by the various alternatives in the space, and learned the best practices for deploying the one you chose to use

It is unreasonable to expect everyone you want to use your software to become an expert in its subtleties before deployment. Of course it's what you'd hope, and in a perfect world it would hold, but in the real world, no matter your market, a lot of your users are going to be lazy and irresponsible. You need to protect your users better than this. To do otherwise is lazy and irresponsible.

Re: A Year with MongoDB

#138

Post by quadhome »

fusiongyro wrote: post #106
quadhome wrote: post #93

Earlier quoted context omitted.

It's a shame this comment is pithy, because I think it's dead-on. There times to integrate at the database level. But, the default should be single-application databases. The rationale is the same as the grandparent's rationale FOR database integration. The odds of needing to share data over time are 1. Given that shared belief, the problem with database integration is that MANY applications need to share facets of t…

> The single database ends up having a huge surface area trying to satisfy every application's needs. This is, more or less, exactly what views are for. > Thus, "build an api" is the best solution. And you can do that within the database with stored procedures, perhaps even with the same language you would use in the front-end (depending). And look at the advantages you have: - No implied N+1 issues because your API…

> - All the information is in the right place to ensure data validity and performance

This is where we part ways.

We're talking specifically about integration. That means each system have different processes and are talking with other people.

If this is a case of three apps exposing the same process over three different channels (HTTP, UDP, morse code); then, database-level integration makes perfect sense.

But, as soon as differing behaviors comes in, then the database level doesn't— by definition— have enough information to ensure validity. One app thinks columns X and Y are dependent in one way, the other app views it another way. Now, one or the both of those apps are screwed for validity. And this problem grows with N+1.

I am certainly not arguing against good databases. Stored procedures, views, etc. are all great even for a single application. But, I am arguing database level integration should be the rare exception to the rule.

Re: A Year with MongoDB

#139

Post by quadhome »

jeffdavis wrote: post #129
quadhome wrote: post #93

Earlier quoted context omitted.

It's a shame this comment is pithy, because I think it's dead-on. There times to integrate at the database level. But, the default should be single-application databases. The rationale is the same as the grandparent's rationale FOR database integration. The odds of needing to share data over time are 1. Given that shared belief, the problem with database integration is that MANY applications need to share facets of t…

> Thus, "build an api" is the best solution. I think there's an asymmetry in your argument. You are describing all of the problems with data management as though they were specific to schemas in a traditional RDBMS; but glossing over how "building an API" solves those same problems, and whether that method is better or worse. In other words, "build an API" is the problem at hand, not the solution. A traditional DBMS…

Yes, I agree, "build an API" is the problem.

I specifically said the problem and the complexity inherent in solving it doesn't disappear with per-application databases.

But, the application (the 'A' part) is where the most context around the information is to be found. Tying multiple applications together at the Data-side (rather than the Application-side) means you don't lose that context... and eliminate your chances of clashing (and the ensuring integrity issues).

Re: A Year with MongoDB

#140

Post by matthewcford »

quadhome wrote: post #93
matthewcford wrote: post #91

Earlier quoted context omitted.

build an api

It's a shame this comment is pithy, because I think it's dead-on. There times to integrate at the database level. But, the default should be single-application databases. The rationale is the same as the grandparent's rationale FOR database integration. The odds of needing to share data over time are 1. Given that shared belief, the problem with database integration is that MANY applications need to share facets of t…

Thanks for expanding on my answer, nailed it.
Post reply on HN