Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can't Wait for NoSQL to Die

teddziuba.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: I Can't Wait for NoSQL to Die

#71

Post by pradocchia »

newhouseb wrote: post #66
rythie wrote: post #42

"Did you know that Cassandra requires a restart when you change the column family definition? Yeah, the MySQL developers actually had to think out how ALTER TABLE works, but according to Cassandra, that's a hard problem that has very little business value. Right." Really did the MySQL people think about it? because it takes ages to do an ALTER. Even when you are doing something like dropping an index it can lock up f…

> In contrast restarting a service is no big deal. Yikes, restarting a service that's so essential to everything else in web infrastructure is certainly a big deal. Where I work (large 30+ million users/month site), we have batches that do all sorts of processing and DB crashes (basically equivalent to a restart) can be a major pain because it can be difficult to figure out exactly what failed and when and how to bes…

> ALTER takes ages to do because of the ACID constraints of MySQL.

No, I don't think it has much to with ACID. Rather, they made a simple single implementation of ALTER TABLE that copies the whole table out on any change whatsoever. Add a column? Recreate the table. Add a table comment? Recreate the table. Drop an index? Recreate the table.

They could have identified cases where in principle the table & metadata could be modified in-place, but that would be a lot harder than a simple copy. It would probably necessitate changes to the legacy architecture, which in turn would require a host of other changes.

Re: I Can't Wait for NoSQL to Die

#72

Post by code_duck »

VBprogrammer wrote: post #59

I found the comparison with 'Real Businesses' particularly funny, given Wal-mart have have 2.1 million employees worldwide and Twitter has 75 million users...their scaling requirements are different by a factor of 35.

You're comparing two completely different businesses on disparate metrics.

How many 'users' does WalMart have worldwide? I'd say at least 500 million on whom they keep a purchase record. Then there's products, credit card numbers, suppliers, etc.

Re: I Can't Wait for NoSQL to Die

#73

Post by VBprogrammer »

code_duck wrote: post #72
VBprogrammer wrote: post #59

I found the comparison with 'Real Businesses' particularly funny, given Wal-mart have have 2.1 million employees worldwide and Twitter has 75 million users...their scaling requirements are different by a factor of 35.

You're comparing two completely different businesses on disparate metrics. How many 'users' does WalMart have worldwide? I'd say at least 500 million on whom they keep a purchase record. Then there's products, credit card numbers, suppliers, etc.

But in terms of their 'real business' it is not the number of customers they have but the number of they have but only their employees who will be using their systems. The purchase records, products and credit card numbers are closer to Tweets than users.

Re: I Can't Wait for NoSQL to Die

#74

Post by bilbo0s »

lmz wrote: post #49
bilbo0s wrote: post #27

Earlier quoted context omitted.

Maybe I should have been clear. You only go to SQL RDBMS if you need it. Which, in the vast majority of cases you will not. Further, you only go to SQL where you need it. For instance, right now someone is developing a street car racing game for Facebook. The XBox kind, not the FarmVille kind. At any rate, one of the features is obviously, playback. Now keeping all of those physics updates in an SQL is pointless. And…

Isn't Cassandra slightly more complicated than MySQL? Sure adding another column is trivial, but the access patterns & indexing need to be determined first.

Not if you use AKKA!

Re: I Can't Wait for NoSQL to Die

#76

Post by efuquen »

rufo wrote: post #6

NoSQL will never die, but it will eventually get marginalized, like how Rails was marginalized by NoSQL. Is it just me, or does this statement make absolutely no sense whatsoever?

Is it just me, or did most of that article make no sense whatsoever?

Re: I Can't Wait for NoSQL to Die

#77

Post by richcollins »

paulgb wrote: post #39
richcollins wrote: post #36

Earlier quoted context omitted.

Why is it faster?

It doesn't force you to fit non-tabular (eg. hierarchical) data into a table structure. Also, no schema means the data structure is more malleable. With the right ORM, this fits in well nicely with polymorphism: I can store objects with some common features in the same collection, but when I retrieve them from the database I get different types of objects which inherit the same base object. mongoengine is one ORM tha…

Does MongoDB manage indexes for you?

Re: I Can't Wait for NoSQL to Die

#78

Post by prodigal_erik »

bilbo0s wrote: post #30
prodigal_erik wrote: post #20

Earlier quoted context omitted.

I once worked in a Notes shop. Notes has no schema for documents and nothing to enforce migrating data from older documents to the current format. After a few years of customers manipulating data with various versions of the code, they had documents in such bizzare combinations of states that it was no longer possible for anyone on our dev team to inspect them and say which behavior would be right for the workflow. S…

That's because you waited years to address the problem. Not only that, you also rewrote the code, as I advise. But you did not take the opportunity to address the structural data issues you were having, contrary to my advice. My strategy is to rewrite the code, if needed, but with an eye towards addressing structural data issues. After a few months use of a web app you have a good idea of any surprising usage pattern…

If you are not doing data migration, after n revisions to the data management code, each record can be in any of 2^n states, depending on which code revisions did or did not modify it. How many revisions can you make before your code can no longer handle some of your older data? I'd say days' worth, not months, because you're trying to iterate a lot faster than we did. And the odds of a complete rewrite understanding all your old data are even worse.

If you are doing data migration, you necessarily have an old and new schema in your head. At that point you're just refusing to write it down and let the tools tell you whether the code agrees with you.

Re: I Can't Wait for NoSQL to Die

#79

Post by gsk »

VBprogrammer wrote: post #59

I found the comparison with 'Real Businesses' particularly funny, given Wal-mart have have 2.1 million employees worldwide and Twitter has 75 million users...their scaling requirements are different by a factor of 35.

Listen to rm-rf. Way back when (2003-2004), I was involved in Walmart's supply chain managament software. I recall a few million rows to be optimized every day (before noon, so the hundreds of trucks plying the road can move things around efficiently).

Re: I Can't Wait for NoSQL to Die

#80

Post by code_duck »

VBprogrammer wrote: post #73
code_duck wrote: post #72

Earlier quoted context omitted.

You're comparing two completely different businesses on disparate metrics. How many 'users' does WalMart have worldwide? I'd say at least 500 million on whom they keep a purchase record. Then there's products, credit card numbers, suppliers, etc.

But in terms of their 'real business' it is not the number of customers they have but the number of they have but only their employees who will be using their systems. The purchase records, products and credit card numbers are closer to Tweets than users.

I don't think the number of people doing data entry or accessing a system matters nearly as much as how much data one has to be tracked. WalMart's data needs exceed that of Twitter, they exceed the needs of Facebook. I don't know if they meet or exceed Google, but I'd imagine it's up there.
Post reply on HN