Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NoSQL Data Modeling Techniques

highlyscalable.wordpress.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: NoSQL Data Modeling Techniques

#11

Post by batista »

bni wrote: post #6

First off I find the NoSQL term in itself very strange. How can you say anything intelligent about "everything that is not using SQL as a query language"? Its like talking about NoJava, instead of talking about Ruby. Props for a well written article with lots of nice graphs but I dont agree with much of its content. A few examples: "software applications are not so often interested in in-database aggregation" In my e…

First off I find the NoSQL term in itself very strange. How can you say anything intelligent about "everything that is not using SQL as a query language"? Its like talking about NoJava, instead of talking about Ruby.

Only it's not like that, because SQL is a well defined way of thinking at problems, not just a particular implementation of it.

So, NoSQL is like saying NonFunctional. Or NonImperative. Or NoRegister VMs, etc...

Re: NoSQL Data Modeling Techniques

#12

Post by batista »

Maro wrote: post #10
bni wrote: post #6

First off I find the NoSQL term in itself very strange. How can you say anything intelligent about "everything that is not using SQL as a query language"? Its like talking about NoJava, instead of talking about Ruby. Props for a well written article with lots of nice graphs but I dont agree with much of its content. A few examples: "software applications are not so often interested in in-database aggregation" In my e…

"software applications are not so often interested in in-database aggregation" Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL, the same way Word.exe doesn't use SQL internally, instead it uses linked lists and hashmaps [1]. "In my experience…

Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL

I don't see anything in Gmail and Google Docs data usage needs that make NoSQL a better fit for them --only the need to scale with no sharing etc.

(OTOH Dropbox might fit the case, they only need some hierarchical store a la filesystem, with versioning).

the same way Word.exe doesn't use SQL internally, instead it uses linked lists and hashmaps

Yes, but Word.exe internally deals with just ONE document and it's data structures, does not manage metadata for many documents and does not have to provide aggregated info on them. That would be Windows Explorer, that manages many .docs. And that one, MS did have a plan to make more SQL-like.

Re: NoSQL Data Modeling Techniques

#13

Post by einhverfr »

wyuenho wrote: post #9
jandrewrogers wrote: post #7

Earlier quoted context omitted.

NoSQL is about the interface, not the implementation. NoSQL databases provide a better impedance match out of the box for some applications. What is often lost in the conversation is that you can do the same thing using a competent SQL database engine if you can deal with the complexity. But you have to use SQL, which for some applications is a poor interface, and you have to configure the engine for your application…

Most startups also don't have complicated data models or a high enough traffic to justify having a DBA and commit premature optimization by denormalizing data right off the bat to speed up retrieval in your NoSQL DB. As many have said before, NoSQL is a premature optimization in that all it does is to remove some restrictions in your technology stack to let you move come complexity such as data validation and the abi…

The impedance mismatch is real which anyone who has seen stored procs written by app developers can verify....

What NoSQL does is give you a light-weight object store which could have some very cool uses. Those uses are also pretty narrow for the reasons the article mentions (assuming the questions you have now are all the questions that are important for example).

However, I think some of the solutions may help in relational environments with the edge cases.

Re: NoSQL Data Modeling Techniques

#14

Post by Maro »

batista wrote: post #12
Maro wrote: post #10

Earlier quoted context omitted.

"software applications are not so often interested in in-database aggregation" Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL, the same way Word.exe doesn't use SQL internally, instead it uses linked lists and hashmaps [1]. "In my experience…

Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL I don't see anything in Gmail and Google Docs data usage needs that make NoSQL a better fit for them --only the need to scale with no sharing etc. (OTOH Dropbox might fit the case, they only need…

NoSQL is a better fit for Gmail because you wouldn't want one gigantic 'emails' or 'contacts' table, which is how you would model it in SQL. You also want to cache the hell out of it, which is easier done in a NoSQL model (e.g. if the cache layer is integrated and hence uses the same API as the persistent stuff).

Re: NoSQL Data Modeling Techniques

#15

Post by batista »

Maro wrote: post #14
batista wrote: post #12

Earlier quoted context omitted.

Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL I don't see anything in Gmail and Google Docs data usage needs that make NoSQL a better fit for them --only the need to scale with no sharing etc. (OTOH Dropbox might fit the case, they only need…

NoSQL is a better fit for Gmail because you wouldn't want one gigantic 'emails' or 'contacts' table, which is how you would model it in SQL. You also want to cache the hell out of it, which is easier done in a NoSQL model (e.g. if the cache layer is integrated and hence uses the same API as the persistent stuff).

NoSQL is a better fit for Gmail because you wouldn't want one gigantic 'emails' or 'contacts' table, which is how you would model it in SQL.

Per gmail account, you'd still dump all the mails into one gigantic emails table, either an SQL or a NoSQL one.

As for the totality of the gmail accounts, well, you'd use sharding, which is the same whether you use NoSQL or SQL.

You also want to cache the hell out of it, which is easier done in a NoSQL model (e.g. if the cache layer is integrated and hence uses the same API as the persistent stuff).

But a SQL DB can also use the same transparent API for the persistent stuff with their standard Query API. It wouldn't even need to parse the SQL statement, just hash it and compare it. And you could also use a layer that makes it transparent, such as an ORM.

Re: NoSQL Data Modeling Techniques

#16

Post by mattmanser »

Maro wrote: post #14
batista wrote: post #12

Earlier quoted context omitted.

Applications that used to be desktops apps are now moving to the web, and these "cloud applications" are prime use-cases for NoSQL. Apps like Gmail, Google Docs, Dropbox are cases where a NoSQL might be a better fit than SQL I don't see anything in Gmail and Google Docs data usage needs that make NoSQL a better fit for them --only the need to scale with no sharing etc. (OTOH Dropbox might fit the case, they only need…

NoSQL is a better fit for Gmail because you wouldn't want one gigantic 'emails' or 'contacts' table, which is how you would model it in SQL. You also want to cache the hell out of it, which is easier done in a NoSQL model (e.g. if the cache layer is integrated and hence uses the same API as the persistent stuff).

Huh, why wouldn't you want those tables? You seem to be confusing the need to scale with how you would design the system.

Somewhere I worked before we used to import 100,000s of a companies email into the project system to display them against the system. They were all in an Emails table which just stored the body, I think there was an email header table and an email contacts table which was relationally linked directly to the Person table.

Worked fantastically, used to run an IMAP server off those tables for outlook integration.

Re: NoSQL Data Modeling Techniques

#17

Post by wyuenho »

einhverfr wrote: post #13
wyuenho wrote: post #9

Earlier quoted context omitted.

Most startups also don't have complicated data models or a high enough traffic to justify having a DBA and commit premature optimization by denormalizing data right off the bat to speed up retrieval in your NoSQL DB. As many have said before, NoSQL is a premature optimization in that all it does is to remove some restrictions in your technology stack to let you move come complexity such as data validation and the abi…

The impedance mismatch is real which anyone who has seen stored procs written by app developers can verify.... What NoSQL does is give you a light-weight object store which could have some very cool uses. Those uses are also pretty narrow for the reasons the article mentions (assuming the questions you have now are all the questions that are important for example). However, I think some of the solutions may help in r…

That's a little weak as an argument isn't it. SQL and the relational model has nothing to do with stored procedures and you certainly don't have to, or need to use stored procedures for reporting. In fact, prior to 2005, MySQL didn't support stored precedures for the longest time. How do you think those people did reporting? I don't know how you do reporting, but mind involves just a ridiculously simple and ugly web app that essentially just generates some HTML tables from a bunch of SQLAlchemy queries.

Why do I get the sense that the real reason most people use NoSQL stems from their destain of SQL, whatever that reason maybe...

Re: NoSQL Data Modeling Techniques

#18

Post by einhverfr »

wyuenho wrote: post #17
einhverfr wrote: post #13

Earlier quoted context omitted.

The impedance mismatch is real which anyone who has seen stored procs written by app developers can verify.... What NoSQL does is give you a light-weight object store which could have some very cool uses. Those uses are also pretty narrow for the reasons the article mentions (assuming the questions you have now are all the questions that are important for example). However, I think some of the solutions may help in r…

That's a little weak as an argument isn't it. SQL and the relational model has nothing to do with stored procedures and you certainly don't have to, or need to use stored procedures for reporting. In fact, prior to 2005, MySQL didn't support stored precedures for the longest time. How do you think those people did reporting? I don't know how you do reporting, but mind involves just a ridiculously simple and ugly web…

I am a total relational guy, BTW. But the fact is that there is a mismatch between how you have to think when doing SQL queries (thinking in sets) and OO programming (thinking in instructions). Anyone who has dealt with stored procedures written by app folks understands what a mess you get when you try to program one side in techniques aimed at the other.

While NoSQL is a good choice for some environments, namely ones where ad hoc reporting is not likely to be needed and where other methods of interop are preferred (LDAP being a great example of something that could benefit from a NoSQL back-end), the fact is that this actually shows that, more often than not, you lose more than you gain by getting rid of the mismatch....

IOW, I think it is a moderately weak case for NoSQL in some environments and a strong case against in a much larger number of environments.....

Re: NoSQL Data Modeling Techniques

#19

Post by bartonfink »

batista wrote: post #11
bni wrote: post #6

First off I find the NoSQL term in itself very strange. How can you say anything intelligent about "everything that is not using SQL as a query language"? Its like talking about NoJava, instead of talking about Ruby. Props for a well written article with lots of nice graphs but I dont agree with much of its content. A few examples: "software applications are not so often interested in in-database aggregation" In my e…

First off I find the NoSQL term in itself very strange. How can you say anything intelligent about "everything that is not using SQL as a query language"? Its like talking about NoJava, instead of talking about Ruby. Only it's not like that, because SQL is a well defined way of thinking at problems, not just a particular implementation of it. So, NoSQL is like saying NonFunctional. Or NonImperative. Or NoRegister VMs…

SQL is a language that is not particularly well defined, nor essential to solving problems. What you're referring to as a "well defined way of thinking" is the relational model. There have been relational databases that did not use SQL as their query language, and although they're not widespread, the same points the NoSQL community makes against e.g. Postgres could be made against them.

Re: NoSQL Data Modeling Techniques

#20

Post by LewisCr »

linuxhansl wrote: post #5
malingo wrote: post #3

I like the comparison of the design themes of relational modeling and NoSQL modeling as, respectively, "what answers do I have?" and "what questions do I have?"

To me the key point is that in relational databases you describe - declaratively - the information you want and leave it to the database to figure out to retrieve it (x); whereas in most NoSQL databases you describe how to retrieve the information. (x) this is not the entire truth, as SQL is actually a (somewhat) unfortunate mix between the declarative relational calculus and the procedural relational algebra.

I don't know enough to understand what makes relational algebra procedural. I read the "Haskell more successful cousin" article and I thought that the relational algebra part was what made SQL and Haskell similar.
Post reply on HN