Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NoSQL: The Baby and the Bathwater

brooker.co.za

Post reply on HN
Print view View original

41–50 of 59 posts

Re: NoSQL: The Baby and the Bathwater

#41

Post by cle »

revscat wrote: post #35
collaborative wrote: post #22

Earlier quoted context omitted.

But you are describing incorrect ways of using NoSQL. NoSQL requires getting the schema right from the get go. Many people don't like it because they are used to relying on SQL (or a language like you mentioned) to smooth things out. NoSQL requires you to ask yourself hard questions - what specific queries are going to consume this data and then model the data as opposed to in SQL where you just store the data and wo…

Which seems to imply a static schema, known more-or-less perfectly at the outset, and unchanging thereafter. That is obviously unreasonable. Application data structures change as new requirements arise, and as existing requirements become better understood.

> Which seems to imply a static schema, known more-or-less perfectly at the outset, and unchanging thereafter. That is obviously unreasonable.

Is it? Most of AWS runs on NoSQL databases, and continues to ship new features that do not fit into existing schema. This assertion then is clearly incorrect.

Re: NoSQL: The Baby and the Bathwater

#42

Post by throwawaaarrgh »

Most databases are still kinda crap. Either it's hard to scale them or maintain them, hard to know the right way to use them, or they strip out useful functionality for the sake of simplicity, forcing reinventing the wheel. We haven't got much actually novel design since like 2006. There's some people trying to bolt on top of/around/under existing databases for backwards compatibility or to avoid reinventing, but not really much actual novel design for a whole database.

Computer Science research seems to either be fully academic, where research into practical solutions doesn't happen, or it's limited to private companies trying to solve one business problem. Very little actual advancement of the state of the art, or solving of long standing limitations.

Re: NoSQL: The Baby and the Bathwater

#43

Post by collaborative »

trashtester wrote: post #32
collaborative wrote: post #22

Earlier quoted context omitted.

But you are describing incorrect ways of using NoSQL. NoSQL requires getting the schema right from the get go. Many people don't like it because they are used to relying on SQL (or a language like you mentioned) to smooth things out. NoSQL requires you to ask yourself hard questions - what specific queries are going to consume this data and then model the data as opposed to in SQL where you just store the data and wo…

Some claim the opposite, though. Even in this discussion. That NoSQL is needed when you need to change the schema in a production system. Anyway, most systems that last more than a couple of years tend to change over time, meaning the schema is likely to need extensions.

Sure, changing the schema is easier in NoSQL. But to do it properly, you still need to really understand how the data is being queried and then change the schema in a controlled "migration". For most people these tasks will be a pain. But NoSQL can really shine in terms of $/query if data is being correctly looked after. It will save you from needing 16 cores in multiple clusters and very expensive bills

Re: NoSQL: The Baby and the Bathwater

#44

Post by collaborative »

jrumbut wrote: post #34
collaborative wrote: post #22

Earlier quoted context omitted.

But you are describing incorrect ways of using NoSQL. NoSQL requires getting the schema right from the get go. Many people don't like it because they are used to relying on SQL (or a language like you mentioned) to smooth things out. NoSQL requires you to ask yourself hard questions - what specific queries are going to consume this data and then model the data as opposed to in SQL where you just store the data and wo…

This is true, it's also very difficult to do in most business situations (getting the schema right from the get go).

It's hard but it's still possible to change it. It's just very important to do it carefully and knowing what the consequences are. I have had to iterate NoSQL schemas several times, and I always prefer do treat their evolution as migrations done out-of-hours

Re: NoSQL: The Baby and the Bathwater

#45

Post by BeFlatXIII »

masklinn wrote: post #24
roenxi wrote: post #5

I don't like NoSQL databases. "NoSQL" should be manna from heaven. It may be impossible to create a less pleasant language than SQL. It isn't composable, it isn't internally consistent, it isn't easy to parse, it claims to be declarative but the ordering of the clauses is completely rigid, it fights every attempt at writing testable or maintainable code. It is hard to read. It is hard to programatically generate. Eve…

> It may be impossible to create a less pleasant language than SQL. It’s not by the way, there’s an entire segment of the esolang space for which it’s the goal (starting with the canonical example that is INTERCAL). But ignoring esolangs, there are lots of very unpleasant programming languages out there. M/MUMPS is a well known one (especially with “legacy” coding styles, or so I gather). I also consider XSLT to be a…

MUMPS always reminds me of Perl code golfing.

Re: NoSQL: The Baby and the Bathwater

#46

Post by trashtester »

collaborative wrote: post #43
trashtester wrote: post #32

Earlier quoted context omitted.

Some claim the opposite, though. Even in this discussion. That NoSQL is needed when you need to change the schema in a production system. Anyway, most systems that last more than a couple of years tend to change over time, meaning the schema is likely to need extensions.

Sure, changing the schema is easier in NoSQL. But to do it properly, you still need to really understand how the data is being queried and then change the schema in a controlled "migration". For most people these tasks will be a pain. But NoSQL can really shine in terms of $/query if data is being correctly looked after. It will save you from needing 16 cores in multiple clusters and very expensive bills

That's interesting.

A lot of your suggestions are actually exactly the same for SQL based databases. If your schema is not fit for the task at hand, it can slow it down by an order of magnitude, and the process of changing schema is also similar.

Though, i would think, a properly designed SQL database would need such full schema refactoring less often, since adding a few tables within the same structure is easier.

It sound like you're describing use cases with greater data volumes than I usually use SQL for. (Mostly Data Science use cases, where larger datasets typically end up in Spark, or similar)

My experience is that SQL based systems work best up to a few 100 millions of records in the largest tables (a few billion at most), and with transactions per second is less than about 10000. Around those volumes is where SQL start to get really expensive.

And often SQL is used for use cases where number of records per table of less than 10 million and transactions per second in the low hundreds or lower.

But I'm probably biased in the opposite direction that you are. For me, performance usually means efficient joins. Which means that even if I'm leaving RDBMS's behind, I still use SQL where I can (such as in Spark).

Re: NoSQL: The Baby and the Bathwater

#47

Post by fifilura »

strbean wrote: post #20
Sankozi wrote: post #17

Earlier quoted context omitted.

I agree completely. When I first heard about NoSQL I thought some databases were finally introducing some replacement for abominable SQL. But no, they were just creating schemaless databases.

I'm so glad this sentiment is being spread more. So tired of explaining that when I complain about SQL I *emphatically do not mean relational databases in general*, only to have everyone counter with "but relational stuff is good!"

Your argument seems a bit childish to me. Like you never really cared to learn the tool.

SQL is relational algebra. Sure you can write it some other way. Lots of projects tried, none succeeded. Wonder why is that?

End of the day - if you just learn the syntax, the hard part will be in the logic. As it should be.

Re: NoSQL: The Baby and the Bathwater

#48

Post by emodendroket »

roenxi wrote: post #15
emodendroket wrote: post #6

Earlier quoted context omitted.

I think the right way to think about NoSQL databases is you have an application where traffic is expected to be heavy enough that it's worth throwing out what SQL gives you for free and having a highly optimized solution where you deal with those problems yourself. Of course, it was a big enough trend that many people jumped on it without having a practical use for it. Arguably psql does offer a NoSQL-like experience…

> Arguably psql does offer a NoSQL-like experience with JSON columns. I hope not. JSON columns purge all the good things a relational database offers and keeps the SQL . It is the worst of every option. That really should be the opposite of the NoSQL experience. Disbarred by the fact that SQL is involved.

I guess it depends what part of NoSQL you’re looking for, but if you want a bit of schemaless data it makes sense.

Re: NoSQL: The Baby and the Bathwater

#49

Post by collaborative »

trashtester wrote: post #46
collaborative wrote: post #43

Earlier quoted context omitted.

Sure, changing the schema is easier in NoSQL. But to do it properly, you still need to really understand how the data is being queried and then change the schema in a controlled "migration". For most people these tasks will be a pain. But NoSQL can really shine in terms of $/query if data is being correctly looked after. It will save you from needing 16 cores in multiple clusters and very expensive bills

That's interesting. A lot of your suggestions are actually exactly the same for SQL based databases. If your schema is not fit for the task at hand, it can slow it down by an order of magnitude, and the process of changing schema is also similar. Though, i would think, a properly designed SQL database would need such full schema refactoring less often, since adding a few tables within the same structure is easier. It…

Efficient joins in NoSQL are done by persisting the relation in a brand new entity. Thus they only make sense when the query is really and constantly needed, not when one is writing up SQL, researching, creating a custom report, etc.

NoSQL DBs take large volumes of data, little CPU, and almost no RAM

SQL DBs take lower volumes of data, loads of CPU and RAM

Storage is cheap, CPU is expensive, hence NoSQL is cheap. Even when your project is not in the 100 of milions records this can be significant because you then are able to offer a cheaper product than the competition's

Re: NoSQL: The Baby and the Bathwater

#50

Post by anotherhue »

roenxi wrote: post #5

I don't like NoSQL databases. "NoSQL" should be manna from heaven. It may be impossible to create a less pleasant language than SQL. It isn't composable, it isn't internally consistent, it isn't easy to parse, it claims to be declarative but the ordering of the clauses is completely rigid, it fights every attempt at writing testable or maintainable code. It is hard to read. It is hard to programatically generate. Eve…

PRQL: a simple, powerful, pipelined SQL replacement

https://news.ycombinator.com/item?id=34181319

Also: https://github.com/ajnsit/languages-that-compile-to-sql

Post reply on HN