Why SQL is beating NoSQL, and what this means for the future of data • Hacker News Board

Re: Why SQL is beating NoSQL, and what this means for the future of data

#91

Post by skissane » Tue, Sep 26, 2017, 4:29 AM UTC

Clubber wrote: post #71

skissane wrote: post #69

Earlier quoted context omitted.

SQL was designed by IBM in the 1970s, to run on mainframes. The community by whom and for whom it was developed was focused on PL/I and COBOL and largely didn't use C. The first implementation of SQL in C was probably Oracle V3 (released in 1983). IBM's equivalent SQL offerings around the same time (SQL/DS for VM/CMS and DB2 for MVS) were written in PL/I dialects and assembly, and COBOL, Fortran and PL/I were support…

I was going by Oracle's "first commercial implementation," as a starting point. I found it it was originally written in Fortran, but later ported to C. I wouldn't have guessed fortran. https://asktom.oracle.com/pls/asktom/f?p=100:11:0::NO::P11_Q... Oracle was originally written in fortran and then redone in C, which it has been written in ever since. In fact, when we ported Oracle to the mainframe for the first time…

By most accounts Oracle V2 was primarily written in PDP-11 assembler. Tom Kyte's suggestion it was written in Fortran contradicts most other sources.

IBM's problem was that they were slow to commercialise the technology despite having a significant head start. IBM built multiple prototype RDBMS systems but they weren't willing to ship the result as a commercial product, and Oracle beat them to it. And even then when they did finally ship, their initial offerings were restricted to their mainframe systems (first VM/CMS and then MVS), while Oracle back then was open to porting their database to just about anything. IBM didn't really come to the cross-platform party until the 1990s, by which time Oracle was well established in that market.

(Disclaimer: Former Oracle employee, although I never worked on the database side of the business, and I'm talking about stuff that happened when I was a kid, or even before I was born.)

Re: Why SQL is beating NoSQL, and what this means for the future of data

#92

Post by 6nf » Tue, Sep 26, 2017, 4:30 AM UTC

jjirsa wrote: post #49

peterwwillis wrote: post #7

SQL wins because big iron wins. At the end of the day, the big, clunky, slow, complicated, aggravating, old systems stick around because they are built to stick around. Looking at, say, an eventually consistent distributed decentralized kv store, one might be tricked into believing it's simple enough to deploy with enough nodes and general enough interfaces that you can build a complicated system on top of it if need…

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead…

> damn near impossible for most MySQL users.

MySQL? Nobody, and I mean NOBOBY is saying you should use MySQL. All of the things you mention is trivial using a decent SQL solution like PostgreSQL.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#93

Post by jjirsa » Tue, Sep 26, 2017, 4:31 AM UTC

6nf wrote: post #92

jjirsa wrote: post #49

Earlier quoted context omitted.

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead…

> damn near impossible for most MySQL users. MySQL? Nobody, and I mean NOBOBY is saying you should use MySQL. All of the things you mention is trivial using a decent SQL solution like PostgreSQL.

Except the person to whom I replied said to use traditional SQL databases:

> Traditional SQL databases end up being many times simpler in practice, and because of their inherent scaling limitations, much easier to manage. And most importantly: their implementations aren't buggy.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#94

Post by ianamartin » Tue, Sep 26, 2017, 4:33 AM UTC

randomdrake wrote: post #40

I don't buy it. The makers of the software that this appears to be a carefully written advertisement for, came to the same conclusion as the rest of the IT world (that some of us saw a mile away): NoSQL was, and still is, only good for very specific things in very specific cases; it's generally dreadful for anything that SQL engines could already do well. The subtitle for this contains: "After years of being left for…

Thanks for writing this. I was about to write almost that exact thing. It's a weird interpretation of database history with a ton of stuff left out. My best guess is that the author of this article had all the data that would be needed to back this up, but it was stored in Mongo and he couldn't find it when he started writing.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#95

Post by jjirsa » Tue, Sep 26, 2017, 4:34 AM UTC

6nf wrote: post #90

jjirsa wrote: post #65

Earlier quoted context omitted.

The reason people don’t start with a relational DB has nothing to do with the data model or query language - people choose databases like Cassandra because scaling to 500 Postgres or MySQL instances holding a combined petabyte of data is horrific, but it’s dirt simple in things like Cassandra The query language is a side effect of the underlying storage engine - you don’t choose it because you want a key value store,…

You make it sound like many people are using petabyte scale databases. This is not even close to true. ESPECIALLY when you're just starting out. Most new projects will start with almost no data at all and the vast majority of projects will never ever get to the petabyte scales.

Many people ARE using petabyte scale databases.

They don't start out that way, but they will be if they're successful, and if they're not they probably shouldn't be running their own databases anyway (go RDS it up or something).

Re: Why SQL is beating NoSQL, and what this means for the future of data

#96

Post by sk5t » Tue, Sep 26, 2017, 4:34 AM UTC

aaron-lebo wrote: post #11

taeric wrote: post #8

I do find the amusing "backronym" rather hard to take at face value. I also found the old critics of SQL databases odd focus on the query language hard to take. The problem wasn't the queries, per se. The problem was the power of the queries for interactive uses and how that doesn't scale up. That is, the problem with most SQL databases is that you need some pretty specialized knowledge in order to construct good que…

The thing is ORMs have made writing easy queries easy and hard queries possible since at the very least 2005 with Rails, and they are easily extensible and composable. It's mostly a solved issue. If you want to do complex data manipulation, surprise, it takes a powerful tool. Seems a lot of the hype is the result of VC $ + naiveity/overselling about what MongoDB and co would be capable of. The marketing is natural -…

One of the last things I'd want to do is fight with an ORM to build a complex query that doesn't fit the ORM's natural tendencies. A micro-ORM to take care of the boilerplate is great, but otherwise, given the choice between two unpleasant options, I'd rather write 100% of my database interaction code in sprocs than an ORM (promising never to use the raw-SQL escape hatch).

Re: Why SQL is beating NoSQL, and what this means for the future of data

#97

Post by elvinyung » Tue, Sep 26, 2017, 4:34 AM UTC

rhencke wrote: post #88

elvinyung wrote: post #61

Earlier quoted context omitted.

You got downvoted, but I agree with you. Traditional RDBMS is inherently single-node and a pain to scale. The actual sad thing here is that there's no mature/reliable commodity (i.e. open source) distributed RDBMS for transaction processing yet.

CockroachDB? I'll grant you it's still young, but it pretty solidly hits all your other checkboxes.

IMO, it's very promising but still extremely immature.

I still believe in Curt Monash's cardinal rule of RDBMS development: http://www.dbms2.com/2013/03/18/dbms-development-marklogic-h...

Re: Why SQL is beating NoSQL, and what this means for the future of data

#98

Post by sk5t » Tue, Sep 26, 2017, 4:37 AM UTC

emodendroket wrote: post #60

aneutron wrote: post #56

By no means am I an expert, but here's the thing. You can't say "Cars are the new thing. Airplanes came to be but now they're not the shiny thing anymore." You don't take your car to go intercontinent, and you don't take the plane to go a couple of kilometers. One thing NoSQL databases excel at, is scaling. Scaling in Cassandra for example means adding a node, everything else is just magic. THAT is what made MySQL an…

I don't honestly think many people have the kind of scale where this is a sensible trade-off to make.

Absolutely. Even a bigint / long sequence column so often feels a bit ridiculous.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#99

Post by x0x0 » Tue, Sep 26, 2017, 4:42 AM UTC

craigkerstiens wrote: post #53

I don't think the article could have said it much better. SQL is super powerful and makes much sense in so many ways. Nearly all apps have a relational structure to them and SQL is a reasonable way to interact with them. Some of my favorite conversations from the Postgres community 5-6 years back were when they were talking about a time when Postgres was being disrupted. The gray bearded DBAs (Hi Berkus and others) w…

Some of the resurgence of sql has to be the size of data you could fit on a single machine / fake with a distributed system increased.

But postgres I think continues to be at the forefront: first, the article misstated slightly; jsonb gives you not only full text search (who needs that?) but indexing all the way down. So you can eg say I want records where "field1 -> field2 has an element named md5 with a given value...". It turns out that a lot of plain normalized tabular data plus a medium about of schema-free jsonb really is the perfect combination for most purposes.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#100

Post by oldandtired » Tue, Sep 26, 2017, 4:44 AM UTC

aaron-lebo wrote: post #11

taeric wrote: post #8

I do find the amusing "backronym" rather hard to take at face value. I also found the old critics of SQL databases odd focus on the query language hard to take. The problem wasn't the queries, per se. The problem was the power of the queries for interactive uses and how that doesn't scale up. That is, the problem with most SQL databases is that you need some pretty specialized knowledge in order to construct good que…

The thing is ORMs have made writing easy queries easy and hard queries possible since at the very least 2005 with Rails, and they are easily extensible and composable. It's mostly a solved issue. If you want to do complex data manipulation, surprise, it takes a powerful tool. Seems a lot of the hype is the result of VC $ + naiveity/overselling about what MongoDB and co would be capable of. The marketing is natural -…

The problem is that SQL (in all its forms) has not been relational. Nor is it complete in terms of its querying ability. This means that it can do unexpected things and more so when you have nulls included in your data.

The Third Manifesto (by Date and Darwen) highlight many of the problems of SQL and SQL DBMS. Though I find TTM to be lacking in various ways, it does, at least, start heading in the right direction in terms of a database query system.

Over the course of decades, I have written much SQL and have had to modify and document many systems that were based on SQL DBMS's. The language is a PITA, but you do what you have to do to get things working.

NoSQL was, to me, a retrograde step, though it has some usefulness in some situations. What I have found is that most databases have not been designed with regard to Relational Database Theory (RDT). In general, the attitude that I have observed has been that RDT is good in theory but bad in practice.

I have built (using mickey mouse systems), fairly complex, multiuser databases based on RDT. I have also seen many databases built (using ORACLE and SQL Server as the DBMS) that have not used RDT and they have been atrocious. One such system, built by some SQL guru, took 25 hours to do a run of one process. Had it been built according to RDT, the entire process (by my estimate) would have taken less than 20 minutes to run.

When designing any kind of database using any of the various technologies that have been developed over the 60 odd years, there are various design principles that will allow a good database (fit for purpose) to be built. It appears in many cases that none of these principles are known or if know are know then not adhered to.

Of course, what others might have found can be quite different.