Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

Post reply on HN
Print view View original

241–250 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#241

Post by dragonwriter »

autokad wrote: post #217
Clubber wrote: post #12

Earlier quoted context omitted.

>So why not start with a relational database? I asked the same question. The biggest reason I've heard is RDBMS's don't horizontally scale well, meaning you can't easily have 50 replicated nodes across the globe and expect it to perform well, or setup easily, because it's fairly complicated with an RDBMS. There's things like Oracle's grid or SQL's high availability clusters, but they get complicated fast, particularl…

RDBMS are such a production kill. imagine you just received two TB data sets, and you'd like to join them and run a process. In hive, you create the schema (pretty simple) and join them. queries may take a few minutes but you get to doing the work you intended all along. In RDBMS, you have to fus with its schema, "you said 125 characters for that string field and you just received 130, FAIL". Its a STRING JUST DEAL W…

> In RDBMS, you have to fus with its schema, "you said 125 characters for that string field and you just received 130, FAIL". Its a STRING JUST DEAL WITH IT.

While standard SQL may have this problem, most concrete RDBMS do not, because they support a simple length-unspecified string type (or, in the case of SQLite, treat column types as documentation and mild hints rather than actual rules.)

> you have to create indexes

Absolutely not. You can, for efficiency, but (ironically, unlike some noSQL systems that require explicit indexes to support queries) you can query and join tables of relational databases without defining explicit indexes. This may, for some queries, be extraordinarily inefficient (it may not, if you've defined the schema well, because many RDBMSs implicitly create indexes based on schema features like PKs and unique constraints), but you can do it.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#242

Post by ChicagoDave »

rmrfrmrf wrote: post #85
ChicagoDave wrote: post #24

I had the task of replicating data from DynamoDB to Redshift on AWS on two large projects on the last few years. The primary factor in using DynamoDB is that is's cheap and scales like nothing else in the galaxy. Then the other shoe fell. The data was needed for reports. Reports require a static schema. NoSQL (and the developers who love it) despises static schema. "I can add properties whenever I want!" This process…

Putting data gathering and reporting into one database is just asking for torture. Yet, it happens all the time.

I wasn't suggesting that. I was saying there's no real reason transactional and reporting data can't start with schematic replication and actually is more efficient.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#243

Post by heedlessly2 »

coldtea wrote: post #176
randomdrake wrote: post #40

I don't buy it. The makers of the software that this appears to be a carefully written advertisement for, came to the same conclusion as the rest of the IT world (that some of us saw a mile away): NoSQL was, and still is, only good for very specific things in very specific cases; it's generally dreadful for anything that SQL engines could already do well. The subtitle for this contains: "After years of being left for…

> Who? Where's the data? This blog post has a lot of links, references, and studies, but where's the data to back up the premise? If you had been following startup blogs and HN, then you don't need any more data to back their premise. It's not like total data matters anyway -- what's important is what use cases people regularly encounter in their periphery and the part of the industry they work on, which might not be…

>For most of us here around HN, judging from posts, comments, and discussions, the NoSQL era was very real

so you're favoring buzzword hype over actual production environments?

Who cares if people are talking about it but not really using it?

Re: Why SQL is beating NoSQL, and what this means for the future of data

#244

Post by thomascgalvin »

le-mark wrote: post #196
memsom wrote: post #191

About 15 years ago, I worked on a team with a product that used an object based database. No SQL didn't exist, but it was kind of in a similar vein. The database was a pure object storage and there was an index b-tree to make sense of it. We could store anything at all in the database. As long as it inherited from a specific class. It used the object serialization features of the language (Object Pascal) to read and…

It's almost as if "everyone" forgot that before SQL (relational database) there was only NoSQL databases of various sorts (hierarchical, document, etc) and SQL and the relational model arose to address their shortcomings. Personally, I knew stuff had gotten stupid when I sat through a presentation by a gemfire evangalist who advised everyone present to just "do your joins in code". If you need to join data you should…

> It's almost as if "everyone" forgot that before SQL (relational database) there was only NoSQL databases of various sorts (hierarchical, document, etc) and SQL and the relational model arose to address their shortcomings.

I think at least 25% of modern software development involves slowly, painfully re-learning lessons we intentionally forgot ten years ago.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#245

Post by Too »

le-mark wrote: post #196
memsom wrote: post #191

About 15 years ago, I worked on a team with a product that used an object based database. No SQL didn't exist, but it was kind of in a similar vein. The database was a pure object storage and there was an index b-tree to make sense of it. We could store anything at all in the database. As long as it inherited from a specific class. It used the object serialization features of the language (Object Pascal) to read and…

It's almost as if "everyone" forgot that before SQL (relational database) there was only NoSQL databases of various sorts (hierarchical, document, etc) and SQL and the relational model arose to address their shortcomings. Personally, I knew stuff had gotten stupid when I sat through a presentation by a gemfire evangalist who advised everyone present to just "do your joins in code". If you need to join data you should…

There are cases where doing a join in code can make sense. E.g. when you already have a whole table cached in application memory, and the primary purpose of the join is more ORM-like and not any kind of aggregation or grouping on a joined column.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#246

Post by geebee »

bogomipz wrote: post #239

The premise that SQL is "beating" NoSQL is pretty dumb. I have never been aware that such a competition between datastore designs exists. Who is that competition between developers? Users? Most of the places I have worked in the last 5 or 6 years have had both relational and non-relational databases. This is not uncommon. In none of those shops was there a competition between the two databases but rather they were co…

"I have never been aware that such a competition between datastore designs exists"

Good! There certainly shouldn't be. Wish I could say the same. Magpie developers, people who want to seem edgy and forward looking, people who like making others look backwards at meetings, people who are worried they'll look like dinosaurs, people who've been around the block long enough to know an approach is totally unnecessary for their project but will be required on the resume for the next project. Trust me, they were all out there, in force.

And of course, many data stores that aren't SQL are absolutely great for the task. Try to solve the 6 degrees of separation problem with SQL joins, then do it with a graph database. There are all kinds of applications that weren't a great fit for a relational model, and many that are. This article does mention the moment where NoSql was defined as "Not Always SQL" (and dismisses it with "nice try"), but honestly, if this had been the approach up front, it would have saved us a lot of grief.

Probably. Maybe. I hope. Oh, who am I kidding. All that stuff I described above would have happened anyway.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#247

Post by stephenr »

jjirsa wrote: post #49
peterwwillis wrote: post #7

SQL wins because big iron wins. At the end of the day, the big, clunky, slow, complicated, aggravating, old systems stick around because they are built to stick around. Looking at, say, an eventually consistent distributed decentralized kv store, one might be tricked into believing it's simple enough to deploy with enough nodes and general enough interfaces that you can build a complicated system on top of it if need…

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead…

Galera + HAProxy or ProxySQL should give you a multi-master MySQL with no app rewrites required, no?

Re: Why SQL is beating NoSQL, and what this means for the future of data

#248

Post by paulddraper »

viraptor wrote: post #76
paulddraper wrote: post #64

Earlier quoted context omitted.

MySQL has a slow query log with all non-indexed queries.

I found it not very useful. Sometimes you will query without indexes. Sometimes it just doesn't matter. Sometimes it's a table with 2 rows and you don't want an index. In practice I don't want to know when it happens. I want an error to be raised instead so the database doesn't suddenly die.

So the problem is (1) it's too noisy and (2) it doesn't actually disrupt the query.

Simultaneously too much and not enough.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#249

Post by tynpeddler »

marktangotango wrote: post #231
slfnflctd wrote: post #225

Earlier quoted context omitted.

It's hairsplitting at that point, though-- somewhat reminds me of similar issues in biology with classifying organisms. Most SQL database packages are designed to have a wide range of queries stored with the database (many will always only exist in front ends or developer tools, of course), for several reasons. One reason is that it sheds light on data structure and business logic. Another is avoiding needless duplic…

It's most certainly not hair splitting. If the data isn't natural to one table, as the GP says, then it's strong indicator of two things: 1. NoSQL of any kind is not the solution 2. Joining at the location of the data is always preferable to joining in application code due to the Principle of Locality[1]. The amount of data sent over a wire should always be minimized. [1] https://en.wikipedia.org/wiki/Locality_of_ref…

I'm not sure that point 2 is always clearcut. Because of the cartesian product issue, searching for large amounts of data with multiple joins can produce a huge number of returned rows containing a lot of redundant data. I've had a few small optimization issues that I fixed by breaking up one large query with several joins with 2 or 3 queries.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#250

Post by davidw »

bni wrote: post #155
davidw wrote: post #70

Earlier quoted context omitted.

> SQL was originally designed for non-programmers SQL sounds like something from the Star Trek original series era to me. Read it with a Shatner voice: Computer, SELECT course WHERE klingons = 0;

And this is a bad thing? Sounds awesome to me, make the computer do stuff for me without specifying how.

Good? Bad? It's just... a thing from that era. Another go-round of "I know, if it reads like English, it'll be easy to use", which in the present day manifests itself in stuff like Cucumber.
Post reply on HN