Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

Post reply on HN
Print view View original

81–90 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#81

Post by sk5t »

Clubber wrote: post #33
SmellTheGlove wrote: post #18

Earlier quoted context omitted.

> It's a Google/Twitter size problem that most industries wouldn't have, but since it's the new shiny thing, you know how that goes. You nailed it on both points. Very few organizations are going to need that kind of scale, but many want to think that they will! Even when you get to horizontal scaling, you can simply replicate what needs to be replicated. Key-value is really just a subset of relational data, right? I…

Also, the unstructured data is another thing. For example, lets say you need to store a bunch of contracts (as in your industry) in PDF/TIFF or whatever. The filesystem isn't a really good place to store it because you have a loose coupling between the index in the table and the file on the filesystem. That can get messy and unless security is tight, some developer or admin could muck something up on the filesystem.…

> That can get messy and unless security is tight, some developer or admin could muck something up on the filesystem

This isn't a great reason not to use a filesystem--it's a reason not to let staff have at the filesystem like cowboys, and not to avoid replicating and backing up the keys and values on the filesystem as if they are really important.

That said, there isn't really a solid replacement for a database transaction involving both fields + blob, but most situations are okay with, say, stowing a blob in S3, and later making a half-hearted attempt to delete it if the associated database transaction fails.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#82

Post by Clubber »

randomdrake wrote: post #40

I don't buy it. The makers of the software that this appears to be a carefully written advertisement for, came to the same conclusion as the rest of the IT world (that some of us saw a mile away): NoSQL was, and still is, only good for very specific things in very specific cases; it's generally dreadful for anything that SQL engines could already do well. The subtitle for this contains: "After years of being left for…

It's a marketing piece, no doubt, but he is explicitly talking about SQL the language on top of NoSQL, not SQL as in RDBMS.

But to your point SQL the language or RDBMS was never dead, it just wasn't the prettiest girl in the room for all the new GI's that just shipped in.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#83

Post by dgudkov »

pjungwir wrote: post #25

I've always found it a little funny that SQL was originally designed for non-programmers, sort of like AppleScript. I used to think neither of those panned out, but in fact there really are a lot of smart not-programmers who can use it. At a company I work with many of the support staff have been learning SQL to help customers pull reports from our data warehousey reporting database. So maybe the article is onto some…

>I work with many of the support staff have been learning SQL to help customers pull reports from our data warehousey reporting database.

Why wouldn't you use a visual query tool for this? Tableau and similar apps generate pretty decent SQL queries nowadays. Even old-school BusinessObjects does a decent job (although requires way more initial modelling).

Re: Why SQL is beating NoSQL, and what this means for the future of data

#84

Post by misterbowfinger »

craigkerstiens wrote: post #53

I don't think the article could have said it much better. SQL is super powerful and makes much sense in so many ways. Nearly all apps have a relational structure to them and SQL is a reasonable way to interact with them. Some of my favorite conversations from the Postgres community 5-6 years back were when they were talking about a time when Postgres was being disrupted. The gray bearded DBAs (Hi Berkus and others) w…

SQL is easily the best option at the moment, but unlike the experimentation in programming languages that we've seen in the last 10 years or so, we've yet to see much experimentation in query languages (as far as I know!)

ReQL, what RethinkDB uses, was an interesting direction because it allows people to start reasoning about a particular query. This, IMO, is the biggest thing that SQL lacks - the ability to reason about a query. I'd love to see a way to extend SQL that solves this.

The other thing that really, really sucks is that SQL has some terrible documentation and tutorials. Even for PG, for example, it just mentally takes a long time to parse through the documentation to understand how to use a function that solves your use case. This might just be a lack of Stack Overflow posts out there, but it's definitely not as friendly as, say, finding answers for Ruby code.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#85

Post by rmrfrmrf »

ChicagoDave wrote: post #24

I had the task of replicating data from DynamoDB to Redshift on AWS on two large projects on the last few years. The primary factor in using DynamoDB is that is's cheap and scales like nothing else in the galaxy. Then the other shoe fell. The data was needed for reports. Reports require a static schema. NoSQL (and the developers who love it) despises static schema. "I can add properties whenever I want!" This process…

Putting data gathering and reporting into one database is just asking for torture. Yet, it happens all the time.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#87

Post by ChicagoDave »

curun1r wrote: post #75
ChicagoDave wrote: post #24

I had the task of replicating data from DynamoDB to Redshift on AWS on two large projects on the last few years. The primary factor in using DynamoDB is that is's cheap and scales like nothing else in the galaxy. Then the other shoe fell. The data was needed for reports. Reports require a static schema. NoSQL (and the developers who love it) despises static schema. "I can add properties whenever I want!" This process…

> Reports require a static schema This may be true, but reports also normally require a different schema from the OLTP database. Data warehousing with star schemas is a thing and it doesn't matter whether your OLTP database is an RDBMS or a NoSQL solution...you're going to have to push data to your warehouse to run those reports. The last thing you want to be doing is running reporting queries against a database that…

The level of effort to move data from DynamoDB to a data-warehouse (which is what Redshift is), is not trivial. It may be that the Data Pipeline tools within AWS are still opaque and incomplete, but but I've seen the same problem with MongoDB implementations.

Reporting is nearly always an afterthought in system architecture and it should be a first-class requirement.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#88

Post by rhencke »

elvinyung wrote: post #61
jjirsa wrote: post #49

Earlier quoted context omitted.

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead…

You got downvoted, but I agree with you. Traditional RDBMS is inherently single-node and a pain to scale. The actual sad thing here is that there's no mature/reliable commodity (i.e. open source) distributed RDBMS for transaction processing yet.

CockroachDB? I'll grant you it's still young, but it pretty solidly hits all your other checkboxes.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#89

Post by kristianp »

pjungwir wrote: post #25

I've always found it a little funny that SQL was originally designed for non-programmers, sort of like AppleScript. I used to think neither of those panned out, but in fact there really are a lot of smart not-programmers who can use it. At a company I work with many of the support staff have been learning SQL to help customers pull reports from our data warehousey reporting database. So maybe the article is onto some…

I think Tutorial D would be more suitable as an ORM than a layer on top of SQL, I feel there would be less of an impedance mismatch if a Tutorial D implementation is a DSL. There is a Haskell implementation, but I'm not going to be doing work in Haskell any time soon.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#90

Post by 6nf »

jjirsa wrote: post #65
SmellTheGlove wrote: post #10

This was a nice read for the history. Honestly, being in a big legacy industry (Insurance), it's as though NoSQL never happened. We're too big, at least my organization, to have made the wholesale change and we've been plugging along mostly in Teradata and DB2 for a long time. Teradata in particular has performed well across a variety of use cases, its only large downside being the cost. Anyhow, only point I'd add is…

The reason people don’t start with a relational DB has nothing to do with the data model or query language - people choose databases like Cassandra because scaling to 500 Postgres or MySQL instances holding a combined petabyte of data is horrific, but it’s dirt simple in things like Cassandra The query language is a side effect of the underlying storage engine - you don’t choose it because you want a key value store,…

You make it sound like many people are using petabyte scale databases. This is not even close to true. ESPECIALLY when you're just starting out. Most new projects will start with almost no data at all and the vast majority of projects will never ever get to the petabyte scales.
Post reply on HN