Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

151–160 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#151
post #49

Earlier quoted context omitted.

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead…

You got downvoted, but I agree with you. Traditional RDBMS is inherently single-node and a pain to scale. The actual sad thing here is that there's no mature/reliable commodity (i.e. open source) distributed RDBMS for transaction processing yet.

I have to bring this up again: NewSQL. There are quite a few new choices out there, Google Spanner, Cockroachdb(https://github.com/cockroachdb/cockroach), TiDB (https://github.com/pingcap/tidb). All of them are very easy to scale while maintaining the ACID transactions.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#152
I like SQL. I've never failed to do anything with it and find it very powerful.

But while that aspect of the SQL experience I'm comfortable with, I'm still just a developer and not a DBA. I don't even _want_ to be a DBA. And as someone putting the pieces of a startup together in a country where DBA skills are practically non-existent (meaning I can't reliably train someone up myself) a cloud hosted NoSQL solution seems a much better bet on paper.

I've noticed that when the topic of SQL vs NoSQL comes up, the issue of cost of infrastructure maintenance always seems to get glossed over.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#153
post #33

Earlier quoted context omitted.

> It's a Google/Twitter size problem that most industries wouldn't have, but since it's the new shiny thing, you know how that goes. You nailed it on both points. Very few organizations are going to need that kind of scale, but many want to think that they will! Even when you get to horizontal scaling, you can simply replicate what needs to be replicated. Key-value is really just a subset of relational data, right? I…

Also, the unstructured data is another thing. For example, lets say you need to store a bunch of contracts (as in your industry) in PDF/TIFF or whatever. The filesystem isn't a really good place to store it because you have a loose coupling between the index in the table and the file on the filesystem. That can get messy and unless security is tight, some developer or admin could muck something up on the filesystem.…

Contractual information can quite often be quite relational in nature. I have worked for organisations that had custom contracts and custom sales documentation (different industries) that were based on templates that could be filled in from a database.

In some cases, the companies were successful in storing the relevant information in a database and use that to generate the documents required based on document templates. This is not hard, but it can be tedious to set up. The companies also have to have a will to walk down that path and many do not see a benefit to this kind of process.

It takes a very insightful designer to manage and lead this process as well as a client who has an incentive to move in this direction. One of the considerations is that the approved contract must be in some unalterable form that can be tested against the contract generation process from the information within the database. So that there is always a means of ensuring that any information in the database cannot be corrupted (in a legal sense). That is a matter of safeguards not database information storage capabilities and there are many different ways to do this kind of security.

The problem of sharing of data and replication is not often thought through properly. Oft times, I have seen replication being performed wholesale instead of piecemeal. Similarly for data sharing and the consequences have been less than efficient. Oft times, the actual requirements for sharing and repliation are very small and it is a matter of understanding what is actually required and what is desired for the business.

Finally, very little data is unstructured if you look at it correctly. Many times, I have seen people talk about unstructured data (including documents of all kinds) and when you look at the data, it is highly structured. It becomes a matter of seeing what that structure is and working out what is variable and what is constant data within the structure.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#154

Earlier quoted context omitted.

> SQL has never been a particularly elegant language I hear that often. But SQL is close to a 1-to-1 mapping with sets of data. It's refreshing to use something as math like when programming. I find that elegant.

You might find the criticisms in this article interesting (on the jooq blog, a java query builder library). https://blog.jooq.org/2016/12/09/a-beginners-guide-to-the-tr...

Indeed that is valid criticism. I suppose after using SQL for a number of years, the logical order is just ingrained. I would be nice for it to be required to be written in the logical order though.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#155
post #70

I've always found it a little funny that SQL was originally designed for non-programmers, sort of like AppleScript. I used to think neither of those panned out, but in fact there really are a lot of smart not-programmers who can use it. At a company I work with many of the support staff have been learning SQL to help customers pull reports from our data warehousey reporting database. So maybe the article is onto some…

> SQL was originally designed for non-programmers SQL sounds like something from the Star Trek original series era to me. Read it with a Shatner voice: Computer, SELECT course WHERE klingons = 0;

And this is a bad thing? Sounds awesome to me, make the computer do stuff for me without specifying how.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#156
NoSQL is a really terrible term. I don't know why so many people seem eager to the argue the merits of an incredibly disparate umbrella label that includes databases that have almost nothing in common. What meaningful things can you say about a category that includes Cassandara, Datalog, LDMB and Neo4J?

I think when a lot of people talk about NoSQL they just want to rant against a certain kind of strawman programmer. You know the one. Young, stupid, naiive, too arrogant to learn nth normal form or define schemas. This programmer probably uses nodejs or some other such heresy and only wants quick results, integrity be damned!

Don't get me wrong, Relational Databases are really good, and fit a lot of problems really well. But there do exist legitimate use cases (not necessarily scale!) where an RDBMS will simply be a poor fit, or a lot more work. Don't dredge up the strawman of the programmer too arrogant to learn SQL because you're too arogant to learn the merits of something that isn't SQL.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#157

This was a nice read for the history. Honestly, being in a big legacy industry (Insurance), it's as though NoSQL never happened. We're too big, at least my organization, to have made the wholesale change and we've been plugging along mostly in Teradata and DB2 for a long time. Teradata in particular has performed well across a variety of use cases, its only large downside being the cost. Anyhow, only point I'd add is…

The thing about the NoSQL trend is that it's still very valid in some areas. Even big and slow companies have some project or supplier that uses a software with a NoSQL service in it.

You just rarely see those bundled redis or elasticsearch servers that are crucial for some app to do its session management and search engine.

The most disturbing part of the NoSQL trend is that people are treating it like a battle between two competing systems.

I have at least one major system under my own belt that uses relational SQL for backend data, ES for search engine and cassandra for TSDB.

I need and trust all those services to work as one unit.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#158

Earlier quoted context omitted.

> Instead of smartly reasoning about the data, it is all to easy to just "JOIN ALL THE THINGS WITH MEGA TEMP TABLES!". This is so common, I'd love for the popular databases to add table flags that prevent it by accident. Letting me configure "this table most not full scan or file sort implicitly" would get rid of half the incident callouts I've been involved in. You could always override it in the query where needed.

At best (if your users aren’t allowed to write SQL), that would change your “it is slow” calls to “it doesn’t work”. At worst, I fear your users would learn to override it by default, as just one other part of the magic incantation needed to please the SQL gods. I think it would be better to have the planner send out emails “this query has to use a full scan” or, “this query is on the brink of changing strategy compa…

> that would change your “it is slow” calls to “it doesn’t work”.

But that's exactly what I'm asking for. "It's slow" means that it will work until the breaking point and then wake me up. "It doesn't work" with the right reporting allows me to teach someone about indexing during office hours.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#159
post #66

A bit surprised to find no mention of the GPU database field (ie: https://www.mapd.com/demos/ , etc) because straight SQL on this hardware is extremely fast when optimized. Feels like all this NoSQL stuff came around when developers no longer wanted to deal w/ DBAs and do full stack(s) on their own. DBAs spent long hours on DDL and DML (stored procedures) for a good reason.

I'd mention an actual capable SQL database a lot before MapD...

Like SQream or Kinetica

Re: Why SQL is beating NoSQL, and what this means for the future of data

#160
post #33

Earlier quoted context omitted.

> It's a Google/Twitter size problem that most industries wouldn't have, but since it's the new shiny thing, you know how that goes. You nailed it on both points. Very few organizations are going to need that kind of scale, but many want to think that they will! Even when you get to horizontal scaling, you can simply replicate what needs to be replicated. Key-value is really just a subset of relational data, right? I…

Also, the unstructured data is another thing. For example, lets say you need to store a bunch of contracts (as in your industry) in PDF/TIFF or whatever. The filesystem isn't a really good place to store it because you have a loose coupling between the index in the table and the file on the filesystem. That can get messy and unless security is tight, some developer or admin could muck something up on the filesystem.…

[deleted]
Post reply on HN