Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

Post reply on HN
Print view View original

231–240 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#231

Post by marktangotango »

slfnflctd wrote: post #225
DonnyV wrote: post #214

Earlier quoted context omitted.

If your going to join data that usually means the application needs a representation of the data that is not naturally stored in 1 table. To me thats a red flag which tells me this "logic" should be stored with the app, not with the database.

It's hairsplitting at that point, though-- somewhat reminds me of similar issues in biology with classifying organisms. Most SQL database packages are designed to have a wide range of queries stored with the database (many will always only exist in front ends or developer tools, of course), for several reasons. One reason is that it sheds light on data structure and business logic. Another is avoiding needless duplic…

It's most certainly not hair splitting. If the data isn't natural to one table, as the GP says, then it's strong indicator of two things:

1. NoSQL of any kind is not the solution

2. Joining at the location of the data is always preferable to joining in application code due to the Principle of Locality[1]. The amount of data sent over a wire should always be minimized.

[1] https://en.wikipedia.org/wiki/Locality_of_reference

Re: Why SQL is beating NoSQL, and what this means for the future of data

#233

Post by taco_emoji »

autokad wrote: post #217
Clubber wrote: post #12

Earlier quoted context omitted.

>So why not start with a relational database? I asked the same question. The biggest reason I've heard is RDBMS's don't horizontally scale well, meaning you can't easily have 50 replicated nodes across the globe and expect it to perform well, or setup easily, because it's fairly complicated with an RDBMS. There's things like Oracle's grid or SQL's high availability clusters, but they get complicated fast, particularl…

RDBMS are such a production kill. imagine you just received two TB data sets, and you'd like to join them and run a process. In hive, you create the schema (pretty simple) and join them. queries may take a few minutes but you get to doing the work you intended all along. In RDBMS, you have to fus with its schema, "you said 125 characters for that string field and you just received 130, FAIL". Its a STRING JUST DEAL W…

> "you said 125 characters for that string field and you just received 130, FAIL"

Most (all?) RDBMS's have something like a varchar(max) which accepts reasonably large strings.

> you have to create indexes and fus with all sorts of relational nonsense

No, you don't.

> its going to read the entire table into memory and re-write it back to disk

Probably not, if you're adding a nullable column.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#234

Post by reboog711 »

sb8244 wrote: post #215
reboog711 wrote: post #213

Earlier quoted context omitted.

What makes you think SQL didn't exist 15 years ago? SQL has been around since 1974 according to Wikipedia. There was a ton of SQL Work during the first dot com era of the mid-90s. On another note, I did a lot of Lotus Notes working during that time, which was clearly a no-SQL database.

Author means that NoSQL as a term / fad didn't exist

I see that now. His space between no and SQL confused me.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#235

Post by nwatson »

dhd415 wrote: post #192
Clubber wrote: post #33

Earlier quoted context omitted.

Also, the unstructured data is another thing. For example, lets say you need to store a bunch of contracts (as in your industry) in PDF/TIFF or whatever. The filesystem isn't a really good place to store it because you have a loose coupling between the index in the table and the file on the filesystem. That can get messy and unless security is tight, some developer or admin could muck something up on the filesystem.…

It's proprietary and therefore not an option for everyone, but SQL Server's "Filestream" feature [1] was designed for the "large blob of unstructured data" use case (small blobs of unstructured data generally work fine in regular tables). It stores large blob data efficiently in individual files directly on disk that are still managed by the database engine and written/read in the same transaction as table data. It's…

Postgres has the BLOB functionality, but also has the JSONB data type that not only stores structured JSON data efficiently, but also allows indexing on the contents of that data. So you can use BLOBs and explicitly store in the table what you need to search on, or else you can avoid that and take the pseudo-NoSQL route and store indexed JSON.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#236

Post by autokad »

taco_emoji wrote: post #233
autokad wrote: post #217

Earlier quoted context omitted.

RDBMS are such a production kill. imagine you just received two TB data sets, and you'd like to join them and run a process. In hive, you create the schema (pretty simple) and join them. queries may take a few minutes but you get to doing the work you intended all along. In RDBMS, you have to fus with its schema, "you said 125 characters for that string field and you just received 130, FAIL". Its a STRING JUST DEAL W…

> "you said 125 characters for that string field and you just received 130, FAIL" Most (all?) RDBMS's have something like a varchar(max) which accepts reasonably large strings. > you have to create indexes and fus with all sorts of relational nonsense No, you don't. > its going to read the entire table into memory and re-write it back to disk Probably not, if you're adding a nullable column.

you obviously have not used databases very much if you think you can join 2 billion row tables on a key that is not indexed in a reasonable amount of time.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#237

Post by EvilGrin »

GrayShade wrote: post #141
happymellon wrote: post #133

Earlier quoted context omitted.

What? No they aren't, I work with satellite imagery processing which is quite large in it raw data format, and after a decade we are not dealing with petabytes of active data, hundreds of gigs for a full earth coverage. Before that I have held positions in finance, dealing with realtime transaction processing. We did not work in petabytes. If you are working in petabytes you are storing crap in your production databa…

> What? No they aren't, I work with satellite imagery processing which is quite large in it raw data format, and after a decade we are not dealing with petabytes of active data, hundreds of gigs for a full earth coverage. This get large rather fast when processing time series from multiple sensors. But I agree with you otherwise, and it's not like you're storing satellite products in a database. Now I'm curious what…

You could just check their Github? :)

https://github.com/nationalsecurityagency

Re: Why SQL is beating NoSQL, and what this means for the future of data

#238

Post by SmellTheGlove »

INTPenis wrote: post #157
SmellTheGlove wrote: post #10

This was a nice read for the history. Honestly, being in a big legacy industry (Insurance), it's as though NoSQL never happened. We're too big, at least my organization, to have made the wholesale change and we've been plugging along mostly in Teradata and DB2 for a long time. Teradata in particular has performed well across a variety of use cases, its only large downside being the cost. Anyhow, only point I'd add is…

The thing about the NoSQL trend is that it's still very valid in some areas. Even big and slow companies have some project or supplier that uses a software with a NoSQL service in it. You just rarely see those bundled redis or elasticsearch servers that are crucial for some app to do its session management and search engine. The most disturbing part of the NoSQL trend is that people are treating it like a battle betw…

> The thing about the NoSQL trend is that it's still very valid in some areas. Even big and slow companies have some project or supplier that uses a software with a NoSQL service in it.

This is a good point and I didn't do a good job of acknowledging it in my prior comment. There are use cases for NoSQL, but to me it's a tool for a narrower set of circumstances. My default is relational, unless I have a clear idea of why that isn't the right fit.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#239

Post by bogomipz »

The premise that SQL is "beating" NoSQL is pretty dumb. I have never been aware that such a competition between datastore designs exists.

Who is that competition between developers? Users? Most of the places I have worked in the last 5 or 6 years have had both relational and non-relational databases. This is not uncommon. In none of those shops was there a competition between the two databases but rather they were complimentary.

This article and title seem to be very self-serving for their own product. Its seems to willfully conflate SQL the interface and SQL a general term for a relational database.

NoSQL has always been something of a misnomer - non-relational would have been a better term but it doesn't sound as buzz-worthy I suppose.

The SQL like interface has been in Cassandra for longer now than it was absent. The gain in SQL like interfaces for non-relational databases is because it's familiar and works really well. Anywhere there is a database there is going to be a need for selection, filtering and projections of tuples.

>"In Amazon’s own words, its PostgreSQL- and MySQL-compatible database Aurora database product has been the “fastest growing service in the history of AWS”.

Is this really surprising that the world's largest cloud provider is selling more databases than anything else? Almost everyone needs a database, given howrelational database, there are more people around that have experience with it.

>"To understand why SQL is making a comeback ..."

No, SQL never went away. Full stop.

>"But don’t take our word for it. Take Google’s"

No, this practice of blindly adopting ideas just because they work for Google needs to stop.

This post sounds as if the author(s) themselves bought into all of the NoSQL hype that buzzword-obsessed tech journalists were spinning and they are just now having an epiphany that much of that hype was just that.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#240

Post by coldtea »

magoghm wrote: post #201
coldtea wrote: post #178

Earlier quoted context omitted.

> I've always found it a little funny that SQL was originally designed for non-programmers That's not true.

"2. SQL is intended to be accessible to users without formal training in mathematics or computer programming. It is designed to be typed on a keyboard. Therefore it is framed in familiar English keywords, and avoids specialized mathematical concepts or symbols." http://researcher.ibm.com/researcher/files/us-dchamber/SQL-e...

I stand corrected.

Had more in mind the relational calculus, for which SQL is just a flawed expression query language for.

Post reply on HN