Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

Post reply on HN
Print view View original

141–150 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#141

Post by GrayShade »

happymellon wrote: post #133
jjirsa wrote: post #95

Earlier quoted context omitted.

Many people ARE using petabyte scale databases. They don't start out that way, but they will be if they're successful, and if they're not they probably shouldn't be running their own databases anyway (go RDS it up or something).

What? No they aren't, I work with satellite imagery processing which is quite large in it raw data format, and after a decade we are not dealing with petabytes of active data, hundreds of gigs for a full earth coverage. Before that I have held positions in finance, dealing with realtime transaction processing. We did not work in petabytes. If you are working in petabytes you are storing crap in your production databa…

> What? No they aren't, I work with satellite imagery processing which is quite large in it raw data format, and after a decade we are not dealing with petabytes of active data, hundreds of gigs for a full earth coverage.

This get large rather fast when processing time series from multiple sensors.

But I agree with you otherwise, and it's not like you're storing satellite products in a database.

Now I'm curious what NSA is using :).

Re: Why SQL is beating NoSQL, and what this means for the future of data

#142

Post by kazagistar »

ojosilva wrote: post #136
SmellTheGlove wrote: post #10

This was a nice read for the history. Honestly, being in a big legacy industry (Insurance), it's as though NoSQL never happened. We're too big, at least my organization, to have made the wholesale change and we've been plugging along mostly in Teradata and DB2 for a long time. Teradata in particular has performed well across a variety of use cases, its only large downside being the cost. Anyhow, only point I'd add is…

>I think the bigger issue is that you can use an RDBMS to do NoSQL-like things, such as key-value stores, with the flexibility to structure your data if you need that later. So why not start with a relational database? I've tried that in the past and failed miserably. 1) values in a key-value table will endup needing to hold nested data structures, such as a JS object/hash. Ie. mykey={...} 2) turning values to JSON (…

1. Postgres has a json and jsonb datatype specifically to be your values.

2. It has built in query and modification tools that can manage the nested objects and array and whatnot.

3. It let's you build indexes to help with all this, including reverse indexes for asking something like "is this string in the array inside this object".

4. You don't have to specify your schema in sone horrid way, you can just throw what you want at it and specify later, and get performance numbers that are competitive and even exceed some popular NoSql stores.

5. You can mix and match freely with all the niceties of a proper SQL database with transactions, joins, constraints, etc, as you see fit to evolve.

So yeah, don't just throw your keys and values into some text table somewhere; but use instead one of the best proper kv stores, that just happens to also be one of the best sql databases, and figure out how you want your schema you need it.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#143

Post by kazagistar »

hvidgaard wrote: post #134
craigkerstiens wrote: post #53

I don't think the article could have said it much better. SQL is super powerful and makes much sense in so many ways. Nearly all apps have a relational structure to them and SQL is a reasonable way to interact with them. Some of my favorite conversations from the Postgres community 5-6 years back were when they were talking about a time when Postgres was being disrupted. The gray bearded DBAs (Hi Berkus and others) w…

> SQL has never been a particularly elegant language I hear that often. But SQL is close to a 1-to-1 mapping with sets of data. It's refreshing to use something as math like when programming. I find that elegant.

You might find the criticisms in this article interesting (on the jooq blog, a java query builder library).

https://blog.jooq.org/2016/12/09/a-beginners-guide-to-the-tr...

Re: Why SQL is beating NoSQL, and what this means for the future of data

#144

Post by Someone »

viraptor wrote: post #44
lmickh wrote: post #34

I like how it pretty much just glosses over decades of familiarity. If there is anything the last 20 years prove, it is that the majority of developers will stick with what they know over what might be a good fit for the job. It goes even deeper in the SQL world down to the specific database flavor. From the ops side I actually find RDBMS more difficult to deal with cause the power of relationships is easy to abuse a…

> Instead of smartly reasoning about the data, it is all to easy to just "JOIN ALL THE THINGS WITH MEGA TEMP TABLES!". This is so common, I'd love for the popular databases to add table flags that prevent it by accident. Letting me configure "this table most not full scan or file sort implicitly" would get rid of half the incident callouts I've been involved in. You could always override it in the query where needed.

At best (if your users aren’t allowed to write SQL), that would change your “it is slow” calls to “it doesn’t work”.

At worst, I fear your users would learn to override it by default, as just one other part of the magic incantation needed to please the SQL gods.

I think it would be better to have the planner send out emails “this query has to use a full scan” or, “this query is on the brink of changing strategy compared to earlier runs”

Re: Why SQL is beating NoSQL, and what this means for the future of data

#145

Post by pmontra »

labster wrote: post #109

If SQL is to databases as Javascript is to browsers — ubiquitous and largely standard across the market — are there any languages that transpile to SQL? People in this thread have commented that SQL is clunky, and JS definitely fit that description for a while. So I'm wondering if there are any alternatives that would prove SQL's clunkiness to me.

Every ORM is a transpiler from a language to SQL.

ActiveRecord transpiles from Ruby to SQL. Django ORM and SQLAlchemy are popular ones to write SQL in Python. Ecto is from Elixir. Every language has its own transpilers. Not all of them are easy to use and they are much harder to use than SQL for complex queries, maybe even impossible to build them.

Frankly I'm a little fed up to have to learn a new way of generating SQL for every language and framework. I'm working on or maintaining projects in Rails, Django, Web2py and Elixir for my customers and I'm coming to wish we were using plain SQL. After all I knew SQL long before any of those other technologies existed, I know how to create prepared statements and how to use positional parameters to avoid SQL injections. Then there is how to efficiently extract the data in the resultsets, create data structures in the original language and possibly cache queries. They don't look so hard (maybe handling joins gracefully.)

Re: Why SQL is beating NoSQL, and what this means for the future of data

#146

Post by jondubois »

this article is overly opinionated and narrow-minded.

Many startups use NoSQL. Personally I prefer NoSQL for most use cases. That said I have nothing against SQL itself but I will NEVER go back to using an ORM - They're a dirty hack; always have been and always will be. I'd rather write the SQL by hand.

I love using MongoDB; it's very simple to use. I liked RethinkDB's ReQL even more (even though the company itself had to shut down, the open source project is still great).

The main problem with SQL databases is that the powerful features offered by SQL don't scale; so if you want to build a future-proof system, you have to avoid using certain features like joins; the advantage of NoSQL is that it's built to scale; the constraints are are feature.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#147

Post by arkh »

jondubois wrote: post #146

this article is overly opinionated and narrow-minded. Many startups use NoSQL. Personally I prefer NoSQL for most use cases. That said I have nothing against SQL itself but I will NEVER go back to using an ORM - They're a dirty hack; always have been and always will be. I'd rather write the SQL by hand. I love using MongoDB; it's very simple to use. I liked RethinkDB's ReQL even more (even though the company itself h…

> you have to avoid using certain features like joins

Too bad it is kinda sorta the thing with a RDBM. Relationnal. When you have to work with data which are related and don't want to do stupid shit like updating the infos of an author in each of its written article you want SQL.

But nothing prevents you from using a NoSQL database on top or even multiple ones. If you have multiple way to use your data, having multiple read or write models is a plus.

About scale: rare are the people who can't just add more hardware to the problem.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#148

Post by Mikushi »

kazagistar wrote: post #142
ojosilva wrote: post #136

Earlier quoted context omitted.

>I think the bigger issue is that you can use an RDBMS to do NoSQL-like things, such as key-value stores, with the flexibility to structure your data if you need that later. So why not start with a relational database? I've tried that in the past and failed miserably. 1) values in a key-value table will endup needing to hold nested data structures, such as a JS object/hash. Ie. mykey={...} 2) turning values to JSON (…

1. Postgres has a json and jsonb datatype specifically to be your values. 2. It has built in query and modification tools that can manage the nested objects and array and whatnot. 3. It let's you build indexes to help with all this, including reverse indexes for asking something like "is this string in the array inside this object". 4. You don't have to specify your schema in sone horrid way, you can just throw what…

Same can be said of MySQL. Not comparing to Postgres, but I've been amazed at how many people do not know that either of these as a powerful JSON type.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#149

Post by e12e »

ww520 wrote: post #110
pjungwir wrote: post #73

Earlier quoted context omitted.

Hi Craig! Postgres refers to itself as an "object-relational database", but do you think the "object" part is meaningful today, or just a label left over from the 90s? As best as I can tell, it just means that Postgres has table inheritance, which is great for partitioning but has never been all that feature-ful. (I guess it is improving a lot with pg 10 though!) Is there any theory or academic research on what an "o…

Mike Stonebraker started Ingres to build a relational database. After Ingres, he and Larry Rowe wanted to see what could be done to add the OO aspect and other complex data types in addition to the relational model; OO was starting to take off back then. They started the Postgres project (Post-Ingres) in UCB to research on object-relational database. It turned out the object-relational part was better done on the cli…

I'll note that postgres is (sort-of) a storage option for one of the (rather) early object databases, the much maligned, yet perhaps under-used zodb:

http://www.zodb.org/

Still evolving in its old age, along with postgres:

http://www.newtdb.org/en/latest/how-it-works.html

Other than that I'm aware of gemstone/s:

https://gemtalksystems.com/products/gs64/

Which can be paired with the maglev ruby implementation for similar (but presumably higher performing, more featurefull) experience with distributed objects in ruby:

http://maglev.github.io/

I suppose that with jsonb, and a node/js server, one could indeed argue that postgres is (again) an object oriented db - but I don't think anyone would push that argument too strongly.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#150

Post by mercer »

pmontra wrote: post #145
labster wrote: post #109

If SQL is to databases as Javascript is to browsers — ubiquitous and largely standard across the market — are there any languages that transpile to SQL? People in this thread have commented that SQL is clunky, and JS definitely fit that description for a while. So I'm wondering if there are any alternatives that would prove SQL's clunkiness to me.

Every ORM is a transpiler from a language to SQL. ActiveRecord transpiles from Ruby to SQL. Django ORM and SQLAlchemy are popular ones to write SQL in Python. Ecto is from Elixir. Every language has its own transpilers. Not all of them are easy to use and they are much harder to use than SQL for complex queries, maybe even impossible to build them. Frankly I'm a little fed up to have to learn a new way of generating…

I've heard good things about Ecto's approach (Elixir). Would you say it's at least better than many other ORM's, or is it similar enough in regards to drawbacks/advantages compared SQL that this 'better' doesn't matter much?
Post reply on HN