Why SQL is beating NoSQL, and what this means for the future of data

Post by sctb » Tue, Sep 26, 2017, 6:03 PM UTC

primeblue wrote: post #187

craigkerstiens wrote: post #53

I don't think the article could have said it much better. SQL is super powerful and makes much sense in so many ways. Nearly all apps have a relational structure to them and SQL is a reasonable way to interact with them. Some of my favorite conversations from the Postgres community 5-6 years back were when they were talking about a time when Postgres was being disrupted. The gray bearded DBAs (Hi Berkus and others) w…

XML databases lol

Post by collyw » Tue, Sep 26, 2017, 7:17 PM UTC

taco_emoji wrote: post #233

autokad wrote: post #217

Earlier quoted context omitted.

RDBMS are such a production kill. imagine you just received two TB data sets, and you'd like to join them and run a process. In hive, you create the schema (pretty simple) and join them. queries may take a few minutes but you get to doing the work you intended all along. In RDBMS, you have to fus with its schema, "you said 125 characters for that string field and you just received 130, FAIL". Its a STRING JUST DEAL W…

> "you said 125 characters for that string field and you just received 130, FAIL" Most (all?) RDBMS's have something like a varchar(max) which accepts reasonably large strings. > you have to create indexes and fus with all sorts of relational nonsense No, you don't. > its going to read the entire table into memory and re-write it back to disk Probably not, if you're adding a nullable column.

Post by vgt » Tue, Sep 26, 2017, 8:09 PM UTC

tyingq wrote: post #258

vgt wrote: post #255

Earlier quoted context omitted.

While I agree that there are situations where "network egress" can become a large part of one's bill, I encourage you to think of the technical impact of having something like Spanner and something like the Google network. A large part of what Spanner does to sustain its high cross-regional availability at strong consistency relies on Google's powerful network [1]. Spanner isn't just source code - it's vertically int…

Sure. I wasn't trying to make it sound like a conspiracy. Just that the constraints of running it only on the Google cloud invalidate some typical use cases. For many companies, you would have to migrate quite a high percentage of your business data out there to avoid huge amounts of egress charges. Spanner also doesn't support INSERT, UPDATE, etc, which may also be limiting.

Post by jjirsa » Tue, Sep 26, 2017, 9:15 PM UTC

bildung wrote: post #223

jjirsa wrote: post #65

Earlier quoted context omitted.

The reason people don’t start with a relational DB has nothing to do with the data model or query language - people choose databases like Cassandra because scaling to 500 Postgres or MySQL instances holding a combined petabyte of data is horrific, but it’s dirt simple in things like Cassandra The query language is a side effect of the underlying storage engine - you don’t choose it because you want a key value store,…

Postgres (as an example) has no upper limit for database size, and the upper limit for a single table was 64TB a while ago, perhaps higher now. And you can buy stock hardware with 6 TB RAM and 76TB SSD e.g. from Dell. If you max out that single server , you could easily hire one database engineer to care about horizontal scaling.

Post by jinqueeny » Tue, Sep 26, 2017, 10:45 PM UTC

elvinyung wrote: post #263

jinqueeny wrote: post #151

Earlier quoted context omitted.

I have to bring this up again: NewSQL. There are quite a few new choices out there, Google Spanner, Cockroachdb( https://github.com/cockroachdb/cockroach ), TiDB ( https://github.com/pingcap/tidb ). All of them are very easy to scale while maintaining the ACID transactions.

I said open source and mature :) IMO really the best option for distributed OLTP RDBMS/NewSQL right now is basically either Citus or Vitess, only because they are built on the backs of extremely mature databases. It really feels like all other options are not mature enough (except maybe VoltDB).