Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why SQL is beating NoSQL, and what this means for the future of data

blog.timescale.com

Post reply on HN
Print view View original

301–310 of 310 posts

Re: Why SQL is beating NoSQL, and what this means for the future of data

#301

Post by jjirsa »

stephenr wrote: post #281
jjirsa wrote: post #280

Earlier quoted context omitted.

Primary key conflicts are still a thing. Especially problematic where you're either not using auto_incrementing PKs, and/or you try to switch writes from one master to the other and app has to be aware of the switch.

Not using auto incrementing keys is surely a rarity with MySQL projects. Even before galera was a thing regular MySQL replication could use auto_increment_(increment|offset) to make each server generate unique id's.

There exists a LOT of data models that don’t use auto incrementinf primary keys. Rarity or not, it’s a real use case and a real problem for master/master replication

Also, even if you do use auto increment with offsets for each server, you’ll have to know how many servers you’ll eventually have, or waste offsets and eventually risk outrunning your data types

Re: Why SQL is beating NoSQL, and what this means for the future of data

#302

Post by stephenr »

jjirsa wrote: post #301
stephenr wrote: post #281

Earlier quoted context omitted.

Not using auto incrementing keys is surely a rarity with MySQL projects. Even before galera was a thing regular MySQL replication could use auto_increment_(increment|offset) to make each server generate unique id's.

There exists a LOT of data models that don’t use auto incrementinf primary keys. Rarity or not, it’s a real use case and a real problem for master/master replication Also, even if you do use auto increment with offsets for each server, you’ll have to know how many servers you’ll eventually have, or waste offsets and eventually risk outrunning your data types

Non auto keys will already need some way to ensure uniqueness that will involve the application expecting conflicts.

For those that do use auto inc, galera will manage the offset/increment value based on cluster size automatically.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#303

Post by collyw »

jjirsa wrote: post #267
collyw wrote: post #265

Earlier quoted context omitted.

The vast majority of hipster NoSQL developers I have debated with are using a single machine.

Those people are objectively wrong.

You try and explain that to them, but whatever database they are using "is fast". Not that I have ever had a problem with relational databases being slow. Certain queries, yes, but a bit of rewriting them and proper indexing usually solves that.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#304

Post by taco_emoji »

autokad wrote: post #300
testvox wrote: post #259

Earlier quoted context omitted.

So? It's not like a nosql db without indexes will be any faster at it.

yes, its absolutely faster. try it. Granted a nosql db like hive is 'cheating' by taking advantage of lots of disks/hardware on lots of machines, but i routinely join billion row tables with million row tables and wait only a minute.

> Granted a nosql db like hive is 'cheating' by taking advantage of lots of disks/hardware on lots of machines

That's not "cheating", it's literally the only relevant factor.

Who would win in a fight: a superintelligent ninja cat with laser nunchucks and body armor, or a dog?

Re: Why SQL is beating NoSQL, and what this means for the future of data

#305

Post by tynpeddler »

oldandtired wrote: post #282
tynpeddler wrote: post #249

Earlier quoted context omitted.

I'm not sure that point 2 is always clearcut. Because of the cartesian product issue, searching for large amounts of data with multiple joins can produce a huge number of returned rows containing a lot of redundant data. I've had a few small optimization issues that I fixed by breaking up one large query with several joins with 2 or 3 queries.

Something we discovered years ago in relation to joins in all of the SQL DBMS systems we used (from the lowly MS-Access to ORACLE and SQL-Server). The order in which you create the joins in the SQL determines how bad the cartesian product cardinality is. The DBMS's seemed incapable of rearranging the joins internally to get the smallest number of records. We had quite a few instances where by a judicious change in th…

That's strange. While the cartesian product is not commutative (because of the order in which items appear in the final product), the cardinality of the result result set should be independent of the order in which records are joined because the cardinality is obtained by multiplying the cardinality of each constituent set.

https://en.wikipedia.org/wiki/Cartesian_product#Cardinality

Re: Why SQL is beating NoSQL, and what this means for the future of data

#306

Post by Clubber »

taco_emoji wrote: post #304
autokad wrote: post #300

Earlier quoted context omitted.

yes, its absolutely faster. try it. Granted a nosql db like hive is 'cheating' by taking advantage of lots of disks/hardware on lots of machines, but i routinely join billion row tables with million row tables and wait only a minute.

> Granted a nosql db like hive is 'cheating' by taking advantage of lots of disks/hardware on lots of machines That's not "cheating", it's literally the only relevant factor. Who would win in a fight: a superintelligent ninja cat with laser nunchucks and body armor, or a dog?

My cat beats my dog in fights all the time, and he doesn't have any of that training or gear.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#307

Post by pmontra »

mercer wrote: post #150
pmontra wrote: post #145

Earlier quoted context omitted.

Every ORM is a transpiler from a language to SQL. ActiveRecord transpiles from Ruby to SQL. Django ORM and SQLAlchemy are popular ones to write SQL in Python. Ecto is from Elixir. Every language has its own transpilers. Not all of them are easy to use and they are much harder to use than SQL for complex queries, maybe even impossible to build them. Frankly I'm a little fed up to have to learn a new way of generating…

I've heard good things about Ecto's approach (Elixir). Would you say it's at least better than many other ORM's, or is it similar enough in regards to drawbacks/advantages compared SQL that this 'better' doesn't matter much?

Ecto is well engineered but it's an overkill for simple queries and it makes them harder than it should. It's ok for average sized queries (maybe composed over a pipeline). It's useless for very complex queries, but every ORM can't do them. Think UNION, HAVING, etc. I write them in SQL and then handle the resultset, which is not in the transpiler. The transpiler is a subset of the functionality of an ORM.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#308

Post by degyves »

jbergens wrote: post #289
tn_ wrote: post #221

Earlier quoted context omitted.

There have been a ton of start-ups that I've talked to / been a part of who used MongoDb thinking their company is going to exponentially explode in MAU and they think they'll save themselves the scaling troubles by using NoSQL. What ends up happening is the codebase gets too gnarly when they try to start doing complex analysis. SQL is appropriate for like 95% of companies. A lot of these places I'm referencing end u…

Complex analytics should probably be done in a db separate from the production db anyway. The analytical db could be an RDBMS, a datacube or something else.

That's when having mongodb+RDBMS with the added complexity is really justified. On common cases, just drop mongodb and use the RDBMS for everything.

Re: Why SQL is beating NoSQL, and what this means for the future of data

#310

Post by bobdc »

smarx007 wrote: post #199

I am really surprised that while everyone talks about SQL vs NoSQL, nobody has mentioned RDF as a model and SPARQL as a language. Graph-like structures based on triples allow the data relations to be represented properly (relational properties), while not limiting the data structure. This is not a shiny hot thing, but instead something developed over many years by (mostly) academics. Take a look at the tutorial: http…

SPARQL is also a standard from a well-known standards body with plenty of open source and commercial implementations, and it's quite popular at many commercial enterprises: http://www.sparql.club
Post reply on HN