Live data from Hacker News

Why your query language should be explicit

blog.hiphipjorge.com

11–20 of 47 posts

Re: Why your query language should be explicit

#11
> But RethinkDB won't optimize the query for you or tell you it's wrong. It'll just run it. It's up to the developer to understand what's going on and optimize accordingly. This might sound like a huge deal, but the simplicity of the language makes it easy to spot these inefficiencies and fix them accordingly.

That's the kind of thing that's only true until it's not. The bigger a query gets, the less likely that you'll be able to eyeball it to see what you screwed up.

Re: Why your query language should be explicit

#12
He says he's been using the explicit query language for a couple of months. On that timescale, I can see how it might still feel okay. But as the months become years and your code grows in complexity to meet an ever lengthening requirements list, it should become apparent why SQL is far superior.

Need to add an index to speed up some queries?

In SQL, you add the index and you're done.

In an explicit query language, you add the index and... oops, that doesn't do anything. You've got to go back and inspect every single query, anywhere in your program, that could potentially benefit from that index, to see whether it actually will, and if so, modify it by hand.

Switching from SQL to an explicit query language converts certain types of programming effort from O(N) to O(N^2). This is one of the reasons SQL was invented in the first place.

Re: Why your query language should be explicit

#13
post #5

I don't agree with this conclusion. I've worked with Postgres, MySQL and Oracle, and found that it is important to have a good understanding of the possible execution plans for a query. I will sometimes construct a query very carefully to achieve a particular plan. And when things go wrong, I run EXPLAIN PLAN, examine statistics, etc., and tweak my query to do what I want. You really have to do that to obtain good pe…

When you do need to be very careful about writing a query, the ORM adds a layer of complexity -- not only do you need to control the SQL, but you now need to ensure that your ORM can produce that SQL

A good ORM will let you run manually written SQL queries and map the results to objects. For example in Django: https://docs.djangoproject.com/en/1.8/topics/db/sql/

Re: Why your query language should be explicit

#14
post #8

This is a great way to spin not having a query planner as a feature, but I'm glad to have one every day that I write and compose semantic bits of SQL that can and should have different execution plans depending on the context in which they're evaluated.

The whole article is like one long spin job. "Explicit" is somehow the happy medium between the query engine putting together a plan based on data statistics and... I'm not even sure what space this claims to be the middle of. Keeping the examples simple enough so that there is an unambiguously right order of operations - filter before you sort, that's genius! There's no way your granddad's RDBMS could have figured that one out! What if the optimal order depends on the values, or changes over time? Just rewrite and recompile?

Re: Why your query language should be explicit

#16
post #5

I don't agree with this conclusion. I've worked with Postgres, MySQL and Oracle, and found that it is important to have a good understanding of the possible execution plans for a query. I will sometimes construct a query very carefully to achieve a particular plan. And when things go wrong, I run EXPLAIN PLAN, examine statistics, etc., and tweak my query to do what I want. You really have to do that to obtain good pe…

When you do need to be very careful about writing a query, the ORM adds a layer of complexity -- not only do you need to control the SQL, but you now need to ensure that your ORM can produce that SQL A good ORM will let you run manually written SQL queries and map the results to objects. For example in Django: https://docs.djangoproject.com/en/1.8/topics/db/sql/

Or even better, an ORM can be layered on top of an SQL abstraction that maps directly to what the database understands: http://docs.sqlalchemy.org/en/rel_1_0/core/tutorial.html

It then becomes easy to go one level below the ORM without giving up all the nice things.

Re: Why your query language should be explicit

#17
Besides the fact that I don't agree with his conclusion - he ignores any of the advantages of an implicit language and therefore the article suffers from the "everybody must be stupid" syndrome...

"Jorge Silva, Dev Evangelist @ RethinkDB. Full-Stack JavaScript Developer."

That final sentence makes me cringe slightly.

Re: Why your query language should be explicit

#18
post #5

I don't agree with this conclusion. I've worked with Postgres, MySQL and Oracle, and found that it is important to have a good understanding of the possible execution plans for a query. I will sometimes construct a query very carefully to achieve a particular plan. And when things go wrong, I run EXPLAIN PLAN, examine statistics, etc., and tweak my query to do what I want. You really have to do that to obtain good pe…

The real trouble comes when your carefully manually tuned version of the query which works optimally for the data and loading and database version you have on date X, is still running on date X + 5 years. When the DB engine has upgraded its optimizer, and the data patterns have changed, and the usage levels are very different. And the original, unoptimized, declarative query would now be able to run faster.

Re: Why your query language should be explicit

#19
post #17

Besides the fact that I don't agree with his conclusion - he ignores any of the advantages of an implicit language and therefore the article suffers from the "everybody must be stupid" syndrome... "Jorge Silva, Dev Evangelist @ RethinkDB. Full-Stack JavaScript Developer." That final sentence makes me cringe slightly.

"There are two types of people in this world. Those who agree with me and those that are stupid."

Programming has a lot of areas where it is split down the middle into two camps (usually two, although occasionally more...)

Not equally, mind you, but two very strongly opinionated camps.

Semicolons come to mind.

Re: Why your query language should be explicit

#20
post #2

The biggest problem with a procedural/explicit query is a dynamic system. Without a query planner, you don't have the luxury of a system rewriting your queries. When Table A ~ Table B, but then Table B >> Table A, your queries are going to be radically suboptimal. Of course, if you're never joining, maybe that's not such a big problem, but you'd have the same issue of a specific range in your table grows disproportio…

> The biggest problem with a procedural/explicit query is a dynamic system. Without a query planner, you don't have the luxury of a system rewriting your queries. When Table A ~ Table B, but then Table B >> Table A, your queries are going to be radically suboptimal.

This was exactly my thinking. Worse yet, a good query optimizer will rewrite your queries as your data changes. So when tables grow or shrink, indexes are added or removed, and even as the distribution of the data within each table changes, you'll get execution plans that roll with the punches.

If you've explicitly codified your execution plan, you don't get any of these advantages. Any changes, and you've got to re-write all your queries.

So many of these NoSQL databases leave me shaking my head. Maybe I just don't get it, but I feel like once you understand how RA, DRC, TRC, and SQL are equivalent, you'll never want to write relational algebra again.

Cost models aren't perfect but they're pretty good. System R-esque systems can evaluate like, A LOT of join orderings. You don't wanna just throw that power out on the whim that the developer might know better.

Post reply on HN