A Critique of SQL, 40 Years Later
carlineng.com
A Critique of SQL, 40 Years Later
1–10 of 260 posts
Re: A Critique of SQL, 40 Years Later
#2"Despite the improvements listed above, in a 2014 interview, CJ Date said “we didn’t realize how truly awful SQL was or would turn out to be (note that it’s much worse now than it was then, though it was pretty bad right from the outset).” This quote leaves me wondering – if Date himself were to write an updated critique, what would it look like? My best guess is most of his criticism would revolve around further departures of SQL from the relational model, but specific examples escape me."
I am not sure how the author has missed the copious amount of material that Date (& Darwin) wrote on this topic? The obvious one being the Third Manifesto (whole book available here [1] since it's out of print now). But even "Database in Depth" [2] has extensive discussion about this despite being a less "opinionated" and "ranty" book (highly recommended BTW)
Yes, not explicitly updated since 2014, but there's really nothing new to say since then?
Date is definitely concerned about the departures from the relational model in SQL. But it's (EDIT) not just about purism. He's also pretty in general peaved off at SQL for similar concerns that the author presents here about syntax and expressiveness. Third Manifesto's "tutorial D" mandates the addition of a richer type system, including higher level types that match on relation constraints, and "operators" (similar to OO methods) on those types; basically a kind of predicate dispatch.
I imagine he's pretty sick of being ignored.
[1] https://www.dcs.warwick.ac.uk/~hugh/TTM/DTATRM.pdf
[2] https://www.oreilly.com/library/view/database-in-depth/05961...
Re: A Critique of SQL, 40 Years Later
#3SQL is having somewhat of a moment in the bigdata world, thanks in part to 'modern datastack' and new age datawarehouses like snowflake,bigquery.
However there are a lot of pushback from 'traditional' dataengineers who were trained on spark/scala. Its bit of hardsell to go from a highly typed language to a free for all text based logic.
I think the following is needed for sql to be finally accepted as 'serious' contender.
create compiled sql language ( not pandas)
1. that compiles to sql and addresses some of the issues bought up in the post like nested aggregations.
2. make code reusable. Eg: apply year over year growth to a table that has the requisite columns. Compiler should check this in ide.
3. make materializations first class concept in the language. No seperate dbt layer.
4. crate a way to package and distribute libraries that you can import into your project .
5. a unit testing framework that makes it easy to test the logic without having to setup test tables in the database.
Re: A Critique of SQL, 40 Years Later
#4While all the criticism is probably correct, and SQL definitely shows its age, it's still very much good enough for pretty much all its current applications. Also, SQL's basics are very easy to learn. These aspects make it very hard to imagine a language that might gain enough traction to actually replace SQL in a foreseeable future.
Re: A Critique of SQL, 40 Years Later
#5SQL is having somewhat of a moment in the bigdata world, thanks in part to 'modern datastack' and new age datawarehouses like snowflake,bigquery. However there are a lot of pushback from 'traditional' dataengineers who were trained on spark/scala. Its bit of hardsell to go from a highly typed language to a free for all text based logic. I think the following is needed for sql to be finally accepted as 'serious' conte…
I think these are great suggestions.
It seems like you're suggesting that someone could design a functional-style programming language that compiles to SQL.
2 & 3 are my biggest pain points. I can't just extract functions like I can with a regular programming language. Instead, SQL queries get increasingly complex with no great tools to manage that.
For 3, products like https://materialize.com/ look interesting for being able to create derived materialized views that can efficiently be kept up to date.
Re: A Critique of SQL, 40 Years Later
#6I always appreciate blog posts like this, there are obviously cases where SQL shines, and in part I think the dataframe abstraction helps with filling a lot of the missing pieces that SQL doesn't handle so well (composability, debuggability, interactivity, etc.)
Even pandas (with all its faults) is more flexible as a language than SQL[1]. I'm of the opinion that there's a better together story in the end, but I guess we will see.
[1] https://ponder.io/pandas-vs-sql-food-court-michelin-style-re...
Re: A Critique of SQL, 40 Years Later
#7While all the criticism is probably correct, and SQL definitely shows its age, it's still very much good enough for pretty much all its current applications. Also, SQL's basics are very easy to learn. These aspects make it very hard to imagine a language that might gain enough traction to actually replace SQL in a foreseeable future.
Many have tried. Most are hobby projects. None have had the full force of a real production ready DBMS behind them.
And really, projects like this are fighting the general ignorance the industry as a whole has about what a relational database actually is. It's better since the NoSQL wave crested and we stopped hearing stupid shit like "my data isn't tabular and doesn't fit in a schema", but there's still a preponderance of people who haven't stopped to learn the lessons that Date and Codd tried to teach decades ago, and constantly reach for graph/network/hierarchical databases without understanding why we originally tried to move beyond those back in the 70s.
Unfortunately because of this most attempts to do "better than SQL" end up not working from first principles and look like a dogs breakfast of ill-formed "that'd be neat" ideas.
And, yeah, as I said the "pure relational" projects that are out there are usually either academic projects or hobby. (Yes I've had a few of my own)
I think there might be hope in the Datalog-type systems that have emerged recently.
Re: A Critique of SQL, 40 Years Later
#8The only thing I would really blame solely on SQL is that UPDATE and DELETE statements don't require you to specify a limit.
I have seen many times in my career where a rogue delete just truncates a table, a simple statement of intent (e.g. LIMIT 1) would tell the query planner that if it is about update/delete more than 1 row, it should error. In fact MySQL actually returns a warning if you do this.
TRUNCATE clearly states your intention to delete everything, but DELETE by default also deletes everything.
There were also definitely some bad paradigms invented as a result of SQL (e.g. all business logic lives in the database as stored procedures or stored functions), but that was bound to happen whenever a shiny new tool comes out and engineers want to use it.
Re: A Critique of SQL, 40 Years Later
#9SQL is having somewhat of a moment in the bigdata world, thanks in part to 'modern datastack' and new age datawarehouses like snowflake,bigquery. However there are a lot of pushback from 'traditional' dataengineers who were trained on spark/scala. Its bit of hardsell to go from a highly typed language to a free for all text based logic. I think the following is needed for sql to be finally accepted as 'serious' conte…
> I think the following is needed for sql to be finally accepted as 'serious' contender.
Whatever way you slice it, SQL is one of the most used languages today [0].
[0] https://spectrum.ieee.org/top-programming-languages-2022
Re: A Critique of SQL, 40 Years Later
#10The only thing that affects my daily life with SQL is that FROM should be before the SELECT keyword. This would _greatly_ improve type-ahead support in SQL IDEs.
Nothing is perfect, but that is really the main beef. Another commentor already nailed having a LIMIT WITH ERROR clause to be specified on UPDATE,DELETE statements and explicitly throw an error otherwise.
SQL is on of my favorite tools to use and I don't see it getting replaced by anything any time soon.