Live data from Hacker News

PRQL – A proposal for a better SQL

github.com

51–60 of 302 posts

Re: PRQL – A proposal for a better SQL

#51

Very cool! A couple questions/suggestions off the top of my head: 1. Did you consider using a keyword like `let` for column declarations, e.g. `let gross_salary = salary + payroll_tax` instead of just `gross_salary = salary + payroll_tax`? It's nice to be able to scan for keywords along the left side of the window, even if it's a bit more verbose. 2. How does it handle the pattern where you create two moderately comp…

> 2. How does it handle the pattern where you create two moderately complex CTEs or subqueries (maybe aggregated to different levels of granularity) and then join them to each other? I always found that pattern awkward to deal with in dplyr - you have to either assign one of the "subquery" results to a separate dataframe or parenthesize that logic in the middle of a bigger pipeline. Maybe table-returning functions would be a clean way to handle this?

I don't have an example on the Readme, but I was thinking of something like (toy example):

  table newest_employees = (
    from employees
    sort tenure
    take 50
  )
  
  from newest_employees
  join salary [id]
  select [name, salary]

Or were you thinking something more sophisticated? I'm keen to get difficult examples!

Edit: formatting

Re: PRQL – A proposal for a better SQL

#52
post #11

First, kudos because it takes courage to take on SQL in this way. Second, this kind of reversed SQL (filter-first, select-last) is much easier to reason about than the original and keep in mind that I prefer to code complex queries in SQL than to build or translate them in the ORM of the project I'm working on. Maybe a transpiler is an inevitable first step but I think that any SQL replacement should be itself the ta…

> Second, this kind of reversed SQL (filter-first, select-last) is much easier to reason about than the original

Given that SQL clauses tend to be unambiguously terminated by the start of the next clause or the end of the statement, it surprises me that no engine has gone to accepting otherwise standard(-ish, as much as real DB vendor dialects are) SQL but without a mandated order of clauses.

And then combine that with dev tools that allow easy rearrangement of clauses, perhaps based on configured preferences so that you don’t even see the original if its not your preferred order, so that “Bob likes old-school SELECT FROM WHERE GROUP BY and Alice likes FROM WHERE GROUP BY SELECT” isn’t a problem.

Re: PRQL – A proposal for a better SQL

#53
shakti / K / kdb+ implements "real SQL", which is concise but readable, and could give you a few ideas. Here's a copy-paste from https://shakti.sh/ under document/sql.d (cannot deep link, unfortunately). The most most magical aspects are automatic joins - both left joins and "foreign key chase" joins. The fk-chase joins, in particular, should be part of every query language, and can possibly be added in a backward compatible way to existing SQL implementations.

example: TPC-H National Market Share Query 8 http://www.qdpma.com/tpch/TPCH100_Query_plans.html what market share does supplier.nation BRAZIL have by order.year for order.customer.nation.region AMERICA and part.type STEEL?

real: select revenue avg supplier.nation=`BRAZIL by order.year from t where order.customer.nation.region=`AMERICA, part.type=`STEEL

ansi: select o_year,sum(case when nation = 'BRAZIL' then revenue else 0 end) / sum(revenue) as mkt_share from ( select extract(year from o_orderdate) as o_year, revenue, n2.n_name as nation from t,part,supplier,orders,customer,nation n1,nation n2,region where p_partkey = l_partkey and s_suppkey = l_suppkey and l_orderkey = o_orderkey and o_custkey = c_custkey and c_nationkey = n1.n_nationkey and n1.n_regionkey = r_regionkey and r_name = 'AMERICA' and s_nationkey = n2.n_nationkey and o_orderdate between date '1995-01-01' and date '1996-12-31' and p_type = 'STEEL') as all_nations group by o_year order by o_year;

Re: PRQL – A proposal for a better SQL

#54
post #49

Nice. Why OCaml though? I think using a more conventional language to construct queries could yield more adoption. It also seems that ORMs kinda of exist to tackle a similar issue, at least in part.

I don’t see anything dealing with OCaml here, other than it being listed as an inspiration.

Re: PRQL – A proposal for a better SQL

#55
post #48

I like the flow direction compared to standard SQL. SQL is supposed to read like a sentence I suppose but I have many times looked at it and really wanted things to be in a more logical order. My main suggestion would be to be a bit less terse and introduce a bit more firm formatting. I'm not a huge fan of the term "split" and feel like jazzing that up to "split over" or even just reviving "group by" would improve re…

This is great feedback, and I agree with you re de-prioritizing terseness.

And I agree with you on both the assignments and `split` being a bit awkward. Kusto just uses `by`, WDYT?

Re: PRQL – A proposal for a better SQL

#56

SPARQL. Representing human information in relational tables goes against how people actually think and use information. We humans think in tremendous numbers of nested hierarchies, and recursive hierarchy traversal is a nightmare in relational databases. A graph is the structure for data that works best, is most efficient, and actually reflects how things are connected in our brains.

I'm a big fan of SPARQL, but the one thing that would concern me about trying to use it outside of the SemWeb context is simply that it assumes data is stored in triples. Legacy databases by and large are not, so you need an adapter to bridge the representations. And while I know some exist, I haven't really used them and am not sure about the performance impact.

Re: PRQL – A proposal for a better SQL

#57

Very cool! A couple questions/suggestions off the top of my head: 1. Did you consider using a keyword like `let` for column declarations, e.g. `let gross_salary = salary + payroll_tax` instead of just `gross_salary = salary + payroll_tax`? It's nice to be able to scan for keywords along the left side of the window, even if it's a bit more verbose. 2. How does it handle the pattern where you create two moderately comp…

> 2. How does it handle the pattern where you create two moderately complex CTEs or subqueries (maybe aggregated to different levels of granularity) and then join them to each other? I always found that pattern awkward to deal with in dplyr - you have to either assign one of the "subquery" results to a separate dataframe or parenthesize that logic in the middle of a bigger pipeline. Maybe table-returning functions wo…

When you add in the ability to reference different tables like that to the piping syntax, it starts to remind me of the M query language: https://docs.microsoft.com/en-us/powerquery-m/quick-tour-of-...

There, each variable can be referenced by downstream steps. Generally, the prior step is referenced. Without table variables, your language implicitly pipes the most recent one. With table references, you can explicitly pipe any prior one. That way, you can reference multiple prior steps for a join step.

I haven't thought through that fully, so there may be gotchas in compiling such an approach down to SQL, but you can already do something similar in SQL CTEs anyway, so it should probably work.

Re: PRQL – A proposal for a better SQL

#58
This is another in a series of these kinds of proposals that look excellent on first glance for perhaps the 75% case but start getting syntactically messy when I want to customize the resultset returned.

On the surface, they're always neat but when you start to dig into how you'd implement something in an RDBMS, it begins to fall apart.

Let's look at the example syntax:

    from employees
    filter country = "USA"                         # Each line transforms the previous result.
    gross_salary = salary + payroll_tax            # This _adds_ a column / variable.
    gross_cost   = gross_salary + healthcare_cost  # Variable can use other variables.
    filter gross_cost > 0
    aggregate split:[title, country] [             # Split are the columns to group by.
        average salary,                            # These are the calcs to run on the groups.
        sum     salary,
        average gross_salary,
        sum     gross_salary,
        average gross_cost,
        sum     gross_cost,
        count,
    ]
    sort sum_gross_cost                            # Uses the auto-generated column name.
    filter count > 200
    take 20

Where in here is it clearly stated which fields are returned? In the original SQL it's right up front but here it's buried into the "aggregate" function, and I'm not clear that this isn't an oversight.

Another example that speaks to the "how do I implement this" side of the equation:

    from employees
    filter country = "USA"                         # Each line transforms the previous result.
    gross_salary = salary + payroll_tax            # This _adds_ a column / variable.
    gross_cost   = gross_salary + healthcare_cost  # Variable can use other variables.
    filter gross_cost > 0
Does this mean that the database must scan all records of the employee table in order to return the result before moving to the next step in the query? Must I index all fields? If not, how does a query planner prepare for this scenario?

The major tradeoff you make in most ORMs is exactly this: You lose out on being able to be explicit about how many queries are sent to the DB (and in many cases how efficient those queries are). Now this would become a language feature? What do I gain for that loss?

I'm not saying that SQL Syntax is perfect; far from it. I'm not seeing how this is an improvement.

I think if you want traction though, a proof of concept using an existing RDBMS would go a long way into providing evidence that this will work and is sufficiently thought out to deal with even the basics of what existing SQL databases have to. Query planning is hard, especially if you want it to be fast.

Post reply on HN