Live data from Hacker News

Show HN: Csql – Python lib for composeable SQL queries

github.com

21–30 of 41 posts

Re: Show HN: Csql – Python lib for composeable SQL queries

#21
post #13

https://ibis-project.org/ is another great alternative. It provides a fluent/linq style api but doesn't abstract the SQL away too much.

This looks more like what I was expecting from the title: a jOOQ (Java eDSL for SQL with type safety by generating classes based on the schema) like library for Python.

Interesting.

Re: Show HN: Csql – Python lib for composeable SQL queries

#23
post #5

Looks like theyre headed in the right direction. ORMs are tricky to get right, when I wrote ClojureQL I looked more towards combinatorics than SQL: https://clojureql.sabrecms.com/en/examples

Your example of multiple sorts is problematic. > SELECT * FROM (SELECT users.* FROM users ORDER BY users.id asc) ORDER BY users.id desc A sort should never (for some value of never) appear in a subquery because it's meaningless; it can't affect the result. In tsql it's explicitly checked for and reported. Not having a go, just fyi. Edit: now I'm more confused by that. The bracketed subquery doesn't have a name but it…

You're absolutely right, the inner sort example makes no sense. I hope I'll have to time to work on this again in the near future. I use CQL every day but there's still a lot I'd like to do with it.

Re: Show HN: Csql – Python lib for composeable SQL queries

#24
post #13

https://ibis-project.org/ is another great alternative. It provides a fluent/linq style api but doesn't abstract the SQL away too much.

> Ibis uses SQLAlchemy internally, but aims to provide a friendlier syntax for analytics code.

What about SQLAlchemy makes it unfriendly for analytics code? The last time I looked at it, SQLAlchemy Core had a pretty good fluent interface for writing SQL queries

Re: Show HN: Csql – Python lib for composeable SQL queries

#25
post #4

I think this is really neat. It’s seems a bit more flexible than the composition approach of SQLAlchemy, too. I also like that the unit of composition is the CTE.

Can you say more about that? I though SQLAlchemy core goal was exactly that. When does it fails, compared to Csql?

My desire was to be able write real SQL directly in the dialect of my database.

SQLAlchemy can't give me that, instead I'd need to learn a special SQLAlchemy-specific query builder syntax that probably won't support all the analytical functions I want to use anyway. It's really a whole different beast to csql.

Re: Show HN: Csql – Python lib for composeable SQL queries

#26
This project seems nice, but there's some detail I have trouble wrapping my head around: why is a template string represented as a lambda containing an f-string, instead of as a plain string? Namely, wouldn't the code below suffice?

    Q("""select 1 from {otherQuery}""")
What does the following code enable that the code above cannot do?

    Q(lambda: f"""select 1 from {otherQuery}""")

Re: Show HN: Csql – Python lib for composeable SQL queries

#27

This project seems nice, but there's some detail I have trouble wrapping my head around: why is a template string represented as a lambda containing an f-string, instead of as a plain string? Namely, wouldn't the code below suffice? Q("""select 1 from {otherQuery}""") What does the following code enable that the code above cannot do? Q(lambda: f"""select 1 from {otherQuery}""")

It's impossible to hook into Python's string interpolation system to the degree required for the first to work. JS and Julia can do it, e.g. in JS (with typescript annotations) it'd just be a matter of defining

   function Q(stringBits: string[]): Query {
      for (bit in stringBits) {
         if (bit is string) {
            add to sql
         } else if (bit is Query) {
            add bit to dependencies
            add bit.name to sql
         }
      }
   }
But this cannot be done currently in Python (see PEP-501), so I'm forcing the user to pass a lambda, which I can get the AST of, with which I can implement the machinery to do the above.

Suggestions for improvements are welcome!

Re: Show HN: Csql – Python lib for composeable SQL queries

#28

As an analyst, I often am faced with a choice between - write a giant unmaintainable SQL query, or - pull everything to my PC and use pandas, to take advantage of its ability to build up results piece by piece. I wrote this library to try and enable that piece-by-piece development approach with SQL queries, without resorting to the mental overhead of full on query builders like SQLAlchemy or Linq. Seeking feedback -…

I like what you've done here. I've personally gone down the route of using SQLAlchemy to build queries. The primary reason is that SQLAlchemy lets me build up reuseable SQL elements, to encapsulate business logic, like metrics and custom dimensions in Python. In particular SQLAlchemy's hybrid expressions, that allow you to tie an SQL expression to a ORM model are super useful for doing this sort of thing. There's also first class support for CTEs, window functions and other advanced SQL features. For me there's just something "dirty" about using raw SQL in code, or even templating it, even with security concerns like injection attacks put aside.

I second the recommendation for dbt. Especially if you're following an ELT architecture where you load your data into a data warehouse and then want to transform it. However dbt is more useful for transforming existing data, or aggregating data in batches. It's not a tool for generating SQL expressions on-the-fly like your library would allow you to do.

Re: Show HN: Csql – Python lib for composeable SQL queries

#29

This project seems nice, but there's some detail I have trouble wrapping my head around: why is a template string represented as a lambda containing an f-string, instead of as a plain string? Namely, wouldn't the code below suffice? Q("""select 1 from {otherQuery}""") What does the following code enable that the code above cannot do? Q(lambda: f"""select 1 from {otherQuery}""")

It's impossible to hook into Python's string interpolation system to the degree required for the first to work. JS and Julia can do it, e.g. in JS (with typescript annotations) it'd just be a matter of defining function Q(stringBits: string[]): Query { for (bit in stringBits) { if (bit is string) { add to sql } else if (bit is Query) { add bit to dependencies add bit.name to sql } } } But this cannot be done currentl…

[deleted]

Re: Show HN: Csql – Python lib for composeable SQL queries

#30

As an analyst, I often am faced with a choice between - write a giant unmaintainable SQL query, or - pull everything to my PC and use pandas, to take advantage of its ability to build up results piece by piece. I wrote this library to try and enable that piece-by-piece development approach with SQL queries, without resorting to the mental overhead of full on query builders like SQLAlchemy or Linq. Seeking feedback -…

Interesting approach! I've been working on it from the other angle: having pandas code generate SQL. If you're interested in checking it out, happy to try and show how it would generate the query in your readme!

https://github.com/machow/siuba

One thing I was wondering about the gnarly ast stuff you mention, what about operator overloading? E.g. Q("Select a from" + subquery + "where a < 1")

Post reply on HN