Live data from Hacker News

I don't need your query language

antonz.org

211–220 of 304 posts

Re: I don't need your query language

#211
post #199

We write programs with Python, Java, Rust, Javascript and so on. Yet we use a very different language, eg. SQL, to query and modify data. Why? Why don't we use eg. Python as well? SQL is different from other languages: it is declarative, meaning it doesn't dictate how to do it, but what the response should be. Maybe that's the reason? But if that's the case, why are declarative languages not more popular? SQL and oth…

The problem with procedural is that it doesn’t optimize very well. The fastest way to get your data for a large number of random queries often depends on the data size, the available indexes, but is also influenced by changes in data size, etc. What is fast for a small dataset might be slow for a larger dataset. The right algorithm also depends on your filters and caching. It’s almost impossible to write procedural q…

I have written 4 query optimizers: HypersonicSQL, H2 database, PointBase Micro, and Apache Jackrabbit Oak. I know procedural language don't have such optimizers. But I argue that you don't always need them. I argue that the database engine query optimizer is often more a risk than a help. I have seen missing indexes far too often. SQL doesn't require that the programmer thinks how the data is accessed, so the result is that too many programmers don't think about it, and so don't add indexes, request too much data, and so on. The SQL statements work fine with small (development) data sets, so the same statements are used with production databases, and you run into problems too late.

I don't think it's hard to write procedural queries that are always fast. You write a loop or map/filter/collect method. You just explicitly need to mention which index to use, is all.

Re: I don't need your query language

#212

Earlier quoted context omitted.

I guess I'm not in the C# community. `let` is pretty annoying to reproduce with extension methods. Also, when you use multiple `from` clauses, you get access to all scopes, whereas with `SelectMany` you only get the parameters of your current lambda.

> you get access to all scopes Read: "you create new heap-allocated closures which wreck your Linq expression's runtime performance" Or: "you create Linq queries that cannot be translated into SQL"

It was all in memory. To the extent that it's a performance tradeoff, it might be one that's worth making depending on the use.

Re: I don't need your query language

#213

Earlier quoted context omitted.

> the _only_ place where it offers a readability improvement over ext-methods is using `join` No love for let? from item in items let frob = Expenseive(item.P1) where frob > 3 selec new { frob, item }

That can be done with ValueTuples instead of Anonymous Types: `items.Select( i => ( i, frob: Expensive( i.P1 ) ) ).Where( t => t.frob > 3 );` ...and ValueTuples are superior to Anonymous Types because you can actually return them from a function or use them as parameters - whereas Anonymous Types cannot cross method-call boundaries (excepting using generics for pass-through). Anonymous Types in C# were a massive mist…

The tuple might be gone by the time the scope ends. I'm aware it's possible to reproduce the behavior without let. The code just looks worse.

Re: I don't need your query language

#214
post #188

SQL is only ever as good as the schema relative to the business or problem domain. The focus on the syntax of the language was always a mystery to me. It's a domain- specific language. It's up to you to make it not suck. If you are forced to work with a schema that is poorly-aligned with the logical reality it intends to represent, you would definitely walk away with a bad taste in your mouth. Hacking around bad norm…

The "domain" in the DSL of SQL is "relational data" not "your business domain"

What does this "relational data" (hopefully) represent?

Re: I don't need your query language

#215

SQL does have a significant drawback w.r.t. how databases are used today (imo): a SELECT query can only return a single resultset of uniform tuples: if you want to query a database for hetereogenous types with differing multiplicity (i.e. an object-graph) then you either have to use multiple SELECT queries for each object-class - or use JOINs which will result in the Cartesian Explosion problem[1] which also results…

This is no longer true in database that have JSON support (which is most of them these days). You can aggregate the result of a subselect into a single column (the underling data doesn’t have to be stored as JSON, you can convert as part of the query)

> You can aggregate the result of a subselect into a single column (the underling data doesn’t have to be stored as JSON, you can convert as part of the query)

For anyone (like me) who is not quite able to visualize this, here is an example:

    SELECT json_agg(trips)
    FROM (
        SELECT 
            json_agg(
                json_build_object(
                    'recorded_at', created_at, 
                    'latitude', latitude, 
                    'longitude', longitude
                )
            ) as trips
        FROM data_tracks
        GROUP by trip_log_id
    )s
From StackOverflow user S-man https://stackoverflow.com/a/53087015

Re: I don't need your query language

#216

I don’t really find the examples convincing. Like, I get that sql could maybe be written in a slightly less horrid way but I would prefer something a lot less horrid. I think I’m much more motivated by analytics queries than the kinds of thing in this example though. I find sql is poorly suited in this case because it is verbose and written backwards, and often requires many layers of subqueries. That said, one can u…

The author’s second example is extremely unconvincing for me. Why would I want to be forced to use a SELECT expression to calculate a mean? Relational algebra is a great abstraction. SQL, however, seems to be poorly thought out and ad hoc. It’s just the first implementation of a relational algebra language that worked. But why should we be stuck with it forever?

Re: I don't need your query language

#217
post #205

I don’t really find the examples convincing. Like, I get that sql could maybe be written in a slightly less horrid way but I would prefer something a lot less horrid. I think I’m much more motivated by analytics queries than the kinds of thing in this example though. I find sql is poorly suited in this case because it is verbose and written backwards, and often requires many layers of subqueries. That said, one can u…

Why do you need the outermost query?

I think I had originally written something like select min(size) min, max(size) max, max-min range, but that didn’t work as the newly introduced names ‘weren’t in scope’ and I didn’t want to type those aggregations out again. You’re right that it could have been avoided.

Re: I don't need your query language

#218
post #60

I don’t really find the examples convincing. Like, I get that sql could maybe be written in a slightly less horrid way but I would prefer something a lot less horrid. I think I’m much more motivated by analytics queries than the kinds of thing in this example though. I find sql is poorly suited in this case because it is verbose and written backwards, and often requires many layers of subqueries. That said, one can u…

I think the problem here is with the query, not the language. You can immediately improve its maintainability and readability by using CTEs. https://antonz.org/cte/

Maintainability is not relevant to ad-hoc analytics queries. That may be dealt with once the correct query has been determined from sufficient iteration.

Re: I don't need your query language

#219

> Here is another common argument: SQL was designed with 1970s businessmen in mind, and it shows. That is a funny way of looking at it. I see that SQL is based on the work of a computer scientist vs DSLs being made by hobbyists, and it shows.

[deleted]

Re: I don't need your query language

#220
post #15

I think of SQL as one of the few good things we have in software development, so like the author I consider it best to try to do as much in SQL as possible. It's not too uncommon I run into code in other languages where I just don't understand what it does, or to write code myself that behaves in ways that surprise me. That almost never happens in SQL. Even a big hairball of a query just takes time to figure out (unl…

I tell new developers that SQL is one of those few things in our field you get to keep forever. That JavaScript framework that takes a year to understand will no longer be used in 7 years. SQL is going to be here forever and learning it is useful your whole career. Other common entries on this list of forever tools: regular expressions, emacs, bash/shell scripting, excel, probably more I’m forgetting. Devs always pus…

> how often it’s the tool the business speaks and feels comfortable giving feedback on technical questions in.

Yes, but as often as not business people end up using Excel as a glorified text editor with built-in tabular formatting. Some of the spreadsheets I've been handed by project stakeholders would make 90s-era pre-CSS HTML using tables for formatting look clean and simple by comparison.

After writing applications in "4GL" sql tools early in my career, I've spent the entirety of the rest of my career trying to avoid writing SQL or having anything to do with the inner guts of those gigantic global variables known as relational databases.

Post reply on HN