The initial release of sqlite was in 2000. Yes, well after GNU-style command line options existed but not by much.
22 years is a long time to deprecate bad command line arguments.
Sqlite team idolizes backwards compatibility. Which is admirable but at the same time it means we are not getting strong static typing of tables before 2035.
Shame. Even bigger shame is that nobody else is taking the torch.
I'm looking through this guys website for 'today I learned' and at first I'm impressed by how many of them there are. But then I start thinking: when you're trying to solve a problem you search for a lot of data. None of his posts are attributed. He's getting all his information from somewhere and then he goes and posts these articles just ripping off other sources. I can understand when its based on your original wo…
Remember what this is.
This is the author's notebook, which happens to be public. It's okay.
22 years is a long time to deprecate bad command line arguments.
Sqlite team idolizes backwards compatibility. Which is admirable but at the same time it means we are not getting strong static typing of tables before 2035. Shame. Even bigger shame is that nobody else is taking the torch.
The documentation should focus on the 'best' way to accomplish the goal, reducing the number of people who encounter and repeat the old way. Doing anything else is a self-fulfilling prophecy. Yeah of course people are still using the old way, if some of them didn't start using it until after you added the new way.
sqlite3 :memory: -cmd '.mode csv' ... It should be a war crime for programs in 2022 to use non-UNIX/non-GNU style command line options. Add it to the Rome Statute's Article 7 list of crimes against humanity. Full blown tribunal at The Hague presided over by the international criminal court. Punishable by having to use Visual Basic 3.0 for all programming for the rest of their life.
Those -cmd are actually unnecessary, you can do
sqlite3 :memory: '.mode csv' '.import taxi.csv taxi' \
'SELECT passenger_count, COUNT(*), AVG(total_amount) FROM taxi GROUP BY passenger_count'
22 years is a long time to deprecate bad command line arguments.
Sqlite team idolizes backwards compatibility. Which is admirable but at the same time it means we are not getting strong static typing of tables before 2035. Shame. Even bigger shame is that nobody else is taking the torch.
I'm genuinely curious about the use case of this. whenever i have CSVs I either import then into R/python of they're small or some DBMS first if they're large. can somebody tell me what niche this is filling?
Sqlite team idolizes backwards compatibility. Which is admirable but at the same time it means we are not getting strong static typing of tables before 2035. Shame. Even bigger shame is that nobody else is taking the torch.
They added strong typing of tables a few months ago as an opt-in feature, hence keeping things backwards compatible: https://www.sqlite.org/stricttables.html
Yep, I know. Wish they went even further but it seems like a good start.
I had to do something very similar for analysing CVE information recently, but I don't remember having to use the :memory: option. I suspect it defaults to that if no .db file is specified. Slightly tangentially, when doing aggregated queries, SQLite has a very useful group_concat(..., ',') function that will concatenate the expression in the first arg for each row in the group, separated by the separator in the 2nd…
I just tried it without :memory: and it dropped me into the SQLite shell without executing the query: % sqlite3 -cmd '.mode csv' -cmd '.import taxi.csv taxi' \ 'SELECT passenger_count, COUNT(*), AVG(total_amount) FROM taxi GROUP BY passenger_count' SQLite version 3.36.0 2021-06-18 18:58:49 Enter ".help" for usage hints. sqlite>
Ok, I just went back to see what I did. In fact I echoed a script into sqlite via stdin, not by using `-cmd ''`. In that case, I didn't need to use the :memory: table, and sqlite exited normally when done.
SQLite's virtual table API ( https://www.sqlite.org/vtab.html ) makes it possible to access other data structures through the query engine. You don't need to know much if anything about how the database engine executes queries, you only need to implement the callbacks it needs to do its job. A few years ago I wrote an extension to let me search through serialized Protobufs which were stored as blobs in a regular data…
I like the virtual table API a lot but it has some serious drawbacks. You don't need to know much and indeed, you can't know much about the execution engine, even if that knowledge would help you. Many parts of the query are not pushed down into the virtual table. For instance, if the user query is: SELECT COUNT(*) FROM my_vtab; ... the query your virtual table will effectively see is: SELECT * FROM my_vtab; SQLite d…
I wrote a module that exposes remote SQL Server/PostgreSQL/MySQL servers as SQLite virtual tables, and joins basically don't work at all if your server is not on your local network. There's nothing I can do about it (other than heuristically guessing what IDs might be coming and request them ahead of time) because SQLite doesn't provide enough information to the virtual table layer.
Is it possible to wait until all of the queries are in and then do some query planning at your end to resolve this? Or is it entirely synchronous with no way to do that?