Viewing profile — chrisjc

Any thought about a pass-through server for various non-local databases? It should be extremely simple for databases that support ADBC (for example Snowflake, PostgreSQL). For othe…

duckdb -cmd 'CREATE TABLE my_data AS FROM READ_PARQUET($$data.parquet$$)' -ui `duckdb -ui sqlitedb.db` should work bc duckdb can read sqlite files. If it doesn't autoload extension…

duckdb -ui pre_seeded_db.db duckdb -ui -init startup.sql where startup.sql contains various statements/commands like `.open pre_seeded_db.db` Alternatively place statements/command…

> In the worst-case scenario, they would be 2-3 years behind the cutting edge, which is not mission-critical. It's also worth considering how such an event might affect the US and …

Haven't tried it out yet, but the release notes look very promising esp relating to Arrow interactions - Push dynamically generated join filters through UNION, UNNEST and AGGREGATE…

Perhaps I don't fully understand what you're saying CREATE TABLE person (name VARCHAR, age BIGINT); INSERT INTO person VALUES ('Alice', 37), ('Ana', 35), ('Bob', 41), ('Bea', 25); …

I've been eagerly awaiting this for a couple of months now. And I've long wondered why there hasn't been such an implementation/extension for Flight especially as there are extensi…

For read-oriented interactions... ADBC --sql--> Flight SQL Server --flight--> Flight Servers (plural) --> datasources or SELECT * FROM WHERE ... where the "relation" is a collectio…

Thank you for all the work you guys do. The Arrow ecosystem is just absolutely incredible. My few gripes related to interop with duckdb are related to Arrow scanning/pushdowns. And…

And all the Arrow parts work together quite nicely. ADBC client --> Flight SQL (duckdb/whatever) --> Flight --> ? The result highlights your exact point: why take your data and tra…

Since you're using python, have you looked into sqlglot? I think it has some pretty-print options. https://github.com/tobymao/sqlglot

Maybe something like this? https://duckdb.org/2024/10/02/pyodide.html

Probably also worth mentioning that DuckDB can interact with SQLite dbs. https://duckdb.org/docs/extensions/sqlite.html https://duckdb.org/docs/guides/database_integration/sqlite.h…

Probably not exactly what you mean or asking for, but the work Motherduck is doing looks promising. https://motherduck.com/blog/differential-storage-building-bl... Hopefully it fin…

Ibis looks very promising. There are also other ways to use both. https://duckdb.org/docs/guides/python/polars.html All of this dataframe compatibility is awesome. (much thanks to …

Perhaps not exactly what you're talking about, but maybe? (unsure bc the with statements are sometimes called "temp tables") https://duckdb.org/docs/sql/query_syntax/with#cte-mater…

Although only experimental and probably off topic to the discussion, it's worth mentioning DuckDB also provides a Spark API implementation. https://duckdb.org/docs/api/python/spark…

Not the original parent, so unsure of their use-case. But I've seen the approach where some/basic development can be done on duckdb, before making its way to dev/qa/prd. Something …

Real awesome project. This would be even better if some of the differential storage functionality makes its way into duckdb. https://motherduck.com/blog/differential-storage-buildi…

Perhaps similar to https://github.com/duckdb/dbt-duckdb , but SQLMesh instead of DBT obviously.

It should be just as easy as adding: authenticator="externalbrowser" Adding a Snowflake connection configuration option that allowed for standard Snowflake connection conventions m…

Yes, DuckDB does do Iceberg. https://duckdb.org/docs/extensions/iceberg

Interested to hear why you switched from DataFusion?

DuckDB is really for OLAP/Columnar (analytical workloads) whereas SQLite is more for OLTP/Row-based (transactional). When you start running aggregations/windows over large amounts …

Via the AWS extension https://duckdb.org/docs/guides/network_cloud_storage/s3_impo... https://duckdb.org/docs/extensions/aws