Viewing profile — philbe77

TypeScript: https://www.npmjs.com/package/@quack-protocol/sdk

Two new Quack client drivers: ADBC: https://github.com/gizmodata/adbc-driver-quack JDBC: https://github.com/gizmodata/quack-jdbc

Hi, I've created an iOS app (GizmoSQL) for running DuckDB as a server (with Arrow Flight SQL). You can even mount a DuckLake - meaning you can have a "data lake in your pocket". It…

Now it is easy to use Power BI with a DuckDB server you can self-host: GizmoSQL. Powered by Arrow Flight SQL - you can run terabyte-scale analytics for dirt-cheap...

Oh, and GizmoData Cloud (SaaS option) is coming soon - to make it easier than ever to provision GizmoSQL instances...

GizmoSQL is definitely a good option. I work at GizmoData and maintain GizmoSQL. It is an Arrow Flight SQL server with DuckDB as a back-end SQL execution engine. It can support ind…

good point :) - we can re-aggregate HyperLogLog (HLL) sketches to get a pretty accurate NDV (Count Distinct) - see Query.farm's DataSketches DuckDB extension here: https://github.c…

hi sdairs, we did store the data on the worker nodes for the challenge, but not in memory. We wrote the data to the local NVMe SSD storage on the node. Linux may cache the filesyst…

This is something we are trying to take a novel approach to as well. We have a video demonstrating some TPC-H SF10TB queries which perform inner joins, etc. - with GizmoEdge as wel…

I've since learned (from a DuckDB blog) - that DuckDB seems to do better when the XFS filesytem. I used ext4 for this, so I may be able to get another 10 to 15% (maybe!). DuckDB bl…

Hi mosselman, GizmoEdge is not open-source. DeepSeek has "smallpond" however, which is open-source: https://github.com/deepseek-ai/smallpond I plan on getting GizmoEdge to producti…

hi djhworld. The 5s does not include the download/materialization step. That parts takes the worker about 1 to 2 minutes for this data set. I didn't know that this was going on Hac…

If I used "spot" instances - it would have been 63 x $0.732/hr for a total of: $45.99 / hr.

I'm not exactly sure yet. My goal was to not have the shards be too large so as to be un-manageable. In theory - I could just have had 63 (or 64) huge shards - and 1 worker per K8s…

:D that is scary!

You should do it then, and post it here. I did do it with one machine as well: https://gizmodata.com/blog/gizmosql-one-trillion-row-challen...

Challenge accepted - I'll try it on a 4XL Snowflake to get actual perf/cost

Cost-wise, 64 4xl Snowflake clusters would cost: 64 x $384/hr - for a total of: $24,576/hr (I believe)

Good point - I'll update it...

Hi nodesocket - I tried to do 4 CPUs per node, but Kubernetes takes a small (about 200m) CPU request amount for daemon processes - so if you try to request 4 (4000m) CPUs x 16 - yo…

At 5 seconds - the query technically cost: $0.3276

Yes - I also had GizmoSQL (a single-node DuckDB database engine) take the challenge - with very good performance (2 minutes for $0.10 in cloud compute cost): https://gizmodata.com/…