DuckDB – Data power tools for your laptop, now in Clojure (2023)
1–10 of 24 posts
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#2I’m a big fan of tmducken. We use it heavily in our prod systems. That said, we’ve recently started exploring ducktape [1] in our new projects and have been really impressed with the performance. It also support more complex types on insertions and queries which has been helpful for us. Not affiliated with the project, but just wanted to show it some love since it’s a bit newer. It was created by an active contributor to tmducken.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#3At Cronitor we use ClickHouse, but we're leaving it behind for our next product and building directly on Parquet and DuckDB.
We think the future of observability in the AI age is self-hosted directly on NVMe backed by cheap and limitless object storage. I don't want to send customer conversations and agent thoughts to a giant multi-tenant borg SaaS database like Sentry or BetterStack.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#4Duckdb CLI is a powerhouse, it can load files as diverse as gzipped json lines, so you can stuff compressed logs straight into a directory yet still easily query them with SQL when you need to.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#5> Developing such a high quality power tool in such an open manner is honorable.
Credit where credit is due, I would say their efforts are more than just "honorable", I could easily prefix that with an "extremely" and maybe add in a ", most excellent" afterwards.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#6i dont really into languages now. llm has solved this abstraction "perfectly".
most of my query now are llm generated. i created custom api to connect llm to any databases im currently using.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#7I’m a big fan of tmducken. We use it heavily in our prod systems. That said, we’ve recently started exploring ducktape [1] in our new projects and have been really impressed with the performance. It also support more complex types on insertions and queries which has been helpful for us. Not affiliated with the project, but just wanted to show it some love since it’s a bit newer. It was created by an active contributo…
tmducken creator here, this is so rad to see - duckdb is a monster for sql on csv.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#8i dont really into languages now. llm has solved this abstraction "perfectly". most of my query now are llm generated. i created custom api to connect llm to any databases im currently using.
Then perhaps this isn't for you.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#9Impressive, you can really do a lot on a single node when it comes to big-data queries nowadays, I agree too many jump straight to a Spark cluster or something similar when you can just write a small script on a single node.
Re: DuckDB – Data power tools for your laptop, now in Clojure (2023)
#10i dont really into languages now. llm has solved this abstraction "perfectly". most of my query now are llm generated. i created custom api to connect llm to any databases im currently using.
You understand this is more than that right? This is saying, next time you need to analyze a big dataset, tell your AI to load it in DuckDB, and if it needs to run a query that SQL cannot do, tell it to do what they did in this blog, and it'll have the query run using Clojure super fast.
It's not really about querying a DB from Clojure, well not in the boring sense of how do I run some SQL over data in an existing DB.
What this is supposed to replace is say a big Spark Cluster for processing/querying large datasets.