Quack: The DuckDB Client-Server Protocol • Hacker News Board

Re: Quack: The DuckDB Client-Server Protocol

#21

Post by pdet » Tue, May 12, 2026, 9:13 PM UTC

mritchie712 wrote: post #15

> Can I use DuckDB with Quack as the catalog database for DuckLake? > Not yet, but we are working on it! Seems like a niche use case, but it's the one I'm most interested in. Our lakehouse uses ducklake with postgres as the catalog. Seems like a DuckDB / Quack catalog would be an excellent alternative.

I think that Quack will become the primary option for a DuckLake catalog in the future, for several reasons. To list a few:

1. No type mismatches for inlining. If you use a non-DuckDB catalog, many types do not have a 1:1 mapping, which introduces additional overhead when operating on those data types.

2. You get the raw performance of DuckDB analytics (and now transactions) over the catalog. DuckDB reading DuckDB is simply faster than any of our Postgres/SQLite scanners.

3. No round-trip for retries. We can easily(tm) run the full retry logic on the DuckDB server side. Right now, these retries trigger multiple round trips for Postgres, making it a performance bottleneck for high-contention workloads.

Disclaimer: I'm a duckdb/ducklake developer.

Re: Quack: The DuckDB Client-Server Protocol

#22

Post by IceWreck » Tue, May 12, 2026, 9:17 PM UTC

szarnyasg wrote: post #17

mritchie712 wrote: post #15

> Can I use DuckDB with Quack as the catalog database for DuckLake? > Not yet, but we are working on it! Seems like a niche use case, but it's the one I'm most interested in. Our lakehouse uses ducklake with postgres as the catalog. Seems like a DuckDB / Quack catalog would be an excellent alternative.

Well, we are really working on it: https://github.com/duckdb/ducklake/pull/1151 So you'll be able to test it in a few days.

Does this mean I can finally connect to a ducklake instnace hosted remotely? i.e. DuckLake is writing to disk on the remote server and my client is just a client.

Because rn even with Postgres as a catalog my client needs access to the underlying storage to use Ducklake.

Re: Quack: The DuckDB Client-Server Protocol

#23

Post by Lemaxoxo » Tue, May 12, 2026, 9:29 PM UTC

jtbaker wrote: post #10

Lemaxoxo wrote: post #3

Earlier quoted context omitted.

+1 I can't think of many use cases for this and Arrow Flight, other than moving data around.

uh, doing analytics type queries on large datasets that postgres would choke on, as an RPC? I'm using it (ducklake specifically) to build a lakehouse RPC server that can scale horizontally based on resource utilization in k8s.

Right, I get that usecase. You have to crunch numbers that sit somewhere, and store the outputs in the same place. DuckLake is great for that. But where does this DuckDB client-server setup fit in?

Re: Quack: The DuckDB Client-Server Protocol

#24

Post by PhilippGille » Tue, May 12, 2026, 9:31 PM UTC

znite wrote: post #19

Does this work with duckdb-wasm?

It's in the article:

> HTTP also allows the DuckDB-Wasm distribution to speak Quack natively! So DuckDB running in a browser can e.g., directly connect to a DuckDB instance running in an EC2 server using Quack.

Re: Quack: The DuckDB Client-Server Protocol

#25

Post by rglover » Tue, May 12, 2026, 9:37 PM UTC

This is rad. I've been eyeballing using DuckDB in my firm's internal app framework and this just solved the "but how do I horizontally scale this" problem. Kudos to the DuckDB folks. Love "Quack" for the protocol name, too.

Re: Quack: The DuckDB Client-Server Protocol

#26

Post by slotix » Tue, May 12, 2026, 9:38 PM UTC

simlevesque wrote: post #2

I like DuckDB but I'm not sure what it wants to be. There's always new ways to use it and it's not easy to see what's the right one.

I read it less as "DuckDB wants to become Postgres" and more as DuckDB becoming an execution layer inside bigger workflows.

The engine is often not the painful part anymore. The pain is the stuff around it: live DBs, S3 paths, Parquet files, credentials, repeatable runs, exports, validation, and the moment a one-off script quietly becomes infrastructure.

Quack makes the remote/server part cleaner, but the bigger trend seems to be DuckDB becoming the SQL layer inside tools, not necessarily the final user-facing tool.

Re: Quack: The DuckDB Client-Server Protocol

#27

Post by szarnyasg » Tue, May 12, 2026, 9:44 PM UTC

IceWreck wrote: post #22

szarnyasg wrote: post #17

Earlier quoted context omitted.

Well, we are really working on it: https://github.com/duckdb/ducklake/pull/1151 So you'll be able to test it in a few days.

Does this mean I can finally connect to a ducklake instnace hosted remotely? i.e. DuckLake is writing to disk on the remote server and my client is just a client. Because rn even with Postgres as a catalog my client needs access to the underlying storage to use Ducklake.

Yes, Quack resolves this problem. In particular, your client (likely a DuckDB instance) will talk to a remote DuckDB that both has access to the underlying storage and can also serve as the catalog itself.

Re: Quack: The DuckDB Client-Server Protocol

#28

Post by wenc » Tue, May 12, 2026, 9:58 PM UTC

simlevesque wrote: post #2

I like DuckDB but I'm not sure what it wants to be. There's always new ways to use it and it's not easy to see what's the right one.

DuckDB is both a standalone and a component. This effort is actually very coherent and brings it back into a familiar usage model — that of a traditional client server RDBMS.

RDBMS have always been multi-user concurrent systems. DuckDB is a very fast local engine that has a multitude of use cases because it is a embeddable in other systems.

It’s like saying what does SQLite wanna be? It’s in your phones, your browser, your desktop apps, iot devices and people have extended it in different directions. The only difference here is that this is first party not third party. But to me it’s a very legible move.

Re: Quack: The DuckDB Client-Server Protocol

#29

Post by jtbaker » Tue, May 12, 2026, 10:07 PM UTC

Lemaxoxo wrote: post #23

jtbaker wrote: post #10

Earlier quoted context omitted.

uh, doing analytics type queries on large datasets that postgres would choke on, as an RPC? I'm using it (ducklake specifically) to build a lakehouse RPC server that can scale horizontally based on resource utilization in k8s.

Right, I get that usecase. You have to crunch numbers that sit somewhere, and store the outputs in the same place. DuckLake is great for that. But where does this DuckDB client-server setup fit in?

Sounds like it means you don't have to wire up the RPC server yourself anymore? Just build a docker container that invokes this quack server command, expose it over the network and connect to it from remote clients using your own access controls?

Ducklake handles the metadata and storage, but a local duckdb instance connected to it still has to do the compute itself. This lets you federate access to the compute.

Fun for me, I just finished a big streaming implementation doing essentially the same thing in Go-gRPC with arrow table record batches. It was fun though.

Re: Quack: The DuckDB Client-Server Protocol

#30

Post by hermitcrab » Tue, May 12, 2026, 10:08 PM UTC

I have a C++ application. Everything is in memory during execution. Saved to disk between session as XML. Works great, except that that it is strictly single user and some of my customers would love me to generalize it for multiple concurrent users reading and writing. Performance requirements are quite low - a few thousand records being updated by 2 or 3 people at a time. Would DuckDb + Quack be a good choice for this? Or are there better choices? I looked at SQLite, but I understand it doesn't operate as client server.