How we give every user SQL access to a shared ClickHouse cluster • Hacker News Board

Re: How we give every user SQL access to a shared ClickHouse cluster

#21

Post by staticassertion » Sat, Mar 21, 2026, 1:42 PM UTC

weird-eye-issue wrote: post #19

staticassertion wrote: post #16

Earlier quoted context omitted.

I'd be so uncomfortable with this. It sounds like you're placing the full burden of access on a single boundary. I mean, maybe there's more to it that you haven't spoken about here, but "everything rests on this one postgres feature" is an unacceptably unsafe state to me.

It's not like RLS is just some random feature they are misusing. It's specifically for security and is absolutely reliable. Maybe you should do a bit more research before making comments like this.

Of course it's designed for security... that has nothing to do with my statement. No single boundary is "absolutely reliable", that's my entire point.

Re: How we give every user SQL access to a shared ClickHouse cluster

#22

Post by cjonas » Sat, Mar 21, 2026, 1:42 PM UTC

Waterluvian wrote: post #14

cjonas wrote: post #12

We just create mini data "ponds" on the fly by copying tenant isolated gold tier data to parquet in s3. The users/agent queries are executed with duckdb. We run this process when the user start a session and generate an STS token scoped to their tenant bucket path. Its extremely simple and works well (at least with our data volumes).

Is that why it’s called DuckDb? Because data ponds?

Idk but I named everything in the related code "duckpond" :)

Re: How we give every user SQL access to a shared ClickHouse cluster

#23

Post by Philip-J-Fry » Sat, Mar 21, 2026, 1:45 PM UTC

staticassertion wrote: post #16

zie wrote: post #3

We do the same thing, every employee can access our main financial/back office SQL database, but we just use PostgreSQL with row level security[0]. We never bothered to complicate it like the post does. 0: https://www.postgresql.org/docs/18/ddl-rowsecurity.html

I'd be so uncomfortable with this. It sounds like you're placing the full burden of access on a single boundary. I mean, maybe there's more to it that you haven't spoken about here, but "everything rests on this one postgres feature" is an unacceptably unsafe state to me.

Conceptually that's no different to any security measures that prevent you from accessing data you're not supposed to? At the end of the day with all data that is colocated you're trusting that some permission feature somewhere is preventing you from accessing data you're not supposed to.

We trust that Amazon or Google or Microsoft are successful in protecting customer data for example. We trust that when you log into your bank account the money you see is yours, and when you deposit it we trust that the money goes into your account. But it's all just mostly logical separation.

Re: How we give every user SQL access to a shared ClickHouse cluster

#24

Post by mritchie712 » Sat, Mar 21, 2026, 1:48 PM UTC

Waterluvian wrote: post #14

cjonas wrote: post #12

We just create mini data "ponds" on the fly by copying tenant isolated gold tier data to parquet in s3. The users/agent queries are executed with duckdb. We run this process when the user start a session and generate an STS token scoped to their tenant bucket path. Its extremely simple and works well (at least with our data volumes).

Is that why it’s called DuckDb? Because data ponds?

Hannes (one of the creators) had a pet duck

Re: How we give every user SQL access to a shared ClickHouse cluster

#25

Post by skeeter2020 » Sat, Mar 21, 2026, 1:49 PM UTC

staticassertion wrote: post #16

zie wrote: post #3

We do the same thing, every employee can access our main financial/back office SQL database, but we just use PostgreSQL with row level security[0]. We never bothered to complicate it like the post does. 0: https://www.postgresql.org/docs/18/ddl-rowsecurity.html

I'd be so uncomfortable with this. It sounds like you're placing the full burden of access on a single boundary. I mean, maybe there's more to it that you haven't spoken about here, but "everything rests on this one postgres feature" is an unacceptably unsafe state to me.

row level security is not a feature specific to Postgres, but more a pretty standard and acceptable way to control access in a multitenant or multicontext environment that pretty much every data provider supports/implements. When it comes to answering a single specific question (like the one RLS targets) I believe you DO want a single, simple answer, vs. something like "it uses these n independent things working in conjunction..."

Re: How we give every user SQL access to a shared ClickHouse cluster

#26

Post by staticassertion » Sat, Mar 21, 2026, 1:51 PM UTC

Philip-J-Fry wrote: post #23

staticassertion wrote: post #16

Earlier quoted context omitted.

I'd be so uncomfortable with this. It sounds like you're placing the full burden of access on a single boundary. I mean, maybe there's more to it that you haven't spoken about here, but "everything rests on this one postgres feature" is an unacceptably unsafe state to me.

Conceptually that's no different to any security measures that prevent you from accessing data you're not supposed to? At the end of the day with all data that is colocated you're trusting that some permission feature somewhere is preventing you from accessing data you're not supposed to. We trust that Amazon or Google or Microsoft are successful in protecting customer data for example. We trust that when you log int…

> At the end of the day with all data that is colocated you're trusting that some permission feature somewhere is preventing you from accessing data you're not supposed to.

Right but ideally more than one.

> But it's all just mostly logical separation.

Yes, ideally multiple layers of this. You don't all share one RDS instance and then get row level security.

Re: How we give every user SQL access to a shared ClickHouse cluster

#27

Post by staticassertion » Sat, Mar 21, 2026, 1:53 PM UTC

skeeter2020 wrote: post #25

staticassertion wrote: post #16

Earlier quoted context omitted.

I'd be so uncomfortable with this. It sounds like you're placing the full burden of access on a single boundary. I mean, maybe there's more to it that you haven't spoken about here, but "everything rests on this one postgres feature" is an unacceptably unsafe state to me.

row level security is not a feature specific to Postgres, but more a pretty standard and acceptable way to control access in a multitenant or multicontext environment that pretty much every data provider supports/implements. When it comes to answering a single specific question (like the one RLS targets) I believe you DO want a single, simple answer, vs. something like "it uses these n independent things working in c…

Right, RLS is great. What they are saying is this:

> every employee can access our main financial/back office SQL database

This means that there is no access gate other than RLS, which includes financial data. That is a lot of pressure on one control.

Re: How we give every user SQL access to a shared ClickHouse cluster

#28

Post by QuantumNomad_ » Sat, Mar 21, 2026, 1:54 PM UTC

Waterluvian wrote: post #14

cjonas wrote: post #12

We just create mini data "ponds" on the fly by copying tenant isolated gold tier data to parquet in s3. The users/agent queries are executed with duckdb. We run this process when the user start a session and generate an STS token scoped to their tenant bucket path. Its extremely simple and works well (at least with our data volumes).

Is that why it’s called DuckDb? Because data ponds?

The DuckDB website has the following to say about the name:

> Why call it DuckDB?

> Ducks are amazing animals. They can fly, walk and swim. They can also live off pretty much everything. They are quite resilient to environmental challenges. A duck's song will bring people back from the dead and inspires database research. They are thus the perfect mascot for a versatile and resilient data management system.

https://duckdb.org/faq#why-call-it-duckdb

Re: How we give every user SQL access to a shared ClickHouse cluster

#29

Post by weird-eye-issue » Sat, Mar 21, 2026, 1:55 PM UTC

staticassertion wrote: post #21

weird-eye-issue wrote: post #19

Earlier quoted context omitted.

It's not like RLS is just some random feature they are misusing. It's specifically for security and is absolutely reliable. Maybe you should do a bit more research before making comments like this.

Of course it's designed for security... that has nothing to do with my statement. No single boundary is "absolutely reliable", that's my entire point.

This is the real world not everybody can get a perfectly isolated database instance. Also you do realize that is not necessarily any more secure than RLS right? Something still has to control what database the code connects to. That could have a flaw just as much as you could have a flaw when setting up RLS.

Re: How we give every user SQL access to a shared ClickHouse cluster

#30

Post by Philip-J-Fry » Sat, Mar 21, 2026, 1:55 PM UTC

staticassertion wrote: post #26

Philip-J-Fry wrote: post #23

Earlier quoted context omitted.

Conceptually that's no different to any security measures that prevent you from accessing data you're not supposed to? At the end of the day with all data that is colocated you're trusting that some permission feature somewhere is preventing you from accessing data you're not supposed to. We trust that Amazon or Google or Microsoft are successful in protecting customer data for example. We trust that when you log int…

> At the end of the day with all data that is colocated you're trusting that some permission feature somewhere is preventing you from accessing data you're not supposed to. Right but ideally more than one. > But it's all just mostly logical separation. Yes, ideally multiple layers of this. You don't all share one RDS instance and then get row level security.

Can you give an example of more than one layer of logical separation at the data layer?

We all know that authentication should have multiple factors. But that's a different problem. Fundamentally at the point you're reading or writing data you're asking the question "does X has permission to read/write Y".

I don't see what you're getting at.