Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlloyDB Omni – run AlloyDB anywhere

cloud.google.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: AlloyDB Omni – run AlloyDB anywhere

#71

Post by Cockbrand »

GabeWeiss_ wrote: post #27
Cockbrand wrote: post #12

Earlier quoted context omitted.

If I understand correctly, AlloyDB is a Postgres compatible re-implementation, so there's no shared code. So this is a bit like Wine compared to Windows, only with the big corp re-implementing the OSS software instead of the other way round.

OMG no, we didn't reimplement Postgres. Definitely started from the PG kernel and made modifications from there.

I often reference Cunningham‘s Law, and I’m glad it’s in full effect :)

Thank you for the clarification!

Re: AlloyDB Omni – run AlloyDB anywhere

#72

Post by GabeWeiss_ »

vira28 wrote: post #68

Since we run Vanilla PG on on-prem, this is interesting. Will try once we get clarity on licensing. (Probably we can't use if the code is not open, so let's see)

As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.

Re: AlloyDB Omni – run AlloyDB anywhere

#73

Post by GabeWeiss_ »

dx034 wrote: post #70
GabeWeiss_ wrote: post #46

Earlier quoted context omitted.

It's twice as fast as out of the box PG for most things, and up to 100x faster for reads, depending on what you're doing. So there's that. Also, from a manageability, on top of the index advisor, there's also vacuum management, so it will figure out when the best time to do the garbage cleanup while minimizing impact on performance.

The 100x analytics seems due to columnar storage like with TimescaleDB, but how do you get improvements on the other things? Is it really faster on a database with good indexes?

It really is, yes. I can't go into a lot of detail on the "why" because it's not open source and the product team would murder me...but I highly encourage folks to try it for themselves. Nothing else convinced me until I did it for myself. :)

Re: AlloyDB Omni – run AlloyDB anywhere

#74

Post by ZeroCool2u »

GabeWeiss_ wrote: post #17
lysecret wrote: post #11

Hugely disappointed by Alloydb. Price and performance for a write heavy workload where off the charts horrible. It didn’t even support disabling an instance and it did only storage autoscaling much worse overall experience than using Aurora we moved back to cloudsql. Does someone have good experience with Alloydb?

I have many thoughts here...but let's start with: Do you have your process of how you tested performance? My guess is the eng team would love to look at what you tested.

Just curious, have you tried Alloy with the PostGIS extension?

I've been trying to get my work let us use BigQuery for our geospatial vector data queries, because we've managed to break all the equivalent AWS products, Snowflake, and Databricks on our dataset. Regular Postgres + PostGIS takes about 30 minutes to run our query and BigQuery does the same in 4 seconds. Unfortunately, BigQuery is a bit of a pipe dream right now for us, so it would be interesting to understand if PostGIS benefits from some of the changes in Alloy.

Re: AlloyDB Omni – run AlloyDB anywhere

#75

Post by yandie »

GabeWeiss_ wrote: post #72
vira28 wrote: post #68

Since we run Vanilla PG on on-prem, this is interesting. Will try once we get clarity on licensing. (Probably we can't use if the code is not open, so let's see)

As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.

What's the plan for the license for this? Especially path to production?

Re: AlloyDB Omni – run AlloyDB anywhere

#76

Post by GabeWeiss_ »

ZeroCool2u wrote: post #74
GabeWeiss_ wrote: post #17

Earlier quoted context omitted.

I have many thoughts here...but let's start with: Do you have your process of how you tested performance? My guess is the eng team would love to look at what you tested.

Just curious, have you tried Alloy with the PostGIS extension? I've been trying to get my work let us use BigQuery for our geospatial vector data queries, because we've managed to break all the equivalent AWS products, Snowflake, and Databricks on our dataset. Regular Postgres + PostGIS takes about 30 minutes to run our query and BigQuery does the same in 4 seconds. Unfortunately, BigQuery is a bit of a pipe dream ri…

The improvements we've made on the read side of things likely won't affect the geo data directly, BUT, depending on what other aggregate data you're combining with the geo data you might see some improvement from the columnar engine. Hard to know without digging in deeper on the queries themselves and schemas you're working with. I don't want to steer you down a path of moving a ton of data and infra just to look. Best guess is that it'll be a bit better, but likely closer to your PG experience vs. the BQ experience.

Re: AlloyDB Omni – run AlloyDB anywhere

#77

Post by GabeWeiss_ »

yandie wrote: post #75
GabeWeiss_ wrote: post #72

Earlier quoted context omitted.

As a heads up, we're unlikely to make the codebase for it open. I might be able to convince the product team to open source some components of it (our GM has talked about this before in a couple articles), the whole thing won't be open sourced.

What's the plan for the license for this? Especially path to production?

I'm assuming you mean "I start using this under a free dev license, and want to shift up to the paid production version"? And what that looks like? If that's the case, I don't know yet. That's literally being discussed/hammered as we speak, and we likely won't have a good answer until we're ready to go public preview (not sure on timeline right now, depends heavily on how the tech preview goes). This stage of the game is literally a "We built an awesome thing, please poke at it and tell us where it does/doesn't work!".

Re: AlloyDB Omni – run AlloyDB anywhere

#78

Post by GabeWeiss_ »

ZeroCool2u wrote: post #74
GabeWeiss_ wrote: post #17

Earlier quoted context omitted.

I have many thoughts here...but let's start with: Do you have your process of how you tested performance? My guess is the eng team would love to look at what you tested.

Just curious, have you tried Alloy with the PostGIS extension? I've been trying to get my work let us use BigQuery for our geospatial vector data queries, because we've managed to break all the equivalent AWS products, Snowflake, and Databricks on our dataset. Regular Postgres + PostGIS takes about 30 minutes to run our query and BigQuery does the same in 4 seconds. Unfortunately, BigQuery is a bit of a pipe dream ri…

My PM also reminded me, depending on the data set size too, if it's bigger than the buffer cache, we might see some improvements over PG as well from that (we've made memory improvements around that which might help potentially).

Re: AlloyDB Omni – run AlloyDB anywhere

#79

Post by ZeroCool2u »

GabeWeiss_ wrote: post #76
ZeroCool2u wrote: post #74

Earlier quoted context omitted.

Just curious, have you tried Alloy with the PostGIS extension? I've been trying to get my work let us use BigQuery for our geospatial vector data queries, because we've managed to break all the equivalent AWS products, Snowflake, and Databricks on our dataset. Regular Postgres + PostGIS takes about 30 minutes to run our query and BigQuery does the same in 4 seconds. Unfortunately, BigQuery is a bit of a pipe dream ri…

The improvements we've made on the read side of things likely won't affect the geo data directly, BUT, depending on what other aggregate data you're combining with the geo data you might see some improvement from the columnar engine. Hard to know without digging in deeper on the queries themselves and schemas you're working with. I don't want to steer you down a path of moving a ton of data and infra just to look. Be…

Really appreciate the reply! Everything you said makes sense. It does feel like it would be really hard to beat BigQuery at this kind of task. The main query we run is just comparing two tables geometry where one is a polygon/multipolygon column and the other is a column of points. We ask, are any of these points in any of these polygons? So it's N x M comparisons, pretty much worst case scenario and there's no aggregation. I've had a hell of time trying to optimize PG for just running this one query. On the bright side, I've gotten to learn about and try a bunch of different databases now though :)
Post reply on HN