61–70 of 86 posts
Very Interesting. Glossing over the article, it states HA within a Region. Curious if anyone is successful in running any Database with multi-region HA? How are you doing it?
https://learn.microsoft.com/en-us/sql/sql-server/failover-cl...
It’s amazing that this isn’t a solved problem, but we have all of this crazy language model stuff. Unfortunately Spanner isn’t open source. Yugabyte and Citus are close but have annoying issues. Cockroach isn’t 100% compatible (and has its own issues) and things like FoundationDB which are truly HA and comparable to Spanner in terms of consistency and fault tolerance are not easily plugged into Postgres as the underl…
Honestly, just use Spanner. It's the perfect database*. If you need to trade freshness for savings, put a cache in front of it. If I were building a new startup in 2023, I would need a mountain of evidence against using Spanner. It's ugly that it locks you into GCP but hey an iPhone locks you into Apple's ecosystem, that's just the price you pay to get good things. * unless you need timeseries, columnar, FTS, geospat…
Earlier quoted context omitted.
Honestly, just use Spanner. It's the perfect database*. If you need to trade freshness for savings, put a cache in front of it. If I were building a new startup in 2023, I would need a mountain of evidence against using Spanner. It's ugly that it locks you into GCP but hey an iPhone locks you into Apple's ecosystem, that's just the price you pay to get good things. * unless you need timeseries, columnar, FTS, geospat…
Spanner is ridiculously expensive. Though I get your point, it’s not like an iPhone. An iPhone is not actually that expensive compared to top end android phones. Spanner is very expensive even compared to hosted cockroach and RDS (Aurora). That being said I’m sure if you're enterprise it's substantially cheaper. Spanner generally is about twice the cost as cockroach which seems cheap but the gap in nominal cost only…
Back of napkin math I did a couple of months ago: Spanner costs about $1/month for 2 writes/second and 10 reads/second. The minimum is $88 which is 176wps and 880rps. If your startup gets more traffic than that, you have good problems. The simplified guarantee model (fully linearizable and serializable everything; automagic sharding, replication and failover) could save you costs elsewhere.
Low-value high-volume data like logs and search queries can be sent to a cheaper self-hosted timeseries/fts database.
I just implemented master with read replica with the bitnami postgres-repmgr image. It's not perfect, but it works and running my own instances in aws instead of rds is going to save me close to 80% when i also add in purchasing a savings plan. By setting this up I've learned more about postgres than i ever ever wanted too. lol
Just a heads up, we were using the bitnami charts in our first attempt at an HA postgres instance in our k8s cluster, and we're currently figuring out how to move away from them because they had so many issues. Granted, we might've messed something up and there are lots of factors, but you should manually check your individual nodes every so often to make sure none of them are going out of sync. Need to connect to ea…
Very Interesting. Glossing over the article, it states HA within a Region. Curious if anyone is successful in running any Database with multi-region HA? How are you doing it?
In SQL Server you can use a multi-subnet failover cluster. https://learn.microsoft.com/en-us/sql/sql-server/failover-cl...
https://learn.microsoft.com/en-us/azure/azure-sql/database/s...
Somewhat related, but I'm looking at hosting solutions for a hobby SaaS I'm building, and I would like to use google cloud run, but the cloud sql pricing seems extremely expensive if you want HA and are just starting up. They also don't seem to offer any cloud sql as part of their free like like AWS does with RDS, it just seems very odd. I looked at Planetscale thinking I might be willing to just use MySQL, but even…
We have FKs and new pricing coming soon.
Earlier quoted context omitted.
I've built my own little pile of evidence against Spanner using it for several years, after previously being on Postgres. It has good potential but isn't there yet. And if you're making a startup, you're probably a long way from the scale where sharded DBs become necessary.
I hate this tired argument. If I'm building a startup I'm building it with the intention to hit scale. If I can use cockroach or yugabyte and get 90% of the benefits of postgres without significant additional cost I'm going to use it. I've been at multiple startups that have had trouble scaling their database and the cost to switch database technologies when you hit scale is massive. And worst of all, the limitations…
It’s amazing that this isn’t a solved problem, but we have all of this crazy language model stuff. Unfortunately Spanner isn’t open source. Yugabyte and Citus are close but have annoying issues. Cockroach isn’t 100% compatible (and has its own issues) and things like FoundationDB which are truly HA and comparable to Spanner in terms of consistency and fault tolerance are not easily plugged into Postgres as the underl…
Doesn't Spanner depend on access to globally-synchronized custom atomic clock hardware? i.e. even if it were open source, it wouldn't really be valuable unless you're a data center operator. As I understand it, it's not a solved problem because there is no silver bullet, but rather trade-offs in every direction, and which solution works for you (including Spanner) is heavily dependent on your use case.
Earlier quoted context omitted.
I've built my own little pile of evidence against Spanner using it for several years, after previously being on Postgres. It has good potential but isn't there yet. And if you're making a startup, you're probably a long way from the scale where sharded DBs become necessary.
I hate this tired argument. If I'm building a startup I'm building it with the intention to hit scale. If I can use cockroach or yugabyte and get 90% of the benefits of postgres without significant additional cost I'm going to use it. I've been at multiple startups that have had trouble scaling their database and the cost to switch database technologies when you hit scale is massive. And worst of all, the limitations…
Here's the problem, it's not 90%. Idk about those two, but you sacrifice a lot using Spanner, maybe enough to delay your launch. Any slightly advanced query becomes hard to optimize, for one. Not to dis Spanner, it's just a very different beast.
CEO won't want to hear "launch is delayed, but it'll scale better later," and it's also not really true. In the initial phases, you can't tell what your actual needs will be at scale, so you'll probably have to rework things anyway. Just using Spanner doesn't solve your scaling problems. Maybe you'll need to totally redo your Spanner schema to actually scale, maybe you find you're better off sharding at the application layer than at the DB.
It’s amazing that this isn’t a solved problem, but we have all of this crazy language model stuff. Unfortunately Spanner isn’t open source. Yugabyte and Citus are close but have annoying issues. Cockroach isn’t 100% compatible (and has its own issues) and things like FoundationDB which are truly HA and comparable to Spanner in terms of consistency and fault tolerance are not easily plugged into Postgres as the underl…