Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PgDog is funded and coming to a database near you

pgdog.dev

Post reply on HN
Print view View original

201–210 of 275 posts

Re: PgDog is funded and coming to a database near you

#201

Post by inigyou »

hylaride wrote: post #61
levkk wrote: post #3

Earlier quoted context omitted.

Not quite. The performance gain is to bring those features to Postgres! Edit: Performance gains are from having the ability to load balance reads (horizontal scaling for read queries) and scale out writes (with sharding). Once instance bottleneck in Postgres has many faces: 1. Behind schedule vacuums because of too many dead tuples (too many writes) 2. The WALWriter is single-threaded and IO-bound - Postgres can only…

So to oversimplify, is the idea to bring an AWS Aurora-style storage mechanism natively to Postgres?

Aurora is one big database, isn't it? PgDog is just a proxy where you tell it which shard to access.

Re: PgDog is funded and coming to a database near you

#202

Post by frollogaston »

parthdesai wrote: post #120
gen220 wrote: post #115

Is there an explainer for people who are broadly familiar with the DB space? It sounds like you're building an equivalent to Vitesse for Postgres, but it's not super clear from the article (which I know is not the point of this, but still :) ). Edit: It also might be interesting to point out how your solution differs from what the folks at Planetscale are building https://planetscale.com/neki

There's multiple solutions coming up in this space: 1. Neki as you mentioned 2. PgDog 3. Multigres, headed by original creator of Vitesse

Citus is an older one that does something like this, right? But it's an extension, not a proxy.

Re: PgDog is funded and coming to a database near you

#203

Post by inigyou »

exabrial wrote: post #127

> The reason DBs like Mongo or Dynamo exist is because Not quite. The reason "DBs" like those exist is purely due to fashion. Lets not kid ourselves into thinking they do anything better, save the exception of making data hard to access, which might be a project goal in some cases.

Dynamo definitely scales better than anything else at the tradeoff of not guaranteeing durability in the case of enough node failures and (like most distributed databases) not allowing interaction between different pieces of data.

Re: PgDog is funded and coming to a database near you

#204

Post by levkk »

inigyou wrote: post #198
chrisvenum wrote: post #28

I am trying to gain a basic understanding of this: Right now I have a 4TB DB on one large box. Is the idea that using a proxy tool like PGDog I could spin up 8 smaller boxes handling ~500GB each and then one medium box for the proxy? Right now I have a project that has very heavy write traffic from multiple services and a web app that reads from this. We are starting to hit the point where no amount of indexing, quer…

That's the idea of sharding. If you read the pgdog docs, you'll notice you need to tell it which shard server to route your request to - it doesn't just magically work. It's still providing value by reusing connections, which are particularly expensive in postgres. Because it's not magic, you do still have to know what's going on under the hood, e.g. no cross-shard transactions. I'd see if my application can benefit…

fwiw, we support cross-shard transactions. They are not magic though, just good old 2pc and a bit of coordination.

Re: PgDog is funded and coming to a database near you

#205

Post by eikenberry »

MeetingsBrowser wrote: post #122
eikenberry wrote: post #104

Earlier quoted context omitted.

That's great news! I'll bookmark this in case I'm forced to manage Postgres again.

What do you use instead?

I tend towards using key-value databases as I find them general purpose enough while being much more robust. I'm not married to any one in particular, depends on the requirements.

Re: PgDog is funded and coming to a database near you

#206

Post by hasyimibhar »

znpy wrote: post #63
yabones wrote: post #30

I'm curious how this might help with our biggest downtime-causer with postgres, which is major version upgrades. Poolers do a great job for failover and load balancing, but we consistently need ~10-20 minutes of downtime once or twice a year to do upgrades. Logical replication between old->new versions could probably help, but it would still require flipping everything over to the new cluster without partial writes o…

It's weird that PostgreSQL still doesn't have a proper, open source, general multi-master implementation. At this point i wonder if i'll ever see that.

What about Multigres[0]? It builds on top of Postgres and adds HA (based on Flexible Paxos[1]), sharding, etc. They're still not production-ready, but I'm highly optimistic they will solve a lot of the problems Postgres have.

For example, with Multigres, you should be able to achieve true zero downtime major version upgrade by simply resharding [2]. With vanilla Postgres + pgBouncer, you can only achieve near-zero downtime (few seconds at most), though it's probably good enough for most use cases.

[0] https://multigres.com/

[1] https://fpaxos.github.io/

[2] https://multigres.com/docs#migrate-across-postgres-versions

Re: PgDog is funded and coming to a database near you

#210

Post by pinkgolem »

eikenberry wrote: post #99

> The reason DBs like Mongo or Dynamo exist is because Postgres has a scaling problem. I've used Postgres at a few places and the #1 problem was always high availability, not scaling. One Postgres cluster could easily handle 100000 transactions per minute, but when a primary node went down it was a page and manually failing over to the spare then manually replacing the spare. The manual tooling was very finicky but a…

Have you tried cnpg? Worked amazingly well for my usecases
Post reply on HN