91–100 of 105 posts
Earlier quoted context omitted.
If you find yourself doing a lot of explicit transactions, it can be a sign that your schema isn't as normalized as it should be.
How does better normalization reduce the need for explicit transactions?
Earlier quoted context omitted.
Just don't bother with Postgres. I've run mysql for years in production and have spent probably 30 minutes thinking about managing it. Unless there's some psql feature you need (unlikely), it'll just become a severe pain in your ass down the road because you set something up "wrong." Just vacuuming can barf completely, leaving you dead in the water. TL;DR: if you aren't a DBA and don't want to play one on TV don't bo…
Agreed. And, any time I ask for help with something DB-related, people just chastise me for not running postgres and blindly tell me that's my problem. That sort of community is not what I want to look forward to.
Earlier quoted context omitted.
How does better normalization reduce the need for explicit transactions?
The need to change two separate tables atomically means that you probably have info duplicated. Also it can be non-ideal in a web backend cause it means keeping DB connections open longer.
Yes, you can scale it quite well vertically. But how about horizontally? It would be nice to have high availability, or even to be able to upgrade the OS and postgres itself without downtime.
AFAIK that's what Multigres[0] and Neki[1] are trying to solve. [0] https://multigres.com/ [1] https://neki.dev/
Earlier quoted context omitted.
Last time I almost used a hash index in Postgres, I learned it was an incomplete feature and not crash-safe yet. This was v9.3? At that same time, MySQL had them and they were ok to use. Later that got fixed, but I haven't tried again since, just been using btree because it seemed like Postgres favored that and it has theoretical advantages too.
They are fully stable and perform very well in Postgres today. There are some caveats, but they don't result in any sort of hiccups or unpredictable behavior.
You've probably already read the Postgres docs on hash indexes, but just in case, it says "hash indexes may not be suitable for tables with rapidly increasing number of rows." I agree with the other commenter that it's worth at least trying without them if you haven't already, even though you're already VACUUMing.
People scale postgres by horizontally sharding it at the app-tier, and end up reducing its query capabilities to a glorified K/V store... They do this routinely and think it's normal. Then they use other things like Cassandra or Cosmos DB to augment the scale gaps that this leaves out (see what OpenAI is doing). Then they typically run a search engine and a cache as well. People think all of this complexity is normal…
Earlier quoted context omitted.
> This assertion is likely wrong We can clearly see with iostat/iotop that the server and its storage isn't overworked. > Is this PostgreSQL managed or self-hosted? We're evaluating on our own bare metal. It's an 8c/16t Zen 3 with 32 GB of RAM. Storage (where everything incl. the WAL is) is NVMe and the drives' true sustainable/synchronous write speed is about ~750 MiB/second. The specs are far, far higher than requi…
That's a phenomenal script that absolutely belongs on a dba.stackexchange question and even the PostgreSQL mailing list. I strongly encourage you to post to https://dba.stackexchange.com/ , as a HN thread is the wrong place for this discussion (there's a lot of tuning ahead of you and others who are in your situation in the future might skip this nested thread) but be forewarned, the TLDR from dba.stackexchange will…
We run quite a few of these setups, so cost absolutely plays a big part in the size and selection of hardware. We're evaluating Postgres because of a mergers and acquisitions event prompting us to consolidate portions of the platforms for the sake of savings and reducing the overall "sprawl". Conversely we're also testing how the other party would fare if some of their stuff were to run under MySQL instead of Postgres. There's a chance we end up leaving things just as they are.
"Overall, we find a Postgres server can handle up to 144K of these writes per second. That’s a lot, equivalent to 12 billion writes per day." Based on a problem I'm facing with Postgres today, I wonder if this really progresses as linearly as the article wants to make it out. We're in the middle of evaluating Postgres as a replacement for MySQL, and experience notable slow-down for plain multi-row inserts due to inde…
if the index updates are your bottleneck you can often get away with using a much lighter index (for example BRIN) at a cost of slightly slower queries. This is very often a great and much overlooked tradeoff.
Earlier quoted context omitted.
That's a phenomenal script that absolutely belongs on a dba.stackexchange question and even the PostgreSQL mailing list. I strongly encourage you to post to https://dba.stackexchange.com/ , as a HN thread is the wrong place for this discussion (there's a lot of tuning ahead of you and others who are in your situation in the future might skip this nested thread) but be forewarned, the TLDR from dba.stackexchange will…
Thanks a bunch for all of the config tuning advice. Can see huge improvements in both flattening the creeping slow-down and really pushing back the point where we eventually hit the wall. Just lowering the fill factor on the indices made a substantial difference even before permitting Postgres to use more RAM. Using BRIN indices for certain columns, as suggested by several others in the discussion, also resulted in s…
I am extremely happy for you!
This is fantastic news. I would really appreciate a followup post on DB SE and/or PostgreSQL ML because I assure you, there are others in your position. A deeply nested HN thread is not something that will be visible - you would be saving a lot of people heartburn.
Also remember, we just got started! The tips I gave you earlier is just a drop i the bucket. In the days of AWS RDS, knowledge about these are a "lost art" but running PostgreSQL yourself is extremely empowering. If you could make the posts, please do and I also urge you to test out https://github.com/pgsty/pigsty
Earlier quoted context omitted.
Temporal is a dumpster fire, they've gotten so much VC funding (recently had D, 300M at a 5bn valuation) with ... nothing to build except ways to trap customers into their SAAS. I give them about a year or two before the wheels fall off, then it's off to Broadcom and friends. But I could be wrong as now they're not in the 'durable execution' space at all, it's 'durable execution for ai' according to their latest conf…
Temporal employee here. I'm very surprised by your comment. It's true that we recently had a Series D and that VC firms recognize the value of what we do. The Temporal Server software is 100% open source (MIT license: https://github.com/temporalio/temporal/blob/main/LICENSE ). It's totally free and you don't even need to fill out a registration form, just download precompiled binaries from GitHub or clone the repo an…