Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen: Amazon RDS for PostgreSQL 17.4

jepsen.io

Post reply on HN
Print view View original

121–130 of 153 posts

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#121

Post by aphyr »

mattashii wrote: post #114
nijave wrote: post #15

It's not entirely clear but this isn't an issue in multi instance upstream Postgres clusters? Am I correct in understanding either AWS is doing something with the cluster configuration or has added some patches that introduce this behavior?

> It's not entirely clear but this isn't an issue in multi instance upstream Postgres clusters? No, it isn't an issue with single-instance PostgreSQL clusters. Multi-instance PostgreSQL clusters (single primary, plus streaming/physical replicas) are affected. What they -too- discovered is that PostgreSQL currently doesn't have consistent snapshot behaviour between the primary and replicas. Presumably, read-only trans…

Thank you matashii--this would definitely explain it. I've also received another email suggesting this anomaly is due to the difference in commit/visibility order between primary and secondary. Is there by chance a writeup of this available anywhere that I can link to? It looks like https://postgrespro.com/list/thread-id/1827129 miiight be related, but I'm not certain. If so, I'd like to update the report.

My email is aphyr@jepsen.io, if you'd like to drop me a line. :-)

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#122

Post by Twirrim »

hliyan wrote: post #102

I wish more writing in the software world was done this way: "Amazon RDS for PostgreSQL is an Amazon Web Services (AWS) service which provides managed instances of the PostgreSQL database. We show that Amazon RDS for PostgreSQL multi-AZ clusters violate Snapshot Isolation, the strongest consistency model supported across all endpoints. Healthy clusters occasionally allow..." Direct, to-the-point, unembellished and an…

I'm so past wanting to read meme laden blog posts. Especially when all too often it's just stretching a paragraph of content. Security vulnerability stuff is probably the worst at it these days.

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#123

Post by sgarland »

hliyan wrote: post #102

I wish more writing in the software world was done this way: "Amazon RDS for PostgreSQL is an Amazon Web Services (AWS) service which provides managed instances of the PostgreSQL database. We show that Amazon RDS for PostgreSQL multi-AZ clusters violate Snapshot Isolation, the strongest consistency model supported across all endpoints. Healthy clusters occasionally allow..." Direct, to-the-point, unembellished and an…

A company I was at had an internal blog where anyone could write an article, and others could comment on it. Zero requirement to do so, and it in no way factored into your rating. I think it was the result of a hackathon one year.

Anyway, I really enjoyed it, because I like technical writing. I found that if I wrote a deeply technical post, I’d get very few likes and comments – in fact, I even had a Staff Eng tell me I should more narrowly target the audience (you could tag groups as an intended audience; they’d only see the notification if they went to the blog, so it wasn’t intrusive) because most of engineering had no idea what I was talking about.

Then, I made a post about Kubecost (disclaimer: this was in its very early days, long before being acquired by IBM; I have no idea how it performs now, and this should not dissuade you from trying it if you want to) and how in my tests with it, its recommendations were a poor fit, and would have resulted in either minimal savings, or caused container performance issues. The post was still fairly technical, examining CPU throttling, discussing cgroups, etc. but the key difference was memes. People LOVED it.

I later repeated this experiment with something even more technical; IIRC it involved writing some tiny Python external library in C and accessing it with ctypes, and comparing stack vs. heap allocations. Except, I also included memes. Same result, slightly lessened from the FinOps one, but still far more likes and comments than I would expect for something so dry and utterly inapplicable to most people’s day-to-day job.

Like you, I find this trend upsetting, but I also don’t know how else to avoid it if you’re trying to reach a broader audience. Jensen, of course, is not, and I applaud them for their rigorous approach and pure writing.

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#124

Post by mdaniel »

KronisLV wrote: post #93
__float wrote: post #90

Earlier quoted context omitted.

It may violate the SQL Server license? Microsoft have not apparently paid for a Jepsen analysis (or perhaps don't want it public :))

> Microsoft have not apparently paid for a Jepsen analysis (or perhaps don't want it public :)) If I was some database vendor that sometimes plays fast and loose (not saying Microsoft is, just an example) and my product is good for 99.95% of use cases and the remainder is exceedingly hard to fix, I'd probably be more likely to pay for Jepsen not to do an analysis, because hiring them would result in people being more…

And yet, this one was done without compensation, so it seems the value of the report and the backing investigation is not only for money

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#125

Post by jbaiter »

sgarland wrote: post #123
hliyan wrote: post #102

I wish more writing in the software world was done this way: "Amazon RDS for PostgreSQL is an Amazon Web Services (AWS) service which provides managed instances of the PostgreSQL database. We show that Amazon RDS for PostgreSQL multi-AZ clusters violate Snapshot Isolation, the strongest consistency model supported across all endpoints. Healthy clusters occasionally allow..." Direct, to-the-point, unembellished and an…

A company I was at had an internal blog where anyone could write an article, and others could comment on it. Zero requirement to do so, and it in no way factored into your rating. I think it was the result of a hackathon one year. Anyway, I really enjoyed it, because I like technical writing. I found that if I wrote a deeply technical post, I’d get very few likes and comments – in fact, I even had a Staff Eng tell me…

It's funny, because I remember the early days of Jepsen, and it relied heavily on memes (the whole name is based on "call me maybe"/carly rae jepsen) and aphyr wasn't (and still isn't) shy about his colorful real life personality :-)

See for example https://aphyr.com/posts/282-call-me-maybe-postgres, which makes heavy uses of memes.

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#126

Post by renewiltord »

benatkin wrote: post #40
renewiltord wrote: post #35

Earlier quoted context omitted.

It's maximal information communication. Use LLM to distill to your own knowledge level. It is trivial with modern LLM. Very good output in general.

It addresses the reader no matter how knowledgeable they are. It's a very good use of hypertext, making it so that a knowledgeable reader won't need to skip over much.

Better with LLM.

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#127

Post by fuy »

augustl wrote: post #105
hliyan wrote: post #102

I wish more writing in the software world was done this way: "Amazon RDS for PostgreSQL is an Amazon Web Services (AWS) service which provides managed instances of the PostgreSQL database. We show that Amazon RDS for PostgreSQL multi-AZ clusters violate Snapshot Isolation, the strongest consistency model supported across all endpoints. Healthy clusters occasionally allow..." Direct, to-the-point, unembellished and an…

Jepsen is awesome, on so many levels!

isolation levels, that is!

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#128

Post by fuy »

havkom wrote: post #107

Good investigation! Software developers nowadays barely know about transactions, and definitely not about different transaction models (in my experience). I have even encountered "senior developers" (who are actually so called "CRUD developers"), who are clueless about database transactions.. In reality, transactions and transaction models matter a lot to performance and error free code (at least when you have volume…

Had similar situation a few years before - switched a (now) billion revenue product from Read Committed to Read Committed Snapshot with huge improvements in performance. One thing to be aware when doing this - it will break all code that rely on blocking reads (e.g. select with exists). These need to be rewritten using explicit locks or some other methods.

Re: Jepsen: Amazon RDS for PostgreSQL 17.4

#129

Post by kchoudhu »

I've suspected that there are consistency issues on RDS for a while now: if you push large quantities of data (e.g. 1MM+ rows) into a database quickly and then try to read the same data out on another connection, you'll periodically get null return sets.

We've worked around it by not touching the hot stove, but it's kind of worrying that there are consistency issues with it.

Post reply on HN