@drewsberry: I wish you had an RSS feed! I tried to subscribe to your blog but if there is one it's not linked.
(Enjoyed the post)
How Postgres stores data on disk – this one's a page turner
21–30 of 98 posts
Re: How Postgres stores data on disk – this one's a page turner
#22Earlier quoted context omitted.
Depends very much on how the SSDs are designed internally. I think these days we have to settle for "can never be sure" of the real page size for atomicity. Pages can also become corrupt in other ways. It is weird that "--data-checksums" isn't the default for new databases, even when it cost a bit in performance. Integrity should be more important than performance.
Was thinking the same thing when I saw those zeros in the checksum field. Perhaps the consequences are significant. Here's a benchmarking exercise I found: https://www-staging.commandprompt.com/uploads/images/Command... With a tidy summary: > Any application with a high shared buffers hit ratio: little difference. > Any application with a high ratio of reads/writes: little difference. > Data logging application with…
On my M1 mac "dd ... | cksum" takes 3 seconds while "dd | shasum" (sha1) takes 2 seconds. So cksum might not be the best tool for performance checking.
There is CPU specific code in the PG source in src/include/storage/checksum_impl.h
It is written as a plain nested loop in C. So performance is fully dependent on the compiler being able to parallelize or vectorize the code.
I would not be surprised if manually written SIMD code would be faster.
Re: How Postgres stores data on disk – this one's a page turner
#23Earlier quoted context omitted.
I believe HN auto-strips such "filler" words, needs to be added again by a mod
Perhaps it should not? Why would such a silly feature be explained over and over rather than removed ?
HN does a lot of title editorializing. Like the one a few days ago that turned the acronym RATs into Rats and completely changed the meaning.
Re: How Postgres stores data on disk – this one's a page turner
#24> Can’t we just store some data on disk and read / write from it when we need to? (Spoiler: no.)
I disagree. SQLite does a good job in uniting the 2 worlds: complex SQL queries with excellent data consistency and simple file(s). Although SQLite is for sure not the one size fits all solution.
Re: How Postgres stores data on disk – this one's a page turner
#25@drewsberry: I wish you had an RSS feed! I tried to subscribe to your blog but if there is one it's not linked. (Enjoyed the post)
Found it: https://drew.silcock.dev/rss.xml
Re: How Postgres stores data on disk – this one's a page turner
#26Earlier quoted context omitted.
Because there are a lot of articles in the news that add this word for no reason.
Ahhh.. wouldn't that be better by using a percentage of filler words/total words in title threshold?? I don't know if a feature simply strips out the filler words in title, it's not always useful and rather harmful is what I would argue.
Humans work better. HN is small scale enough that a moderator can come along, collapse the off topic comments and fix the title, and it's not an issue.
Re: How Postgres stores data on disk – this one's a page turner
#27Title missing the leading "How"
[deleted]
Re: How Postgres stores data on disk – this one's a page turner
#28@drewsberry: I wish you had an RSS feed! I tried to subscribe to your blog but if there is one it's not linked. (Enjoyed the post)
[deleted]
Re: How Postgres stores data on disk – this one's a page turner
#29This URL is blocked by my company's network because of a certain substring in the URL lol
Re: How Postgres stores data on disk – this one's a page turner
#30A bit of curiosity: how did Postgres choose 8k pages? shouldn’t it be the FS page size to help with atomicity?
Depends very much on how the SSDs are designed internally. I think these days we have to settle for "can never be sure" of the real page size for atomicity. Pages can also become corrupt in other ways. It is weird that "--data-checksums" isn't the default for new databases, even when it cost a bit in performance. Integrity should be more important than performance.
I'm not sure how much bearing internal storage organization should have on Postgres' page size. Since pg explicitly chooses not to implement their own storage organization layer, there's always a filesystem between a pg database and the underlying storage.