Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Postgres stores data on disk – this one's a page turner

drew.silcock.dev

Post reply on HN
Print view View original

31–40 of 98 posts

Re: How Postgres stores data on disk – this one's a page turner

#31

Post by hans_castorp »

da_chicken wrote: post #19
nraynaud wrote: post #12

A bit of curiosity: how did Postgres choose 8k pages? shouldn’t it be the FS page size to help with atomicity?

8k is a very common page size, but 4k isn't unheard of. Oracle's default is 4k. The issue is that page size caps row size (for on-row storage). Also, if you have a smart clustering index, larger pages can be more efficient use of index addressing. So it's a trade-off.

Oracle defaults to 8k as well:

https://docs.oracle.com/en/database/oracle/oracle-database/1...

> Default value 8192

Re: How Postgres stores data on disk – this one's a page turner

#32

Post by magicalhippo »

speed_spread wrote: post #18
silvestrov wrote: post #15

Earlier quoted context omitted.

Depends very much on how the SSDs are designed internally. I think these days we have to settle for "can never be sure" of the real page size for atomicity. Pages can also become corrupt in other ways. It is weird that "--data-checksums" isn't the default for new databases, even when it cost a bit in performance. Integrity should be more important than performance.

If physical integrity is already provided by the backing filesystem, such as ZFS, wouldn't --data-checksums be redundant?

If data is served from ARC, it's primary cache, ZFS does not perform a checksum check before handing it to you, as the data was checked when it got read into the ARC.

If you use ECC you're quite safe, but ECC can't detect multi-bit errors, just single and double bit errors.

So if you care much about your integrity, you might want Postgres to do its checksum check as well.

Re: How Postgres stores data on disk – this one's a page turner

#34

Post by MattJ100 »

indoordin0saur wrote: post #29

This URL is blocked by my company's network because of a certain substring in the URL lol

A classic case of the Scunthorpe problem: https://en.wikipedia.org/wiki/Scunthorpe_problem

In this case the substring is part of the author's name. Such names are not at all uncommon.

Re: How Postgres stores data on disk – this one's a page turner

#35

Post by davidmurdoch »

crngefest wrote: post #33
indoordin0saur wrote: post #29

This URL is blocked by my company's network because of a certain substring in the URL lol

What why ? Would people browse bigcocks.net unless it’s explicitly blocked? What about cox? Is „tube“ on a blocklist as well?

Because a sales person at some "security" company convinced a CTO that it was as good idea.

Re: How Postgres stores data on disk – this one's a page turner

#36

Post by drewsberry »

llimllib wrote: post #21

@drewsberry: I wish you had an RSS feed! I tried to subscribe to your blog but if there is one it's not linked. (Enjoyed the post)

Thanks for the feedback, I really appreciate it :-) I've added the RSS feed to my home page now, as the other poster noted the URL is https://drew.silcock.dev/rss.xml.

Re: How Postgres stores data on disk – this one's a page turner

#37

Post by actionfromafar »

davidmurdoch wrote: post #35
crngefest wrote: post #33

Earlier quoted context omitted.

What why ? Would people browse bigcocks.net unless it’s explicitly blocked? What about cox? Is „tube“ on a blocklist as well?

Because a sales person at some "security" company convinced a CTO that it was as good idea.

The security company just follows "best practices" :)

Re: How Postgres stores data on disk – this one's a page turner

#40

Post by forinti »

indoordin0saur wrote: post #29

This URL is blocked by my company's network because of a certain substring in the URL lol

I once worked for a company that blocked Cuban sites because of .cu (which is the Portuguese word for the end of your digestive system), but did not block porn sites (or so I was told ;-).
Post reply on HN