Live data from Hacker News

Postgres Postmaster does not scale

recall.ai

91–94 of 94 posts

Re: Postgres Postmaster does not scale

#91

Earlier quoted context omitted.

Your arguments make sense for concurrent queries (though high-latency storage like S3 is becoming increasingly popular, especially for analytic loads). But transactions aren't processing queries all the time. Often the application will do processing between sending queries to the database. During that time a transaction is open, but doesn't do any work on the database server.

It is bad application architecture. Database work should be concentrated in minimal transactional units and connection should be released between these units. All data should be prepared before unit start and additional processing should take place after transaction ended. Using long transactions will cause locks, even deadlocks and generally should be avoided. That's my experience at least. Sometimes business transa…

I'm talking about relatively short running transactions (one call to the HTTP API) which load data, process it in the application, and commit the result to the database. Even for those a significant portion of the time can be spent in the application, or communication latency between db and application. Splitting those into two shorter transactions might sometimes improve performance a bit, but usually isn't worth the complexity (especially since that means the database doesn't ensure that the business transaction is serializable as a whole).

For long running operations, I usually create a long running read-only transaction/query with snapshot consistency (so the db doesn't need to track what it reads), combined with one or more short writing transactions.

Re: Postgres Postmaster does not scale

#92

Earlier quoted context omitted.

You forgot the "sudo" before "tee" > write stdin to a file that doesn't also write it to stdout You mean like "dd of=/path/file" ?

I physically/literally squinted when I saw disk destroyer. I know it's useful for other things, but it has become a fearful instinct at this point.

If my memories serves me right it was meant to be "Copy Convert" but "cc" was already taken for "C Compiler" so they went to the next letter in the alphabet (alpha beta), hence "dd". Thanks for listening to my TED talk of useless and probably false information :)

edit: i will be stealing disk destroyer, though

Re: Postgres Postmaster does not scale

#93
post #92

Earlier quoted context omitted.

I physically/literally squinted when I saw disk destroyer. I know it's useful for other things, but it has become a fearful instinct at this point.

If my memories serves me right it was meant to be "Copy Convert" but "cc" was already taken for "C Compiler" so they went to the next letter in the alphabet (alpha beta), hence "dd". Thanks for listening to my TED talk of useless and probably false information :) edit: i will be stealing disk destroyer, though

    According to Dennis Ritchie, the name is an allusion to the DD statement found in IBM's Job Control Language (JCL), where DD is short for data definition  https://en.wikipedia.org/wiki/Dd_%28Unix%29#History
Post reply on HN