I have a Postgres database with a few terabytes of data. I need to process a large quantity of data (terabytes) which will involve queries/upsertions into the database.
How do you best do this and finish in a reasonable amount of time (1-2 weeks)?
For reference, I have 64GB of RAM and working with several 5TB drives (a mix of NVMe, SSD and spinning discs).