Moving a billion Postgres rows on a $100 budget
blog.peerdb.io
Moving a billion Postgres rows on a $100 budget
1–10 of 71 posts
Re: Moving a billion Postgres rows on a $100 budget
#2How about 11b and a horrible python script over to parquet to your wee little NAS for your homelab?
Re: Moving a billion Postgres rows on a $100 budget
#3If someone is price conscious, why move from postgres to snowflake?
Re: Moving a billion Postgres rows on a $100 budget
#4If someone is price conscious, why move from postgres to snowflake?
That was my first thought as well — like who is the audience for this?
My first thought was that you could stay on Postgres and save that $100 by using the secret power of Open Source.
Well said.
Re: Moving a billion Postgres rows on a $100 budget
#5If someone is price conscious, why move from postgres to snowflake?
Its not about being price conscious, its about finding an efficient way to replicate data into your data warehouse.
Re: Moving a billion Postgres rows on a $100 budget
#6If someone is price conscious, why move from postgres to snowflake?
Its not about being price conscious, its about finding an efficient way to replicate data into your data warehouse.
OP here, thanks for chiming in. Yep, the blog talks about how efficient you can be while replicating data from Postgres to Data Warehouses. Also just for some context, the inspiration behind writing this blog is a common concern from customers that data-movement is expensive and also the insanely crazy margins in this space.
Re: Moving a billion Postgres rows on a $100 budget
#7(edit: mostly offtopic observation follows)
I only knew Snowflake the id selection algorithm, so was a bit confused, but googling "snowflake db" showed me this blurb and now I'm even more confused.
> Snowflake enables organizations to learn, build, and connect with their data-driven peers. Collaborate, build data apps & power diverse workloads in the ...
Re: Moving a billion Postgres rows on a $100 budget
#8Does the elastic license fall under open source as defined by OSI?
Re: Moving a billion Postgres rows on a $100 budget
#9If someone asked me how much it would cost to move a billion Postgres rows, I would say "probably under $100."
I just had to move 500 million "rows" into S3, and it came in at about $100. I would expect S3 to be more expensive.
Re: Moving a billion Postgres rows on a $100 budget
#10How about 11b and a horrible python script over to parquet to your wee little NAS for your homelab?
now we're talking about innovation