51–60 of 71 posts
Postgres seriously needs a columnstore backed table instead of just a rowstore. MSSQL has this and it is magic. SingleStore has it, and it is wonderful. I'm willing to give a bounty of $1000 to whoever adds that into main postgres tree. Snowflake is great as a warehouse. it's latency is shit when it comes to fast lookups and aggregates. If you can tolerate >1s api calls, that is fine. It takes forever to insert a few…
SELECT create_hypertable('public.challenge_1br', by_range('time'));
Now, enjoy your better than Snowflake query performance performance at no extra cost.
(edit: mostly offtopic observation follows) I only knew Snowflake the id selection algorithm, so was a bit confused, but googling "snowflake db" showed me this blurb and now I'm even more confused. > Snowflake enables organizations to learn, build, and connect with their data-driven peers. Collaborate, build data apps & power diverse workloads in the ...
Snowflake & Azure Synapse are competing products, if that helps.
Snowflake is an amazing Agile database that has a great ecosystem. Teradata still remains king if you want this type of cloud datawarehouse, and becomes price competitive with Snowflake if you actually use it intensively (transactions are cheaper on teradata than snowflake)
But in the end, a good managed postgresql is probably enough for 80% of the clients of Synaps/Snowflake anyway. It's just that CTO's are starting to lose technical knowledge and are more politicians nowadays
Postgres seriously needs a columnstore backed table instead of just a rowstore. MSSQL has this and it is magic. SingleStore has it, and it is wonderful. I'm willing to give a bounty of $1000 to whoever adds that into main postgres tree. Snowflake is great as a warehouse. it's latency is shit when it comes to fast lookups and aggregates. If you can tolerate >1s api calls, that is fine. It takes forever to insert a few…
This is the most disrespectful thing I've read on HN.
Earlier quoted context omitted.
That was my first thought as well — like who is the audience for this? My first thought was that you could stay on Postgres and save that $100 by using the secret power of Open Source. Well said.
Seems like the title would be more accurate specifying postgres to snowflake In the end, there's companies paying to use snowflake, & despite what one may believe they aren't price oblivious. Having their application in postgres is a cost optimization, but then still relying on snowflake for data warehouse integrations
I love snowflake but their pricing is, in fact, absurd.
It's amusing how much effort is put into ETL these days. I remember when ETL departments were filled with the sludge of the programming world. It took ETL departments weeks to generate a CSV, and it would inevitably have massive numbers of errors because they didn't actually follow the format that was specified on the form they forced everyone to use.
Why not just set up a replica pgsql, then break the connection and upgrade it to primary? It's amusing how much effort is put into ETL these days. I remember when ETL departments were filled with the sludge of the programming world. It took ETL departments weeks to generate a CSV, and it would inevitably have massive numbers of errors because they didn't actually follow the format that was specified on the form they…
Engineers, on average, want to be challenge, which impacts options chosen.
There may also have been other factors, not mentioned here.
How about 11b and a horrible python script over to parquet to your wee little NAS for your homelab?
I'm confused. Everything sounds very expensive to me. The last table which compares it with the other vendors is surprising. Even Stich Data (cheapest) costs $1 to move 240K records: (1B / 4,166.67 = 240K). Is this real? So, their solution costs $1 to process 13.6M records. Sounds like this is not very share-worthy. What I'm missing here?
Welcome to 2024 and the generation of developers raised in the cloud native world who think this is normal. A billion rows is nothing and having $100 appear in conjunction with that is absurd unless you are doing some kind of really heavy compute or AI model training on that data. By 2030 we’ll have those costs well up over a thousand dollars and it’ll take five or six separate SaaS systems wired together to do this.…
Earlier quoted context omitted.
That was my first thought as well — like who is the audience for this? My first thought was that you could stay on Postgres and save that $100 by using the secret power of Open Source. Well said.
Based on my experience working with Postgres users since a decade (ex-Citus/Microsoft), I don't think Postgres is there yet to support every possible workload - ex: medium to larger scale (ex:1TB+) workloads in Real Time analytics, Data Warehousing, Search etc. Sure at smaller scales, it is very versatile to support any workload. That is why it is super common for companies to complement Postgres with other data stor…
No system targets every possible workload. If that’s your goal, change your goal to make it more achievable.
With work (e.g. a cluster of Postgres instances), you can do quite a lot with Postgres if you think it through. If you want an out of the box solution to a specific problem, it may make sense to move away from Postgres. Of course another option is just to change the overall design/deployment of your Postgres cluster.
At some point you’ll need to do some custom work. Unless you plan on moving away from your new system next year because it doesn’t cover every possible workload.