11–20 of 111 posts
Databricks broke the record by 2x) and is 10x more cost effective, in an audited benchmark. Snowflake should participate in the official, audited benchmark. Customers win when businesses are open and transparent…
Databricks broke the record by 2x) and is 10x more cost effective, in an audited benchmark. Snowflake should participate in the official, audited benchmark. Customers win when businesses are open and transparent…
Databricks and snowflake should pay an independent third party to re-run these. In-house benchmarks by either company don't count with results this different.
If you're in networking, it's throughput, latency or fairness. If you're in graphics its your shaders or polygons or hashes. If you're in CPUs its your clock speed. If its cameras, it's megapixels (but nobody talks about lens or real measures of clarity) If you're in silicon it's your die size (None of that has mattered for years, those numbers are like versions not the largest block on your die) If you're in finance, it's about your returns or your drawdowns or your sharpe ratios.
I'm a little bit surprised how seriously databricks is taking this, but maybe it's because one of the cofounders laid this claim. Ultimately what you find is one company is not very good at setting up the other company's system, and the result is the benchmarks are less than ideal.
So why not have a showdown? Both founders, streamed live, running their benchmarks on the data. NETFLIX SPECIAL!
There are also some technical terms I don't know at all, and when I've searched for them, the top results are all more Azure stuff. Like wtf is a datalake?
Databricks broke the record by 2x) and is 10x more cost effective, in an audited benchmark. Snowflake should participate in the official, audited benchmark. Customers win when businesses are open and transparent…
Audited how? If you look at the Snowflake response the numbers being posted by Databricks look outright faked or otherwise false.
The official review process is significantly more complicated than just offering a static dataset that's been highly optimized for answering the exact set of queries. It includes data loading, data maintenance (insert and delete data), sequential query test, and concurrent query test.
You can see the description of the official process in this 141 page document: http://tpc.org/tpc_documents_current_versions/pdf/tpc-ds_v3....
Consider the following analogy: Professional athletes compete in the Olympics, and there are official judges and a lot of stringent rules and checks to ensure fairness. That's the real arena. That's what we (Databricks) have done with the official TPC-DS world record. For example, in data warehouse systems, data loading, ordering and updates can affect performance substantially, so it’s most useful to compare both systems on the official benchmark.
But what’s really interesting to me is that even the Snowflake self-reported numbers ($267) are still more expensive than the Databricks’ numbers ($143 on spot, and $242 on demand). This is despite Databricks cost being calculated on our enterprise tier, while Snowflake used their cheapest tier without any enterprise features (e.g. disaster recovery).
Edit: added link to audit process doc
This is the sort of FUD testing that gets thrown back and forth between companies of all kinds. If you're in networking, it's throughput, latency or fairness. If you're in graphics its your shaders or polygons or hashes. If you're in CPUs its your clock speed. If its cameras, it's megapixels (but nobody talks about lens or real measures of clarity) If you're in silicon it's your die size (None of that has mattered fo…
Disclaimer: Databricks cofounder who authored the original blog post.
For the more technically inclined, don’t let any corporate blog post / comms piece live in your head rent-free. If you’re a customer, make them show you value for their money. If you’re not, make them provide you tools / services for free. Just don’t help them fuel the pissing contest, you’ll end up a bag holder (swag holder?).
Performance is only one part of the story. The major advantage Snowflake (and to some extent Presto/Trino) brings to the table is it's pretty much plug and play. Spark OTOH usually requires a lot of tweaking to work reliably for your workloads.
Spark does take a lot of tuning, but then I'm guessing Databricks offer that service as part of your licensing fee? (I'd hope so if they're selling a product based on FOSS code, there has to be a value add to justify it)