71–80 of 168 posts
As much as I love seeing competition in the space and am enjoying my popcorn, I really don't understand what Databricks is doing here: this feels like a childish foodfight rather than an obsession with the customer...
This reminds me of the old performance ads of Oracle where they would show you how everything ran better on Oracle. They used to put those ads at airports, business lounges and the back cover of newspapers and magazines read by non-technical executives like the FT and Economist. Everyone technical knew they would game every environment to come out with superior results. I suppose it worked. As the top executives buy…
A key part of the Oracle strategy is making it a breach of license to publish any benchmarking data. No performance data about Oracle's database is allowed to be published without their approval, which means no negative results are published.
This is kind of understandable. Benchmarking complex software is complicated. It’s easy to give totally wrong picture of things either accidentally or deliberately.
The irony here is that what Databricks is doing to Snowflake is exactly what Snowflake did to AWS and Redshift. Same playbook - show that you’re better in a key metric that’s easy to understand (performance) to get the attention, but then pitch the paradigm change. In Snowflake’s case, that was separation of storage and compute. In Databrick’s case, it’s the Lakehouse Architecture. I think the reason why Snowflake is…
To be fair Apache Spark, which started long before either company existed, was built on the assumption that compute and storage should be separate. Unlike Hadoop, Spark did not come with any storage system and could read from any source.
Databricks was founded before Spark 1.0 released by Spark's creators.
Hadoop was created at a time when network and disk were much slower, RAM was less abundant. Bringing compute to the data made sense, but it typically doesn't anymore.
The irony here is that what Databricks is doing to Snowflake is exactly what Snowflake did to AWS and Redshift. Same playbook - show that you’re better in a key metric that’s easy to understand (performance) to get the attention, but then pitch the paradigm change. In Snowflake’s case, that was separation of storage and compute. In Databrick’s case, it’s the Lakehouse Architecture. I think the reason why Snowflake is…
In what way is lakehouse architecture beneficial over something like Snowflake or BigQuery? I understand the appeal over having lake and warehouse as separate components, but with those native cloud warehouses, you can already do everything a lake does.
With the lakehouse, you can use python, R and Scala, (not just SQL) to interface with your data. You can use multiple compute engines (spark, Databricks, presto) so you are not locked into one compute engine.
I recall being a junior programmer, and wishing I could talk to my MySQL database in python code to do some processing that was difficult to express in SQL, that day is finally here.
This reminds me of the old performance ads of Oracle where they would show you how everything ran better on Oracle. They used to put those ads at airports, business lounges and the back cover of newspapers and magazines read by non-technical executives like the FT and Economist. Everyone technical knew they would game every environment to come out with superior results. I suppose it worked. As the top executives buy…
The "Unbreakable" Marketing Campaign:
https://www.oreilly.com/library/view/the-oracle-hackers/9780...
https://www.zdnet.com/article/invincible-oracle-not-so-secur...
Databricks is trying to punch up at the market leader. Every decent marketer knows that you should never do the opposite and punch down.
Earlier quoted context omitted.
maybe clickhouse?
Clickhouse is good if you're building application. It has lot of great features and incredible performance, but there's an expectancy that people using it know what they're doing and can work around its limitations (like limited support for joins and sql in general). Something like Snowflake works much better when you're building a platform that you can give to two hundred data analysts or various skills spread over…
> broad feature set
My experience is that the feature sets of Snowflake and Databricks are very similar. Both have time travel support. Snowflake has materialized views, but Databricks has Delta Live Tables. Databricks has a distributed Pandas API, but Snowflake recently introduced Snowpark. Databricks also has autoscaling and they recently launched a serverless offering that makes autoscaling super fast aswell.
SF1. Databricks generated the TPC-DS datasets from TPC-DS kit before time started. Databricks starts time then generated all queries. Then load from S3 to Snowflake tables by - (i'm not sure about these next parts) - creating external stages and then "copy into" statements I guess? Or maybe just using copy into from an s3 bucket, that part doesnt matter much. But its not clear did they also allow target tables to be partitioned/clustering keys at all? Then all of the queries are executed 1-99 for TPCDS 100TB
Its just hard to say exactly what "They were not allowed to apply any optimizations that would require deep understanding of the dataset or queries (as done in the Snowflake pre-baked dataset, with additional clustering columns)" means exactly. Like what does that exactly mean. At a glance though, this looks very impressive for Databricks, but just want to be sure before I submit to an opinion. SF1. Databricks generated the TPC-DS datasets from TPC-DS kit before time started. Databricks starts time then generated all queries. Then load from S3 to Snowflake tables by - (i'm not sure about these next parts) - creating external stages and then "copy into" statements I guess? Or maybe just using copy into from an s3 bucket, that part doesnt matter much. But its not clear did they also allow target tables to be partitioned/clustering keys at all? Then all of the queries are executed 1-99 for TPCDS 100TB
Its just hard to say exactly what "They were not allowed to apply any optimizations that would require deep understanding of the dataset or queries (as done in the Snowflake pre-baked dataset, with additional clustering columns)" means exactly. Like what does that exactly mean. At a glance though, this looks very impressive for Databricks, but just want to be sure before I submit to an opinion.
Earlier quoted context omitted.
Databricks is a shit platform that encourages terrible data practices and accretion of technical debt.
Finally somebody that has used Databricks! I can't believe all the praise I read elsewhere in the comments here. Databricks is broken in so many ways, it is beyond me how anyone can like using this.
so, alternatives? Aside from the Azure/GCP/AWS internal offeringa I know about Snowflake and Firebolt, Databricks is new to me.
maybe clickhouse?
* Databricks pivoted from analytics to ML and it's not just marketing. Clickhouse is all about OLAP use cases.
* Clickhouse competes with Druid/Pinot/Timescale, Spark competes with Flink.