Snowflake is expensive if not monitored properly, on top of that they provide minimal observability. There are some good features like auto suspend and auto resume for cost savings but still there is scope of optimisations. For ex, they will charge you for minimum 1 minutes even if your query is running only for 2 seconds.
Why is Snowflake so expensive
131–140 of 217 posts
Re: Why is Snowflake so expensive
#132Snowflake is so expensive that Capital One has developed a toolkit for managing your instance.
Re: Why is Snowflake so expensive
#133Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…
Wow their statement about not participating in benchmarking wars is alarming. In this day and age, when benchmarking tools are so inexpensive and almost everything is very transparent, why not participate. Or even better engage with a neutral third party such as Jepsen to get on an even playing field and duke it out.
[deleted]
Re: Why is Snowflake so expensive
#134Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…
Wow their statement about not participating in benchmarking wars is alarming. In this day and age, when benchmarking tools are so inexpensive and almost everything is very transparent, why not participate. Or even better engage with a neutral third party such as Jepsen to get on an even playing field and duke it out.
Because their value prop isn't being #1 on benchmarks. It's about
* being easy to manage * being able to scale up and down compute so you can get good performance without having to keep a bunch of machines running.
Re: Why is Snowflake so expensive
#135Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…
It's a terrible article. The author misunderstands competition and how much it drives products in this area. Snowflake is incentivized to make their product better on every dimension. If Snowflake don't improve, customers will leave in droves - like when they moved to Snowflake . In practice, as has been pointed out in other comments, they do improve their performance (for competitive reasons) and it does cost them m…
I don't think it misunderstands business competition. In fact it understands the concept of competition very well, and develops an insightful critique into the perverse incentives that are borne from competition.
It benefits no one except for a couple thousand people to so blatantly play their customers in this way. In fact, it's worse, as it incentivizes that same behavior of other market actors in the space.
Re: Why is Snowflake so expensive
#136Earlier quoted context omitted.
It's a terrible article. The author misunderstands competition and how much it drives products in this area. Snowflake is incentivized to make their product better on every dimension. If Snowflake don't improve, customers will leave in droves - like when they moved to Snowflake . In practice, as has been pointed out in other comments, they do improve their performance (for competitive reasons) and it does cost them m…
I don't think it misunderstands business competition. In fact it understands the concept of competition very well, and develops an insightful critique into the perverse incentives that are borne from competition. It benefits no one except for a couple thousand people to so blatantly play their customers in this way. In fact, it's worse, as it incentivizes that same behavior of other market actors in the space.
What exactly in the article suggest the author understands the pressure of competition on incentives?
The author states that Snowflake are not incentivized to increase performance due to short term revenue concerns but doesn't mention they are also incentivized to do the opposite from a competitive perspective. The result is incomplete enough that it ends up being flat wrong with respect to the behavior that the company actually engages in.
The author missed the fact Snowflake did the very thing he/she suggested they were incentivized not to do, recently, at a cost of $97 million. The CEO explained why they are doing it and how they are actually incentivized. I don't know how the article could miss the mark by more than it has. The company literally does the opposite of what he/she suggested.It's not like they are the only one either, AWS has a history of reducing prices. Why? Once again, competition.
Re: Why is Snowflake so expensive
#137I don't know if this is the case at Snowflake, but there are similar seemingly misaligned incentives with CircleCI's build-seconds-based pricing model. However, the generally accepted wisdom there was that improving performance had always led to more builds being run - and so still come out as a net-positive. This had happened a bunch of times as we upgraded CPUs or storage drivers or the version - there'd be a short…
I work at Circle, but not on this specifically, and echo the same experience. This (https://en.wikipedia.org/wiki/Khazzoom–Brookes_postulate) was cited last week in a meeting that I was in, for example.
Re: Why is Snowflake so expensive
#138I'm of the mind that Snowflake and Databricks are losing their value prop now that Delta Lake is open source and Iceberg is maturing. What's to stop me from rolling my own Spark clusters and just using one of those? Is anyone doing this?
> What's to stop me from rolling my own Spark clusters and just using one of those? Is anyone doing this? Ops. Unless your core competency is running reports and spark nodes, it's probably cheaper to outsource the management of Spark and friends than to hire people to make sure it's always up and running. To be fair I haven't touched Spark in many years but having to page someone who was good enough to spark to debug…
[Disclaimer: Databricks employee] There's also a lot of value in DBSQL, Unity catalog (data management), and serverless for autoscaling that can all save money in terms of just running raw Spark. But if you want to operate Spark yourself, cool do it. We're happy for that, it builds the base of Spark committers over time and increases the quality of our products.
Re: Why is Snowflake so expensive
#139I predict[0] we'll see more people choosing Clickhouse over Snowflake in the next 5 years. Clickhouse will get reasonably feature compatible with Snowflake and give people a better escape hatch if they want to self-host their data stack. Clickhouse, Inc is building a cloud product that abstracts away the complexity and there's already companies like Altinity that will spin up a cluster for you in minutes. 0 - https:/…
Isn't Clickhouse a hosted SQL DBMS? Not really comparable to a cloud data lake. Snowflake/Databricks scales infinitely across cloud object stores like S3. Clickhouse is run as a single (or sharded) process that uses the local file system like any other SQL database, and requires volume provisioning as your data scales. It also has a fixed run cost (EC2 or wherever it's hosted) versus an "on-demand" model where read c…
ClickHouse only has fixed run cost if you configure it that way. We run ClickHouse clusters in AWS / GCS using block storage in our cloud platform. You can scale VMs up and down vertically in minutes, and scale horizontally in the same amount of time. The model works great for SaaS use cases that require constant response at all times and scale over days or weeks rather than minutes. Real-time analytic apps that show tenant dashboards or generate recommendations for users on ecommerce sites have this characteristic.
I don't think there's really a right or wrong answer here, just trade-offs.
Disclaimer: I work on Altinity.Cloud, a platform for managed ClickHouse
Re: Why is Snowflake so expensive
#140Earlier quoted context omitted.
I don't think it misunderstands business competition. In fact it understands the concept of competition very well, and develops an insightful critique into the perverse incentives that are borne from competition. It benefits no one except for a couple thousand people to so blatantly play their customers in this way. In fact, it's worse, as it incentivizes that same behavior of other market actors in the space.
What exactly in the article suggest the author understands the pressure of competition on incentives? The author states that Snowflake are not incentivized to increase performance due to short term revenue concerns but doesn't mention they are also incentivized to do the opposite from a competitive perspective. The result is incomplete enough that it ends up being flat wrong with respect to the behavior that the comp…
> The CEO explained why they are doing it and how they are actually incentivized.
The CEO explained why he thinks it's a good long term plan... but for now, they get money i.e. are actually incentivized by slow code. The CEO's incentives are theoretical ones.
And the market, which ultimately control whether the CEO gets to continue that plan or not, did not seem to agree it was a good plan.