Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Snowflake so expensive

blog.devgenius.io

Post reply on HN
Print view View original

171–180 of 217 posts

Re: Why is Snowflake so expensive

#171

Post by danielmarkbruce »

morelisp wrote: post #162
danielmarkbruce wrote: post #147

Earlier quoted context omitted.

By this reasoning, everyone would shirk at work. If you think incentives only act over short time horizons, I don't know how you explain an enormous amount of human behavior. The market didn't even understand it. Most of the people trading equities, especially around earnings announcements, don't know what a data warehouse is or what matters in that market. All they saw was "miss".

I didn't say the CEO was wrong or that long-term thinking is bad! I said the actual incentives are still misaligned. (I mean, a lot of people do shirk at work, and it even works out well for them.) I think you have a weird and probably not useful definition of "actual" if "monthly revenue" is not actual but "projected monthly revenue two years from now" is actual. (Or maybe I've just lived in Germany too long.)

You are right, I've used the word "actual" incorrectly. What I should have said was "net". Ie, both short term and long term revenue incentivize behavior and in this case the net result was increasing performance, ie long term incentive > short term incentive.

Re: Why is Snowflake so expensive

#173

Post by simo7 »

danielmarkbruce wrote: post #124
cs702 wrote: post #7

Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…

It's a terrible article. The author misunderstands competition and how much it drives products in this area. Snowflake is incentivized to make their product better on every dimension. If Snowflake don't improve, customers will leave in droves - like when they moved to Snowflake . In practice, as has been pointed out in other comments, they do improve their performance (for competitive reasons) and it does cost them m…

The main flaw of the article is not controlling for product category.

I suspect most data warehouses have similar NDRs.

In many companies a data warehouse is the place where you dump all your data and let everyone run poorly written programs against it.

Add to that poor engineering culture in data teams (often lead by non-technical people) and costs are bound to skyrocket.

Re: Why is Snowflake so expensive

#175

Post by lokar »

deepGem wrote: post #31
cs702 wrote: post #7

Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…

Wow their statement about not participating in benchmarking wars is alarming. In this day and age, when benchmarking tools are so inexpensive and almost everything is very transparent, why not participate. Or even better engage with a neutral third party such as Jepsen to get on an even playing field and duke it out.

Benchmark results rarely predict actual application perf. You need to run your own queries against your own data. Do a real POC.

Re: Why is Snowflake so expensive

#176

Post by buremba »

I believe they need to focus on the performance at least nowadays because both Databricks & BigQuery are also great products and they push Snowflake in terms of feature-parity and performance.

That being said, Snowflake is also pushing for the marketplace model where you publish your app natively to move your code where the customers environment is. If they become successful, the performance might not be the one of the incentives for the companies to go with Snowflake and the switching cost might be higher as companies will move more of their business logic embedded in the system.

Re: Why is Snowflake so expensive

#177

Post by AdamProut »

danielmarkbruce wrote: post #124
cs702 wrote: post #7

Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…

It's a terrible article. The author misunderstands competition and how much it drives products in this area. Snowflake is incentivized to make their product better on every dimension. If Snowflake don't improve, customers will leave in droves - like when they moved to Snowflake . In practice, as has been pointed out in other comments, they do improve their performance (for competitive reasons) and it does cost them m…

We regularly benchmark the "big 3" Cloud Data warehouses - Redshift, Snowflake and Big Query at SingleStore. Their performance is very close to the same (within 10-20%) on most benchmarks on reasonable sized data sets (10s of TB).

I agree if the performance of one of them fell behind the others for any prolonged period of time the cost to the laggard in market share would be much much worse then short term revenue gain of "being slow on purpose".

Re: Why is Snowflake so expensive

#178

Post by idunno246 »

joelthelion wrote: post #122
Croftengea wrote: post #104

Earlier quoted context omitted.

I was wondering the same thing. This sums up pretty good I guess: > The best way to describe Snowflake is that it is a brute force method to run complex queries without creating indexes. ( https://news.ycombinator.com/item?id=32554072 )

But why not create indexes? I mean, I understand why sometimes you're you don't want an index. But building an entire warehouse around the idea of "no indexes", really ?

these tend to be for one-off analytical queries. you want ever user with flag X >10 joined against five other tables each with similar filters. you don't know ahead of time what that query is, your analyst thought of it this morning, so you cant make indices ahead of time. and itll never run again so you don't need to take the performance hit keeping an index. and someone has to decide which indices to keep, but app engineers arent best utilized figuring out indices for analysts.

the indices is nice, but the bigger selling feature for me is if you have many services, and each services data are in the warehouse, you can join against them all together.

Re: Why is Snowflake so expensive

#179

Post by tomnipotent »

PaulWaldman wrote: post #41
beoberha wrote: post #32

I disagree with the assertion that Snowflake has no incentive to improve performance. While I don’t work for Snowflake, I work for a competitor and we’re constantly looking to improve performance to make customers happy. For the exact reason that the article claims Snowflake wouldn’t innovate, I’d assert that they would. If they are expensive and slow, and a competitor is faster and cheaper, eventually they will see…

Chrun for these services take a long time. They are "sticky" and have the baggage of enterprise agreements. With the switching costs never being zero, if SLAs are being met, it's exceedingly difficult to switch vendors. Alternatively there is a faster impact on new sign-ups when falling behind competitors on costs and benchmarks.

> have the baggage of enterprise agreements

Snowflake let's you roll into pay-as-you-go after a contract expires.

Re: Why is Snowflake so expensive

#180

Post by buttaphingas »

joelthelion wrote: post #122
Croftengea wrote: post #104

Earlier quoted context omitted.

I was wondering the same thing. This sums up pretty good I guess: > The best way to describe Snowflake is that it is a brute force method to run complex queries without creating indexes. ( https://news.ycombinator.com/item?id=32554072 )

But why not create indexes? I mean, I understand why sometimes you're you don't want an index. But building an entire warehouse around the idea of "no indexes", really ?

It's all around the ethos of ease of use. Snowflake does a lot of smarts in the background so that you don't have the overhead of managing indexes. And not just indexes, there is just less human intervention required overall compared to something like Teradata or even a modern lakehouse.

That said, they've kind of introduced it with the Search Optimization Service, which is like an index across the whole table for fast lookups, but even that is automatically maintained in your behalf.

Post reply on HN