[Disclaimer: I work for a company that offers a Snowflake Cost Optimizer product]
We’re an open-source monitoring & alerting tool and many of our users were using it to set alerts on their warehousing (Snowflake) costs. The problem with Snowflake is particularly worse due to its lack of query level attribution of costs and no in-built features for monitoring or recommendations on improvements. We’re building a Snowflake Cost Optimizer (https://www.chaosgenius.io/snowflake-cost-optimizer.html) and are hearing the same feedback from our customers as the author mentions. Snowflake is definitely coming up with features towards better cost transparency but I wonder if it’s too little too late.
Why is Snowflake so expensive
101–110 of 217 posts
Re: Why is Snowflake so expensive
#102I disagree with the assertion that Snowflake has no incentive to improve performance. While I don’t work for Snowflake, I work for a competitor and we’re constantly looking to improve performance to make customers happy. For the exact reason that the article claims Snowflake wouldn’t innovate, I’d assert that they would. If they are expensive and slow, and a competitor is faster and cheaper, eventually they will see…
Could you say more about the relative market position of your two companies?
I don't know the market at all, but Snowflake is certainly large and successful (IPOed in 2020, $50bn market cap). I could readily imagine that a company doing so well might not feel the incentive to improve very strongly. Or that they might see themselves more as a sales/marketing-led company than one where technical quality is a key driver. Whereas you folks as a challenger would have a lot more incentive to differentiate yourselves.
Re: Why is Snowflake so expensive
#103Earlier quoted context omitted.
Those app rental scooters that are littered around city centers: you pay for distance as well as for time. And that's why they don't go very fast.
Also because going fast on those things is fucking dangerous, especially when (like most people riding them) you're not wearing a helmet.
And have been drinking (it's what I saw a lot of)
Re: Why is Snowflake so expensive
#104Alright I’ll bite finally. What do these companies do? Neither Snowflake’s front-facing website, nor the Wikipedia article, nor this post tell me why people pay all this money. I know a bit about the effort involved in chucking around 100 petabyte datasets, and there are numerous niches a SaaS could fill in there, but it’s very murky from the outside.
I was wondering the same thing. This sums up pretty good I guess:
> The best way to describe Snowflake is that it is a brute force method to run complex queries without creating indexes.
Re: Why is Snowflake so expensive
#105Earlier quoted context omitted.
At a previous startup, we dumped CircleCI and switched to Jenkins on our own EC2 instance. We had a lot less problems. (This was way back in 2016, I'm sure things have improved now.)
Yup! I ended up doing TeamCity over Jenkins, but they do the same thing. Amazing how fast a 32C / 64T EYPC server in my basement can be..
I can only imagine! I have a 3950X here (16C / 32T) with 128 gigs of RAM and it is incredible. Total overkill for a home lab though.
Re: Why is Snowflake so expensive
#106> Snowflake has no incentive to push a code change that makes things 20% faster because that can correspond to 10–20% drop in short-term revenue.
If they improve performance they can lower the cost to customers, which will make the product more attractive to prospective customers. But if they are already swimming in cash they may not feel the need to gain more customers.
Only threats prompt companies to improve things. Threat of a competitor, threat of losing all their money, threat of bad PR, threat of regulation, threat to the stock price, etc.
I see this every day in companies that don't care about managing their cloud costs. They waste money like crazy because they literally don't care if they lose money, because some exec doesn't care, or they got enough funding until the next round, etc. A couple years later another exec asks why the CISO/CTO is spending so much money without any ROI, and then everybody has to stop everything they're doing to shave pennies off cloud costs.
Companies run by individual executives are insane. I don't understand why people allow companies to be run this way. I think a co-op where employees could be active participants in the running of the company would allow for more sane decision-making.
Re: Why is Snowflake so expensive
#107Alright I’ll bite finally. What do these companies do? Neither Snowflake’s front-facing website, nor the Wikipedia article, nor this post tell me why people pay all this money. I know a bit about the effort involved in chucking around 100 petabyte datasets, and there are numerous niches a SaaS could fill in there, but it’s very murky from the outside.
I was wondering the same thing. This sums up pretty good I guess: > The best way to describe Snowflake is that it is a brute force method to run complex queries without creating indexes. ( https://news.ycombinator.com/item?id=32554072 )
Column stores on DFS are without a doubt tricky beasts. It’s a very rich field technically.
I guess I’m trying to get a read on whether their core competency / moat is distributed columnar query technology or sales/support/marketing.
Re: Why is Snowflake so expensive
#108Great article. On the surface, it's about Snowflake. At a deeper level, the article is about the perverse incentives motivating SaaS businesses to do seemingly dumb, inefficient things and avoid seemingly obvious optimizations by default. Many SaaS businesses are perfectly happy to let customers shoot themselves in the foot if it generates more revenue. The BigQuery example (presently, by default, `select * from tabl…
> As the article so well puts it, every SaaS company has a vested financial interest "to leave optimization gremlins in."
It depends on the time scale. A SaaS optimizing for, say, a 1-3 year financial return will see their interests through a different lens than one optimizing for a multi-decade return. Leaving optimization gremlins in isn't aligned with customers' interests in the long run, so the customers will eventually find alternatives if the SaaS doesn't eventually align itself with customers.
Re: Why is Snowflake so expensive
#109Earlier quoted context omitted.
Those app rental scooters that are littered around city centers: you pay for distance as well as for time. And that's why they don't go very fast.
Also because going fast on those things is fucking dangerous, especially when (like most people riding them) you're not wearing a helmet.
Absolutely. I don't have a problem with most scooter owners, but here in a city with a lot of tourism, the rental scooters are often a menace. I was waiting outside of a restaurant and took half a step back to let some people through. I brushed against something moving fast, and it was some tall yahoo going downhill at max speed on a scooter. If I'd taken a full step back, somebody would have needed medical treatment.
Learning how to ride one of those in a city takes time, practice, and thought. Which you will surely get if you buy one. But apparently not so for the rentals.
Re: Why is Snowflake so expensive
#110From what I can tell, the author is incorrect about the example given in "Optimizer gremlins". I tested an example on my own data and micro-partition pruning was active.
The issue with dbt models in Snowflake is that if you ever perform a full-refresh and don't sort it, you ruin any natural clustering that arises from an incremental model. I've run into this issue many times. Auto-clustering gets too expensive at scale and Snowflake doesn't give you much guidance on alternatives.