Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

twitter.com

Post reply on HN
Print view View original

221–227 of 227 posts

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#221

Post by Macha »

johnvanommen wrote: post #145
coinbasethrow wrote: post #90

I worked at Coinbase until very recently and can confirm this is Coinbase They paid upfront for 3 years of usage, and yes they were burning > $20m/year on datadog

Back in the day I was auditing our support contracts for a place that I worked at. Basically figuring out if we were getting what we were paying for. My favorite "overpriced support contract" was for an Oracle product. The cost of support was seven figures, and in the entire year a single phone call had been placed to support.

The best bit when I worked at one of those companies that had an expensive Oracle contract was this dynamic:

1. Can we use MySQL for ?

No, use Oracle, we have a support contract

2. Can we call that support contract in now?

No, let's try the inhouse expertise first.

3. Can we check if they have any better solutions?

No, it's "solved" now

Like, what were we paying for? I have to assume there's per-engagement costs as well as the ongoing costs, given how hestitant our contract owning team were to let us anywhere near Oracle.

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#222

Post by dineshkumar_cs »

monero-xmr wrote: post #2

What is best cost effective and decent alternative to DD in 2023? I also feel like they are robbing me blind. Great product though.

Use Prometheus or Influx for storing metrics, ELK or Opensearch or loki for logs, Grafana for visualization, and Jaeger for tracing.

You would need a team to configure this setup and make it right over time. It's worth the investment instead of paying a cloud market leader.

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#223

Post by AsmodiusVI »

salynchnew wrote: post #200
paulddraper wrote: post #110

Earlier quoted context omitted.

Datadog costs more to monitor your AWS t3.medium instance than the actual instance. I asked them how them can justify that. They recommended I use "modern infrastructure" which means Docker.

(Disclaimer: I work at Chronosphere, a Datadog competitor) This is a big issue in the observability space. We have written a few blog posts on this, but basically it’s easy to fall into a trap where cardinality and high dimensional monitoring causes your metrics to pop, causing costs to skyrocket. You have a few experiments, are running a bunch of smaller k8s pods per cluster and whoosh! you might be looking at milli…

Some of the vendors in the space are absolutely going in on OTel. DD seems to be working actively against it.

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#224

Post by abnerg »

Gergely Orosz dug into this a bit more and surfaced quite a build your own vs renew vs renew with highly negotiated terms. (https://blog.pragmaticengineer.com/datadog-65m-year-customer...)

A lot of this discussion reminds me of this talk:"Netflix built its own monitoring system - and why you probably shouldn't" (https://www.infoq.com/presentations/netflix-monitoring-syste...) where Roy Rappport describes Netflix as a "monitoring system that happens to stream movies"

As someone who spent a few years at New Relic and Lacework, I can also say that pricing observability fairly is crazy hard when you account for different architectures, usage pricing, and the humans experience the value.

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#225

Post by ashwaganda »

benatkin wrote: post #142
paulddraper wrote: post #110

Earlier quoted context omitted.

Datadog costs more to monitor your AWS t3.medium instance than the actual instance. I asked them how them can justify that. They recommended I use "modern infrastructure" which means Docker.

Sounds like New Relic 2.0

not even close. NR doesn't bill overages, doesn't do per-host. we switched and saved tens of thousands monthly

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#226

Post by pranay01 »

ricardobeat wrote: post #85
ren_engineer wrote: post #71

OpenTelemetry is going to be an existential threat to DataDog and other companies that effectively rely on vendor lock-in to exploit customers. Not sure how companies rationalize these types of services at scale when there are so many open source options to run for a fraction of the cost. You could hire 100+ engineers and still save money compared to a 65M bill https://news.ycombinator.com/item?id=34540419

I’ve seen an attempted move from DD to OT and it was a nightmare of undocumented features and little compounded issues. Tracing was non functional. It doesn’t seem mature enough yet.

I am not sure when you tried OpenTelemetry, but it is decently mature now, esp. for tracing. I am a maintainer at SigNoz (https://github.com/signoz/signoz) and we have good support for tracing using Otel for most of the common frameworks.

I agree it was a bit rapidly evolving in early days, but now its much more mature.

You can check out our docs for distributed tracing here - https://signoz.io/docs/instrumentation/

Re: A cryptocurrency company had a $65M bill, per Datadog’s Q1 earnings call

#227

Post by salynchnew »

api wrote: post #89
ren_engineer wrote: post #71

OpenTelemetry is going to be an existential threat to DataDog and other companies that effectively rely on vendor lock-in to exploit customers. Not sure how companies rationalize these types of services at scale when there are so many open source options to run for a fraction of the cost. You could hire 100+ engineers and still save money compared to a 65M bill https://news.ycombinator.com/item?id=34540419

> You could hire 100+ engineers and still save money compared to a 65M bill I see cloud costs like this a lot and it really puzzles me. It seems like people would rather pay 10X+ more to just not have to think about it than even to hire other people to think about it, because then you have to think about hiring and HR. "Here's a blank check. Just make it go away." Of course corporate consultants run on that, so I gue…

FWIW, trying to hire 100+ engineers now is probably a lot easier than it would have been in early/mid 2021....
Post reply on HN