Anyone else spending around this much?
How we saved money by replacing Mixpanel with BigQuery and K8S
41–50 of 70 posts
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#42This article epitomizes the old adage: software engineers are terrible at estimating. I have absolutely no stake in Mixpanel, and it has its own flaws, but seeing something as misleading as this on the front page of HN means I have to write a clarifying, if not somewhat edifying, comment =/ 1. Right off the bat, there's the cost of building all of this + maintaining it. Something like Mixpanel at scale requires at le…
I think you are not counting: QA, user experience, DevOps, technical writing, security auditing, user training, integration support... What happens if the system collecting all of your data has vulnerability? suddenly all of your apps and data are owned. What if it's inefficient? then you have DDoSed yourself. Non-functional requirements cannot be solved with a functional requirement mindset. What if you are losing d…
Having built something like this myself the components you actually need to write are so small and simple the scope for security issues is minimal. All the infra scales automatically.
Also, this post only addresses data flowing in one direction into BQ. Presumably they're then using something like Tableau or the Google/AWS offerings to build dashboards based on this data. There aren't many places for bugs to hide unless the BQ perms are wide open.
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#43The "backend" would have been a perfect problem to be solved by Google's Cloud Functions (i.e. serverless), rather than a Kubernetes cluster with Node and nginx.
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#44I am using appengine flex and google compute instances. Do containers make your setup more cost-effective?
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#45There's one other "big hammer" for saving money at MixPanel: "send fewer events".
We saved 60% (which was no small amount of money) by realizing that sending more than 1 of an event per user per hour was: A) Not necessary B) Not even something you could query in MixPanel.
more thoughts in: https://blog.ratelim.it/blog/how-to-save-money-on-event-trac...
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#46Does anyone have any experience of negotiating your costs down with Mixpanel? Years ago I remember my CTO ringing up our hosting provider and in the space of a phone call bringing some of our costs down 90%.
Definitely can negotiate, but it’s not going to be 90%. MixPanel is pretty solidly enterprise. At this level of spend they’re sending a team of a couple implementation people onsite and price is def negotiated.
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#47Did you not consider snowplow analytics with something like periscope over the top?
Thanks for the mention! We are working on our port of Snowplow to run cloud natively on GCP currently - you can see our RFC here:
https://discourse.snowplowanalytics.com/t/porting-snowplow-t...
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#48Why not use App Engine instead of Container Engine? You could likely save more that way.
What? Order of pricing cheap to expensive is GKE, GCE or Container Engine, App Engine.
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#49Earlier quoted context omitted.
I think you are not counting: QA, user experience, DevOps, technical writing, security auditing, user training, integration support... What happens if the system collecting all of your data has vulnerability? suddenly all of your apps and data are owned. What if it's inefficient? then you have DDoSed yourself. Non-functional requirements cannot be solved with a functional requirement mindset. What if you are losing d…
Having built something like this myself the components you actually need to write are so small and simple the scope for security issues is minimal. All the infra scales automatically. Also, this post only addresses data flowing in one direction into BQ. Presumably they're then using something like Tableau or the Google/AWS offerings to build dashboards based on this data. There aren't many places for bugs to hide unl…
[deleted]
Re: How we saved money by replacing Mixpanel with BigQuery and K8S
#50Just to flesh out the vendor comparison... Five weeks of, say, three engineers with $150k salaries ($200k fully loaded) is $60k. People say that the majority of software is after initial development. Let's say 20% is development [0]. That means this project costs $300k total and lives for 5 years before the team decides to rewrite it. That brings us to $60k / year. This is good savings! My general guidance to folks t…
> There's a reason MixPanel (and any other vendor) has so many engineers. You're paying them to have already made the mistakes that you're about to make. The real reason Mixpanel (et al) have so many engineers is: A) They need to scale their service to reliably provide to N corporations which is frequently non-trivial. B) They need to implement the 80% of features that aren't critical path for Company A because some…
The number of engineers they have is irrelevant to the point. The point is you pay someone to worry about a specific thing so you don't have to and (hopefully) their expertise leads to a better product than you could have built.