Earlier quoted context omitted.
Segment (and any other SaaS that wants to own the event firehose) has pricing issues as the customer scales. As a customer you are charged on data volume[0], but your data grows at the rate = (Users x Engagement X Features X Data Sources). That rate can easily be the square of your revenue growth. This pricing issue makes it easy for Segment's fast-growing customers to justify an in house replacement. This puts huge…
So what’s the end state solution? The tools change and/or mature slower than a fast company will grow. Where are you going from here? Are you eventually running your own cluster? Or, are you paying per Gb per second how GCP bills with decoupled compute / memory / storage? Or maybe with a little bit more overhead with the orchestration cost on top of consumption with how azure bills for ADF. Just from reading blog pos…
The “how” question has changed every 3 years in the last 10 that I’ve done data work. That trend is driven primarily by data scale and partially that analytics and ML access patterns have evolved a lot.