We're lucky to find and deploy this project early at our startup. Being a cybersecurity company, we cannot have out customers data leave our aws account.
Rudder, an open source Segment alternative
51–60 of 103 posts
Re: Rudder, an open source Segment alternative
#52So what's the pricing model? Your site lists "pricing" and shows no info. - Are you charging for support? - Do you/will you have a paid enterprise tier that will increasingly be the only tier with a viable feature set? - What's keeping you from dumping on Segment's market until you hit traction then ratcheting up to Segment's pricing? - Who are you? Who are your investors?
Pricing: Honestly, we haven't figured out the business model yet. Like other open-source products, it will likely be a combination of support + enterprise features (like HA, auto-scale etc) but again we don't know what those enterprise features would be. Ratch up Pricing: Good question and not sure how to answer. Our vision is to be like other open-source companies like Mattermost, ES. Our base version (which would w…
Thank you for being candid and straightforward in your responses. If you can get this to work really well and make it easy to deploy, I think it can be a great service to the FOSS community and also a profitable support/enterprise business for you. Good luck.
Re: Rudder, an open source Segment alternative
#53Earlier quoted context omitted.
Pricing: Honestly, we haven't figured out the business model yet. Like other open-source products, it will likely be a combination of support + enterprise features (like HA, auto-scale etc) but again we don't know what those enterprise features would be. Ratch up Pricing: Good question and not sure how to answer. Our vision is to be like other open-source companies like Mattermost, ES. Our base version (which would w…
Thank you for being candid and straightforward in your responses. If you can get this to work really well and make it easy to deploy, I think it can be a great service to the FOSS community and also a profitable support/enterprise business for you. Good luck.
Thanks :)
Re: Rudder, an open source Segment alternative
#54Is it really so difficult for engineers to create a task to process a Kafka topic? It takes one day to write a program to consume from a topic of events and push to an API like Amplitude, and you have total flexibility in how you push to those integrations.
Why would you use postgres for an event professing system? This seems like an inefficient architecture.
Re: Rudder, an open source Segment alternative
#55We're lucky to find and deploy this project early at our startup. Being a cybersecurity company, we cannot have out customers data leave our aws account.
> we cannot have out customers data leave our aws account.
Unintentionally funny?
Re: Rudder, an open source Segment alternative
#56Is it really so difficult for engineers to create a task to process a Kafka topic? It takes one day to write a program to consume from a topic of events and push to an API like Amplitude, and you have total flexibility in how you push to those integrations. Why would you use postgres for an event professing system? This seems like an inefficient architecture.
Great question. Complication arises because of failures. One or more destinations may be down for any length of time, individual payloads may be bad etc. To handle all these you need to retry with timeouts while not blocking other events to other destinations. Also, not all events may be going to all destinations.
We build our own streaming abstraction on top of Postgres. Think layman's leveled compaction. We will write a blog on that soon. The code (jobsdb/jobsdb.go) should have some comments too in case you want to check out. Segment had a similar architecture and had a blog post on it but can't seem to find it. Also, eventually we will replace postgres with something low-level like RocksDB or even native files.
Yes, in theory you can use Kafka's streaming abstraction and create create a topic per destination. Two reason's we didn't go that route
1) We were told Kafka is not easy to support in an on-prem environment. We are not Kafka expert but paid heed to people who have designed and shipped such on prem softwares.
2) More importantly, for a given destination, we have dozens of writers all reading from same stream. Only ordering requirement is events from a given device (end consumer) are in order. So we assign same user to same writer. However, the writers themselves are independent. If a payload fails, we just block events from that user while other users continue. Blocking the whole stream for that one bad payload (retried 4-5 times) will slow things down quite a bit. If we had to achieve the same abstraction on Kafka, we would have had to create dozens of topics per destination.
Re: Rudder, an open source Segment alternative
#57Earlier quoted context omitted.
I think you’ve got a strong value proposition, especially in a GDPR/CCPA world. Have you figured out a channel and integration partner strategy yet? That’s probably the hill that you’re going to need to fight on.
No. I don't think we even understand the problem :) You mean, it would be hard to work with all the 200 or so 3rd party providers segment integrates with without having 1-1 relationships with them?
Solve it the same way Singer solved it for the opposite problem - provide a strong library/convention for how things are done, and trust the community to contribute while also providing integrations for the most used services.
When I first found Singer, it had less than a dozen taps & targets. Now has multiple dozens.
Re: Rudder, an open source Segment alternative
#58I want to see this married with the Meltano project from Gitlab - it would create an unprecedented end-to-end data environment.
Re: Rudder, an open source Segment alternative
#59Is it really so difficult for engineers to create a task to process a Kafka topic? It takes one day to write a program to consume from a topic of events and push to an API like Amplitude, and you have total flexibility in how you push to those integrations. Why would you use postgres for an event professing system? This seems like an inefficient architecture.
Great question. Complication arises because of failures. One or more destinations may be down for any length of time, individual payloads may be bad etc. To handle all these you need to retry with timeouts while not blocking other events to other destinations. Also, not all events may be going to all destinations. We build our own streaming abstraction on top of Postgres. Think layman's leveled compaction. We will wr…
Referring to this Segment blog post? Was my first read on the space, found it pretty informative.
Re: Rudder, an open source Segment alternative
#60Earlier quoted context omitted.
Great question. Complication arises because of failures. One or more destinations may be down for any length of time, individual payloads may be bad etc. To handle all these you need to retry with timeouts while not blocking other events to other destinations. Also, not all events may be going to all destinations. We build our own streaming abstraction on top of Postgres. Think layman's leveled compaction. We will wr…
Referring to this Segment blog post? Was my first read on the space, found it pretty informative. https://segment.com/blog/exactly-once-delivery/
Thanks. Will check out :)