Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative • Hacker News Board

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#81

Post by tomnipotent » Thu, Nov 04, 2021, 6:28 PM UTC

vklmn wrote: post #70

tailspin2019 wrote: post #67

This looks really cool. I'm keen to try it. It looks like it might play well with my current logging system of choice, Seq [0]. Do you support inbound webhooks? I can see webhooks as a destination but not as a source? [0] https://datalust.co/seq

You can hack almost anything using inbound Event API ( https://jitsu.com/docs/sending-data/api ) and JavaScript transformations ( https://jitsu.com/blog/javascript-transform )

A standard webhook source abstraction would be very useful, that captures the URI, POST payload and HTTP headers.

This way I can setup my source in Jitsu, get a unique URL, and then paste that URL into the tool generating webhook events (e.g. Shopify). A normalized schema based on the JSON payload doesn't need to be created for this to be useful.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#82

Post by MaxiaNN » Thu, Nov 04, 2021, 6:30 PM UTC

MaxiaNN wrote: post #78

Great story. How do you feel Jitsu compares to Rudderstack?

Read the explanations below in the comment thread and the contribution from RudderStack itself.

Conclusion, Jitsu is wide event based tracking whereas RudderStack is focused on ID specific customer events.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#83

Post by vklmn » Thu, Nov 04, 2021, 6:31 PM UTC

ajbosco wrote: post #77

Why would I use this instead of Airbyte?

Two reasons a) you can push events from your apps b) if you want to have more connectors available (Singer, and few native connectors)

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#84

Post by ThePhysicist » Thu, Nov 04, 2021, 6:33 PM UTC

soumyadeb wrote: post #14

vklmn wrote: post #10

Earlier quoted context omitted.

I don't believe that YC funded RudderStack, but they funded Segment indeed! It's not uncommon. If I have to guess, every batch or so have at least 2 companies directly competing with each over

Yes, RudderStack is not YC funded. Though I have learnt a lot from YC (and PG) through the years and attribute our entrepreneurship to them. Founder of RudderStack here.

Ah sorry, I mixed you up with Freshpaint, which is actually a YC startup and Segment competitor that pivoted to their current business model.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#85

Post by kposehn » Thu, Nov 04, 2021, 6:37 PM UTC

vklmn wrote: post #68

kposehn wrote: post #61

Great product. I'm a frequent user of Segment from the early days and have been curious to see when an open-source competitor comes around that will match feature-for-feature. Thoughts: 1. You've got most major ads sources that I care about, but it seems that there is a higher bar to implementation. Segment lets me just plug in Google & FB ads and dump the entire shebang right into my data warehouse. A lot of marketi…

Thank you! 1. Thats exactly the reason we have native connectors for Facebook and Google Ads (we didn't use ones from Airbyte and Singer). Jitsu can pull any combination from FB/GAds — it's almost like SQL! Airbyte/Singer just can't do that. Later we're going to vet other connectors too and decide if we need to re-implement them 2. We have functions too! https://jitsu.com/blog/javascript-transform

Ah, ok - I didn't see the transform compared to Functions. Very cool and I like the multiplexing.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#86

Post by xtreding » Thu, Nov 04, 2021, 6:38 PM UTC

tomnipotent wrote: post #62

vklmn wrote: post #58

Earlier quoted context omitted.

The same way as Kafka. Jitsu nodes (=collection agents) writes to write ahead log, and then either sends data to destination right away, or sends data in batches.

Thanks! I take it this file is where I can get started to learn more: https://github.com/jitsucom/jitsu/blob/0aaa74b59eb9d8c885c80... I see that it instantiates an "AsyncLogger" - does the service wait until data is written to the log prior to returning success to the client? Is the WAL the same source used to feed both database storage destinations and other SaaS destinations?

Hi! My name is Sergey, I’m a Jitsu product engineer. I’ll gladly answer your question! AsyncLogger works asynchronously by design. There is a go channel which writes JSON’s to the log file. Answering your question: the service doesn’t wait until data is written to the log prior to returning success to the client. WAL log is designed for keeping events JSON’s between Jitsu instance restarts to prevent data loss. When you deploy your Jitsu application, it will handle service restart signals (e.g. sigterm) and closes database connections as well as other resources. All incoming events are stored in WAL log in this time. So, after the Jitsu starts, all events from WAL log will be passed to the main events JSON pipeline and stored to the destinations.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#87

Post by bmm6o » Thu, Nov 04, 2021, 6:44 PM UTC

einpoklum wrote: post #18

> We help companies collect events from their apps, websites, and APIs and send them to databases. For those who don't know what "Segment" is (like me) - this Jitsu thing seems to only be relevant to web-based/web-oriented apps.

There's an http endpoint so it's easy to use from a browser, but of course it's usable from any process that can post to the endpoint.

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#88

Post by zimmerx » Thu, Nov 04, 2021, 6:51 PM UTC

vklmn wrote: post #11

tarun_anand wrote: post #7

Adding on to the previous comment, how does this compare to rudderstack?

We're very similar indeed. But we attack the same problem from different angles: - We truly believe that our product should be accessible for small teams too. That's why se made Jitsu very easy to deploy. I'm not sure you can deploy Rudder on Heroku, or on any service with a single Docker file. - Our ETL component is open-source (and based on other great OSS projects - Airbyte & Singer). RudderStack haven't published…

Rudderstack user here (and ex Segment). Thanks for sharing this stuff, very cool project.

Just to answer some of this:

- rudderstack has deploy ready helm charts, which I'd argue are significantly better than docker compose or docker setups because they set up all the other niggly parts. Would be cool to see that here :)

- rudderstack has gone quite far away from replacing segment. It's true that their core API is compatible and I think your transformation layer is really cool. However it can be used for those use cases because rudderstack doesn't really care about users or user IDs and can be used for any sort of data generally.

There's a piece in the docs talking about the fact that you don't get caught by adblock - whilst this may be true when someone launches it, that's not true of your platform. That's just the fact that a lot of smaller businesses will not get their URLs added to the ad block lists. I think it's a bit misleading to mention that in such a way because technically we're all tracking users and ad block is a way for users to choose not to be tracked, not be tricked into being tracked because someone has masked the tracking script ;) if a huge client (a la Adidas or something) decided to use your scripts I'm sure someone would eventually add it to the ad block lists that get propagated.

One of the things that would be cool would be some sort of opt in configuration. Segment has some awful consent SDK that is really bad, would be cool to see what you do there. GDPR is a big deal and browser fingerprinting is data processing. It's worth looking at your comments on being GDPR compliant btw https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-finge...

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#89

Post by polskibus » Thu, Nov 04, 2021, 7:12 PM UTC

vklmn wrote: post #6

santiagobasulto wrote: post #3

I really like this, congratulations on the launch. And this is such a huge space that there's definitively room for other options (aside from Segment). I'm a little bit out of the loop in this event processing space. Do you think Jitsu could replace lower-level event processing implementations as Kafka/Kinesis? Or this is thought for more "high level" marketing stuff.

That's a good question. We're aiming to replace Kafka in some cases. There're many ways how people use Kafka. But it could be roughly divided into two buckets - Kafka as a company wide message bus: dozen's of (micro)services sending data there, and consumers listens to data. Each service doesn't know which other service will consume the data. For that case, we're not looking to replace Kafka — we're going to work alo…

In the second point, presumably you mean only one-way communication, right?

Re: Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

#90

Post by tomnipotent » Thu, Nov 04, 2021, 7:24 PM UTC

xtreding wrote: post #86

tomnipotent wrote: post #62

Earlier quoted context omitted.

Thanks! I take it this file is where I can get started to learn more: https://github.com/jitsucom/jitsu/blob/0aaa74b59eb9d8c885c80... I see that it instantiates an "AsyncLogger" - does the service wait until data is written to the log prior to returning success to the client? Is the WAL the same source used to feed both database storage destinations and other SaaS destinations?

Hi! My name is Sergey, I’m a Jitsu product engineer. I’ll gladly answer your question! AsyncLogger works asynchronously by design. There is a go channel which writes JSON’s to the log file. Answering your question: the service doesn’t wait until data is written to the log prior to returning success to the client. WAL log is designed for keeping events JSON’s between Jitsu instance restarts to prevent data loss. When…

Is the WAL only used during restart, or also during normal operations? Trying to create a mental model of how data flows through the system and into destinations.