21–30 of 39 posts
Obligatory: We already have a open-source tail -f; it's called `tail -f`. (Kind of /s, but much like the infamous dropbox comment[0], it sucks that there exists a problem for this to solve in the first place.) 0: https://news.ycombinator.com/item?id=9224
This is awesome, the UI looks beautiful. I've noticed you've provided Go, Python, and Node SDKs. What's the general tech stack for these? I assume your usage of Protobufs is for a consistent schemas between languages? I ask because I'm curious as to how much work it is to define new SDKs for other languages, as I'd love a Java implementation - Ideally the SDK should be a pretty thin wrapper, simply calling the gRPC s…
Tldr: go, grpc, Protobuf, wasm, deno, reactflow, ts
And yep, you’re right - we are using protobuf to have a common schema between all SDKs, the server and UI.
Re: sdk implementation - it’s basically implementing grpc methods, knowing how to exec wasm and doing a couple of extra things at instantiation. In real terms - it took us about a week to implement the python SDK - that’s with learning how to do wasm, Protobuf and grpc in python + 1 week afterwards to iron out edge cases.
Re: Java - that was going to be the next sdk we do but we have no idea if it needs to be a specific Java version? Should we target lowest possible Java version? We need to have a solid wasm runtime support - so maybe that limits us to newer versions of Java. Is that a problem?
I did Java a looong time ago - so need some outside input at this point haha
Obligatory: We already have a open-source tail -f; it's called `tail -f`. (Kind of /s, but much like the infamous dropbox comment[0], it sucks that there exists a problem for this to solve in the first place.) 0: https://news.ycombinator.com/item?id=9224
I’d urge you to check out the live demo and “tail” an app at runtime - it might be able to explain what we we are doing better than I can.
Obligatory: We already have a open-source tail -f; it's called `tail -f`. (Kind of /s, but much like the infamous dropbox comment[0], it sucks that there exists a problem for this to solve in the first place.) 0: https://news.ycombinator.com/item?id=9224
But can `tail -f` handle data ?
And if that’s already clear - sorry :)
This is awesome, the UI looks beautiful. I've noticed you've provided Go, Python, and Node SDKs. What's the general tech stack for these? I assume your usage of Protobufs is for a consistent schemas between languages? I ask because I'm curious as to how much work it is to define new SDKs for other languages, as I'd love a Java implementation - Ideally the SDK should be a pretty thin wrapper, simply calling the gRPC s…
Hey there - we have documented the tech stack here: https://docs.streamdal.com/en/resources-support/open-source/ Tldr: go, grpc, Protobuf, wasm, deno, reactflow, ts And yep, you’re right - we are using protobuf to have a common schema between all SDKs, the server and UI. Re: sdk implementation - it’s basically implementing grpc methods, knowing how to exec wasm and doing a couple of extra things at instantiation. In…
Re Java - If you're looking to maximise compatability, then yeah you should aim to target an older JDK. Virtually all Java projects use at least JDK 8 so that can be a baseline, however many enterprise projects would use closer to JDK 18 at a guess (Google's internally aiming to migrate to 21 in 2024). Generally if there are libraries or features from newer JDKs that you do want to use, I'd say just go for it, since JDK 11, the releases have been yearly (there was a three year gap between JDK 8 and 9) and more incremental.
What I would recommend is using Kotlin rather than Java, Kotlin's completely interoperable with Java, but provides a much nicer development experience. That way Kotlin clients get niceties such as named parameters [1] (which with data classes [2] can pretty well replicate StreamdalClient) [1] and Protobuf DSLs [3] and Java clients still get a first class, completely interoperable API.
No idea what WASM support is like for Java, I suspect it's lagging behind other implementations, however the most popular framework is Teavm.
1: https://kotlinlang.org/docs/functions.html#named-arguments 2: https://kotlinlang.org/docs/data-classes.html 3: https://developers.googleblog.com/2021/11/announcing-kotlin-...
Earlier quoted context omitted.
Hey there - we have documented the tech stack here: https://docs.streamdal.com/en/resources-support/open-source/ Tldr: go, grpc, Protobuf, wasm, deno, reactflow, ts And yep, you’re right - we are using protobuf to have a common schema between all SDKs, the server and UI. Re: sdk implementation - it’s basically implementing grpc methods, knowing how to exec wasm and doing a couple of extra things at instantiation. In…
Thanks for the info, sounds like you have a pretty solid tech stack :) Re Java - If you're looking to maximise compatability, then yeah you should aim to target an older JDK. Virtually all Java projects use at least JDK 8 so that can be a baseline, however many enterprise projects would use closer to JDK 18 at a guess (Google's internally aiming to migrate to 21 in 2024). Generally if there are libraries or features…
And re: kotlin - I last worked/played with it in 2016 and recall that it was MUCH nicer to work in compared to Java.
I just did a quick cursory look and it seems like Kotlin only has slightly slower builds compared to Java and rest of perf is basically the same due to generating similar bytecode. Neat!
Could you use this for batch data jobs as well? I would imagine having integration with batch job frameworks like Spark would make this more valuable from an organisation perspective.
Also, small note on the website, as an example from the "What is streamdal" page, I feel like I'm bombarded with emojis, bold and italic text, links, etc.
Thanks for sharing. Looks simple and easy to use. Could you use this for batch data jobs as well? I would imagine having integration with batch job frameworks like Spark would make this more valuable from an organisation perspective. Also, small note on the website, as an example from the "What is streamdal" page, I feel like I'm bombarded with emojis, bold and italic text, links, etc.
And re: emojis - we’ll tone it down - we were all working hard on docs late into the night and may have gotten a little wild with emojis haha :)
I have been a long time customer of the enterprise version of Streamdal, and I can confidently say Daniel and Ustin are absolutely KILLER engineers. Any time we spoke I always was impressed by their super deep experience and understanding of modern challenges!
So good to see you getting some love on HN. Excited to implement this in some personal projects as well!
Those SDKs are simply libraries though, aren't they? "SDK" often stands for more than that (e.g., development tools, bits of code not properly packaged), and may be appalling if you don't want to wrap a project around such an SDK, as opposed to merely incorporating a library.
But then I wonder why it has to be a library at all, limited to just 3 languages: why not to implement a more unixy interface, perhaps with named pipes? And/or a library with C API (so that it can be called from any common language), providing file descriptors to write into. With the former approach, basic named pipes or files and actual tail -f can be used, too.