Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Alien – Self-hosting with remote management (written in Rust)

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#31

Post by jcgrillo »

quesera wrote: post #26
jcgrillo wrote: post #12

Earlier quoted context omitted.

> This can be a ton of data though, so we're trying to figure out what to compress and how. We also have the challenge of figuring out how to scrub logs of any potentially sensitive information. This is fundamentally a data modeling problem. Currently computer telemetry data are just little bags of utf-8 bytes, or at best something like list >. IMO this needs to change from the ground up. Logging libraries should emi…

Grand solutions require broad coordination, and they often devolve back into a modified-but-equivalent version of the previous problem. :( Stream-of-bytes is classically difficult model to escape. Many have tried.

Yeah. There are good reasons things are bad. But there's also a foolish consistency. Like, you can just do things! If you decide monitoring is important you can decide not to outsource it. Most everyone doesn't, though. Probably because they don't think it's very important, and the existing tools get it done well enough, and it's the muscle memory of the subjectively familiar (if objectively fantastically overpriced).

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#32

Post by stronglikedan »

alongub wrote: post #17
nodesocket wrote: post #13

Earlier quoted context omitted.

I also used to work with on-premise installs of Kubernetes and their “security” postures prevented any in-bound access. It was a painful process of requesting access, getting on a zoom call and then controlling their screen via a Windows client and putty. It’s was beyond painful and frustrating. I tried to pitch using a tool like Twingate which doesn’t open any inbound ports, can be locked down very tight using SSO,…

At least they didn't ask you to TeamViewer into a Windows Server 2012 box and open Event Viewer..

That would be my preference compared to the situation you're replying to. Event Viewer is powerful if one takes some time to learn it.

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#33

Post by alongub »

stronglikedan wrote: post #32
alongub wrote: post #17

Earlier quoted context omitted.

At least they didn't ask you to TeamViewer into a Windows Server 2012 box and open Event Viewer..

That would be my preference compared to the situation you're replying to. Event Viewer is powerful if one takes some time to learn it.

Fair point

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#34

Post by quesera »

jcgrillo wrote: post #31
quesera wrote: post #26

Earlier quoted context omitted.

Grand solutions require broad coordination, and they often devolve back into a modified-but-equivalent version of the previous problem. :( Stream-of-bytes is classically difficult model to escape. Many have tried.

Yeah. There are good reasons things are bad. But there's also a foolish consistency. Like, you can just do things! If you decide monitoring is important you can decide not to outsource it. Most everyone doesn't, though. Probably because they don't think it's very important, and the existing tools get it done well enough, and it's the muscle memory of the subjectively familiar (if objectively fantastically overpriced)…

Well, in the early days of infrastructure growth, when designing bespoke monitoring systems and protocols would be relatively low-cost, it's still nowhere near the highest-ROI way to spend your tech team's time and energy.

And to do it right (i.e. low-risk of of having it blow up with negative effects on the larger business goals), you need someone fairly experienced or maybe even specialized in that area. If you have that person, they are on the team because of their other skills, which you need more urgently.

SaaS, COTS, and open source monitoring tools have to cater to the existing customers. The sales pitch is "easy to integrate". So even they are not incentivized to build something new.

It boils down to the fact that stream-of-bytes is extremely well-understood, and almost always good enough. Infinitely flexible, low-ceremony, no patents, and comes preinstalled on everything (emitters and consumers). It's like HTTP in that way.

And the evolution is similar too. It'll always be stream-of-bytes, but you can emit in JSON or protobuf etc, if it's worth the cognitive overhead to do so. All the hyperscalers do this, even when the original emitter (web servers, etc) is just blindly spewing atrocious CLF/quirky-SSV text.

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#35

Post by jcgrillo »

quesera wrote: post #34
jcgrillo wrote: post #31

Earlier quoted context omitted.

Yeah. There are good reasons things are bad. But there's also a foolish consistency. Like, you can just do things! If you decide monitoring is important you can decide not to outsource it. Most everyone doesn't, though. Probably because they don't think it's very important, and the existing tools get it done well enough, and it's the muscle memory of the subjectively familiar (if objectively fantastically overpriced)…

Well, in the early days of infrastructure growth, when designing bespoke monitoring systems and protocols would be relatively low-cost, it's still nowhere near the highest-ROI way to spend your tech team's time and energy. And to do it right (i.e. low-risk of of having it blow up with negative effects on the larger business goals), you need someone fairly experienced or maybe even specialized in that area. If you hav…

> It'll always be stream-of-bytes, but you can emit in JSON or protobuf etc, if it's worth the cognitive overhead to do so.

This is the crux of it. That's great until you encounter a need for a schema, and then it's "schema-on-read" or some similar abomination. And the need might not manifest until you're pushing like 1TB/day or more of telemetry data with hundreds or thousands of engineers working on some >1MLoC monstrosity. Hard to dig out of that hole.

The situation is tragically optimal--we've achieved some kind of multiobjective local maximum on a rock in the sewer at the bottom of a picturesque alpine valley and declared victory. We should do better.

Or maybe I'm overly optimistic.

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#37

Post by pruthviraja »

alongub wrote: post #28
pruthviraja wrote: post #27

Interesting approach. The managed self-hosting gap is real..we have run into this exact pain point with kubernetes based deployments where customers modify their cluster configs and things break silently. If I may ask how does Alien handle rollback if an update fails in a customer environment?is there any plan for on-prem/bare metal support beyond the big three clouds?

Alien is basically a huge state machine where every API call that mutates the environment is a discrete step, and the full state is durably persisted after each one. If something fails mid-update, it resumes from exactly where it stopped. You can also point a deployment to a previous release and it walks back. This catches and recovers from issues that something like Terraform would just leave in a broken state. For…

i think the durable state machine approach is smart...that resume from where it stopped behavior is a big deal during incident response when you really dont want to rerun an entire deployment just because one step failed. K8s as a deployment target would be huge especially for the on-prem enterprise crowd. Will definitely keep an eye on that

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#39

Post by quesera »

jcgrillo wrote: post #35
quesera wrote: post #34

Earlier quoted context omitted.

Well, in the early days of infrastructure growth, when designing bespoke monitoring systems and protocols would be relatively low-cost, it's still nowhere near the highest-ROI way to spend your tech team's time and energy. And to do it right (i.e. low-risk of of having it blow up with negative effects on the larger business goals), you need someone fairly experienced or maybe even specialized in that area. If you hav…

> It'll always be stream-of-bytes, but you can emit in JSON or protobuf etc, if it's worth the cognitive overhead to do so. This is the crux of it. That's great until you encounter a need for a schema, and then it's "schema-on-read" or some similar abomination. And the need might not manifest until you're pushing like 1TB/day or more of telemetry data with hundreds or thousands of engineers working on some >1MLoC mon…

> The situation is tragically optimal--we've achieved some kind of multiobjective local maximum on a rock in the sewer at the bottom of a picturesque alpine valley and declared victory. We should do better.

But it's a very comfortable rock. pointy in all the right places.

Re: Show HN: Alien – Self-hosting with remote management (written in Rust)

#40

Post by jcgrillo »

quesera wrote: post #39
jcgrillo wrote: post #35

Earlier quoted context omitted.

> It'll always be stream-of-bytes, but you can emit in JSON or protobuf etc, if it's worth the cognitive overhead to do so. This is the crux of it. That's great until you encounter a need for a schema, and then it's "schema-on-read" or some similar abomination. And the need might not manifest until you're pushing like 1TB/day or more of telemetry data with hundreds or thousands of engineers working on some >1MLoC mon…

> The situation is tragically optimal--we've achieved some kind of multiobjective local maximum on a rock in the sewer at the bottom of a picturesque alpine valley and declared victory. We should do better. But it's a very comfortable rock. pointy in all the right places.

til it ain't
Post reply on HN