Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go is portable, until it isn't

simpleobservability.com

Post reply on HN
Print view View original

111–120 of 138 posts

Re: Go is portable, until it isn't

#111

Post by bb88 »

davvid wrote: post #73

> We did not want to spend time maintaining a backward compatible parser or doing code archaeology. So this option was discarded. Considering all of the effort and hoop-jumping involved in the route that was chosen, perhaps this decision might be worth revisiting. In hindsight, maintaining a parser might be easier and more maintainable when compared to the current problems that were overcome and the future problems t…

That's what I was thinking too. A go native library is 10 times better in the go ecosystem than a c library linked to a go executable.

Also in the age of AI it seems possible to have it do the rewrite for you, for which you can iterate on further.

Re: Go is portable, until it isn't

#112

Post by regularfry »

All of this, every last bit of complexity and breakage and sweat, is downstream of this:

> Journal logs are not stored in plain text. They use a binary format

And it was entirely predictable and predicted that this sort of problem would be the result when that choice was made.

Re: Go is portable, until it isn't

#113

Post by zbentley »

IshKebab wrote: post #94
combiBean wrote: post #84

Was there not a third option: Calling the journalctl CLI as a child process and consume the parsed logs from the standard output? This might have avoided both the requirement to use CGO and also to write a custom parser. But I guess I am missing something.

It's generally less robust to run CLI tools and scrape the output. Usually it isn't intended to be machine readable, and you have to handle extra failure modes, like incompatible tool versions, missing tools, incorrect parsers, etc. It's the lazy-but-bad solution.

I think a lot is riding on that “generally”. You’re right that the default approach/majority of cases should avoid shelling out wherever possible, but there are a large minority of situations where doing that does make sense, including:

Calling a CLI tool which will be present everywhere your program might reasonably be installed (e.g. if your program is a MySQL extension, it can probably safely assume the existence of mysqld).

The CLI tool you want to call is vendored into or downloaded by your wrapper program, reducing installation requirements overhead (this is not always a good idea for other reasons, but it does address a frequently cited reason not to shell out).

The CLI tool’s functionality is both disjoint with the rest of your program and something that you have a frequent need to hard-kill. (Forking is much more error prone than running a discrete subprocess; you can run your own program as a subprocess too, but in that case the functionality is probably not disjointed).

Talking to POSIX CLI tools in a POSIX compatible way (granted most things those tools do are easier/faster in a language’s stdlib).

Re: Go is portable, until it isn't

#114

Post by IshKebab »

pjmlp wrote: post #106
IshKebab wrote: post #95

Earlier quoted context omitted.

> Cross compilation and cross platform are synonymous in compiled languages Err, no. Cross-platform means the code can be compiled natively on each platform. Cross-compilation is when you compile the binaries on one platform for a different platform.

Not at all, cross platform means executing the same application in many platforms, regardless of the hardware and OS specific features of each platform. Cross-compilation is useless if you don't actually get to executed the created binaries in the target platform. Now, how do you intend to compile from GNU/Linux into z/OS, so that we can execute the generated binary out from the C compiler ingesting the code written…

> cross platform means executing the same application in many platforms, regardless of the hardware and OS specific features of each platform.

That is a better definition yes. But it's still not synonymous with cross-compilation, obviously. Most cross-platform apps are not cross-compiled because it's usually such a pain.

Re: Go is portable, until it isn't

#116

Post by valbaca »

regularfry wrote: post #112

All of this, every last bit of complexity and breakage and sweat, is downstream of this: > Journal logs are not stored in plain text. They use a binary format And it was entirely predictable and predicted that this sort of problem would be the result when that choice was made.

Unix philosophy strikes again

Re: Go is portable, until it isn't

#118

Post by khazit »

davvid wrote: post #73

> We did not want to spend time maintaining a backward compatible parser or doing code archaeology. So this option was discarded. Considering all of the effort and hoop-jumping involved in the route that was chosen, perhaps this decision might be worth revisiting. In hindsight, maintaining a parser might be easier and more maintainable when compared to the current problems that were overcome and the future problems t…

There is an existing pure Go library [1] written by someone else. The issue is that we weren’t confident we could ship a reliable parser. We even included an excerpt from the systemd documentation, which didn’t exactly reassure us:

> Note that the actual implementation in the systemd codebase is the only ultimately authoritative description of the format, so if this document and the code disagree, the code is right

This required a lot of extra effort and hoop-jumping, but at least it’s on our side rather than something users have to deal with at deploy time.

[1]: https://github.com/Velocidex/go-journalctl

Re: Go is portable, until it isn't

#119

Post by khazit »

jeremyjh wrote: post #109
IshKebab wrote: post #94

Earlier quoted context omitted.

It's generally less robust to run CLI tools and scrape the output. Usually it isn't intended to be machine readable, and you have to handle extra failure modes, like incompatible tool versions, missing tools, incorrect parsers, etc. It's the lazy-but-bad solution.

journalctl is designed for these use cases and has options to solve those issues. The lazy part here is you not doing any research about this tool before dismissing it as "not best practice", which is exactly what the fuckups who wrote this article did.

We dismissed using journalctl at the very start. We’ve had similar experiences with other CLI tools: the moment you start embedding them inside a program, you introduce a whole new class of problems. What if journalctl exits? What if it outputs an error? What if it hangs? On top of that, you have to manage the subprocess lifecycle yourself. It’s not as easy as it may seem.

You can also argue that sd_journal (the C API) exists for this exact reason, rather than shelling out to journalctl. These are technical trade-offs, doesn't mean we're fuckups

Re: Go is portable, until it isn't

#120

Post by mbrumlow »

I really hate this type of blog. It pollutes the world with this attitude of “I messed up, how I have to frame the problem in a way, and write a blog that lets my ego stay intact”, which results in blogs like this showing in decision making process as “why you should not use go”. And mostly people never look past the title.

The fact is go is portable, it provides the ability to cross compile out of the box and reasonably executed on other platforms it supports. But in this case, a decision that had little to do with go, the desire to use c code, a non go project, with their go project made things harder.

These are not “just a set of constraints you only notice once you trip over them”, this is trivializing the mistake.

Entire blog can be simplified to the following.

We were ignorant, and then had to do a bunch of work because we were ignorant. It’s a common story in software. I don’t expect everybody to get it right the first time. But what we don’t need is sensational titled blogs full of fluff to try to reason readers out of concluding the obvious. Somebody in charge made decisions uninformed and as a result the project became more complicated and probably took longer.

Post reply on HN