Live data from Hacker News

Etcd, or, why modern software makes me sad

roguelazer.com

71–80 of 648 posts

Re: Etcd, or, why modern software makes me sad

#71
post #5

Earlier quoted context omitted.

What are some of the arguments against `systemd`? I've used it in several production systems and don't really have an opinion on it.

I've grown to think its a very vocal minority that objects simply ideological reasons, then finds techincal reason to back that up Even https://nosystemd.org/ mostly just lists bugs After moving all my services from varius init systems to systemd, I'd never go back

I'm mostly a happy systemd user. Unit files are much easier to work with and more consistent than init scripts, the faster bootup (at least on my arch and Ubuntu systems, maybe you could make Gentoo even more minimal with initscripts), and the container/isolation features won me over.

That said, I can understand the objection to software that would previously support BSD or macOS now only providing systemd units or depending on other parts of the systemd project that in turn depend on the init making at a pain for portability across unices. Or software like gnome taking up huge amounts of time for the BSDs to support.

Re: Etcd, or, why modern software makes me sad

#72
As far as I can tell, the only thing this article is saying is that the author doesn't like gRPC, preferring hand-rolled APIs. (The rest of the rant doesn't really talk enough about the problems to respond to. The author doesn't like Kubernetes. The author doesn't like systemd. The author doesn't like software-defined networking. No reason is given as to why, so there is really no way to have a constructive conversation about it.)

Hand rolled APIs are easier to understand, but harder to maintain. It's great if you're only ever going to have one client, but once you need more than one, it's sure tedious to write and rewrite it for every language you want to support. Using gRPC means that you can auto-generate the client, and while they might not be as wonderful as writing each one of them by hand, at least you can get a client for whatever language you're using. And, the clients all behave the same way -- trying to figure out how to add interceptors to every bespoke client you need is quite tedious. (Look at how long it took AWS to get contexts in go, or how hard it is to add OpenTelemetry to the random hand-rolled HTTP client, etc. With gRPC, you just do those things once!)

Using protos as the transport layer lets you make backwards-compatible changes smoothly; adding fields is safe, renaming fields is safe, etc. The same is not true of using JSON -- if you call something "foo", you can't just one day rename it to "bar". Clients won't know what "bar" is. So you have to update clients and servers at the same time, and you can never "make before break". You see this all the time when someone rolls out a client/server update for a browser app -- your browser cached the Javascript, and it can't talk to the server anymore until that cache expires. It's nasty. I don't understand why people do that to themselves.

gRPC also adds defined semantics for TCP connection length; with HTTP/1.1, maybe you can reuse your connection, maybe you can't, it depends. You can't have multiple requests in flight on the same connection, even if you can reuse it. HTTP/2 fixes this, but gRPC has first-class channels and behavior is well-understood for request/response, streams, etc.

It is unfamiliar and not as easy to debug on the command-line as "curl http://api.example/foo", but once you get up and running, easy things are easy and hard things are possible.

As for Kubernetes, I dunno, it hasn't been bad for me. I tend to use the managed offerings, so I don't have to spare 5 machines for an etcd cluster / masters, or maintain them. I build a container and Kubernetes ensures that it runs forever. If it dies, it's restarted. If more replicas are added, they start receiving traffic. I can manage 100% of the configuration in Git, so if my cluster or cloud provider blows up, I can re-apply somewhere else and have a 99.9% chance of it all working within 15 minutes. Before containers and k8s, production felt very much like a "yolo" thing to me. Most of the world set up some VPSs, logged in, configured them, and prayed that everything would work well. Your website went "down for maintenance" every time you did a release. You needed to distribute root credentials, hoping that you could fully trust everyone on your team to not mess anything up. It mostly worked, but through sheer brute force rather than any system working behind the scenes to make things run smoothly. With k8s, you can delegate this tedium to software. A developer can update a config file, have the PR approved, and software will ensure that the new version of the software starts, is assigned some load, and the old version is shut down. It's smooth, hard for someone to manually mess up, and quite productive.

I get that people have made their own ad-hoc orchestration and they like it. I guess that's OK. What I like about k8s is that it's a common language for this sort of thing -- I've used it for 3 projects, and they've all looked about the same. I've never had to learn anything company-specific; I can just run my software. I don't think that's a bad thing.

(The solid alternative to k8s, it seems, is just paying someone else to run everything but the one application that your company makes. Can't figure out how to run Prometheus? Just buy a Datadog subscription. Can't figure out how to run a frontend proxy? Just buy an Application Load Balancer. Can't figure out how to search and retain application logs? Just buy a Splunk subscription. Can't figure out how to install MySQL? Just buy it from your cloud provider. That is all great, but you end up spending a lot of money because you can't efficiently manage software and instead spend all your time writing rants about orchestration frameworks. Sometimes I wonder.)

Whenever I see articles like this, I have to ask what the transition from "big UNIX" to Linux would have looked like on HN. I am sure some people hated it, and I'm sure some of those people had good reasons. But things got smoothed out and it turned out that Linux and its ecosystem was pretty good. If you maintain a production application today, it probably runs on Linux, and that's good because that's one less thing you have to teach your team. I kind of see k8s in the same place. Lots of people have trouble running multiple pieces of software in production. k8s is a common language for doing those things. You can build it yourself or you can buy a managed service. You can extend it to do the crazy things you need it to do. It's not a bad thing, and I think it's pretty disingenuous to compare it to systemd.

Re: Etcd, or, why modern software makes me sad

#73
post #5

Earlier quoted context omitted.

What are some of the arguments against `systemd`? I've used it in several production systems and don't really have an opinion on it.

Almost like with etcd, it started simple, but got expanded into a huge incomprehensible monster by the original developers. Politically and business-wise, systemd is a clear win for Lennaert and his clique, as well as the whole Redhat/IBM. But I am sure that future historians will view the impact of systemd on computing as largely negative.

> But I am sure that future historians will view the impact of systemd on computing as largely negative.

I don't really think so. They probably will point out a lot problems with at some point somehow got fixed but it should be net-neutral or positive.

The thing is todays linux is running manny services and to do so nicely you want to have some form of service manager which does startup/shutdown/restart/circicute braking and helps with interconnect. Which is what systemd does and what mainly differentiates it from many previous systems which where mainly "just" start-up helpers. Through there are IMHO a large amount of problems with systemd the general approach is IMHO good, just the implementation isn't so much.

Re: Etcd, or, why modern software makes me sad

#74
post #30
post #5

Earlier quoted context omitted.

What are some of the arguments against `systemd`? I've used it in several production systems and don't really have an opinion on it.

Here's a good article that I saw on HN a few months ago: https://blog.darknedgy.net/technology/2020/05/02/0/index.htm... Also: https://suckless.org/sucks/systemd/ Personally I avoided it for a while because of the hate, but haven't really had issues with it since using it.

Ditto. my personal experience with systemd has been "wow, it's easier than ever to write an init script that runs when it's needed".

Looking at the components that make up a modern UNIX system, it's definitely time to think "how can we make a new operating system, evolved from UNIX, but with a more coherent core running the majority of run time orchestration".

Re: Etcd, or, why modern software makes me sad

#75
post #61
post #35

Author is really good at discrediting themselves straight off the bat in one sentence: > for a ~~bullshit~~ unsuccessful project called CoreOS Container Linux that was EOL'd several years ago CoreOS was actually quite successful to me as an outside observer. It had decent paid user base as well as people using it without paying. It has showed people that Chrome OS can be used to build atomically updating host OS with…

My team ran about 50% of a Fortune 500 software company on CoreOS up until a few months ago when we migrated to Flatcar. We even paid a lucrative support contract while we ran it. I can count the number of major OS issues we had with it over several years on one hand- and we've had even better success with Flatcar so far. Calling CoreOS unsuccessful is a massive misunderstanding of the market.

Your company running on it (and not flatcar, heh) is what shows a massive misunderstanding of the market.

Just because people buy snake oil, it doesn't make it a good product.

Re: Etcd, or, why modern software makes me sad

#77
post #51

Earlier quoted context omitted.

Your statements are overly hostile and hyperbolic. There's been plenty of great software out of Google and lots of good ideas taken for granted. I'll give you folks trying to build things the Google way without Google tools or scale is more often than not the wrong thing to do, but that doesn't invalidate the good.

Name a single piece of good software from Google.

Gmail, google maps, and google docs are all excellent

Re: Etcd, or, why modern software makes me sad

#78
post #5

"Kubernetes is the worst thing to happen to system administration since systemd." I'll take that quote into my fortune file.

What are some of the arguments against `systemd`? I've used it in several production systems and don't really have an opinion on it.

I have used systemd successfully in production and have a positive opinion of it. I like that it is integrated, documented, and self consistent.

None of the negative ideological complaints against it resonate with me, as none of them seem to impact me (yet, anyways). My experience with past init systems is that they are cryptic and error prone.

Re: Etcd, or, why modern software makes me sad

#79
post #46
post #39

Earlier quoted context omitted.

> This guy used to run infra at uber and was incredibly salty about every single new technology. There were a lot of bad ones, but every conversation was about as constructive, free of evidence, and bitter as this blog post. I'm going to assume you're a developer? Infra people are usually much more apprehensive to take on new technology. Crucially I would describe classically trained sysadmins as 'pessimists to the c…

saying "no" does not a devops engineer make

if your company has a position called 'devops engineer,' they've failed as an entire company to resemble anything called devops.

Re: Etcd, or, why modern software makes me sad

#80
post #75
post #61

Earlier quoted context omitted.

My team ran about 50% of a Fortune 500 software company on CoreOS up until a few months ago when we migrated to Flatcar. We even paid a lucrative support contract while we ran it. I can count the number of major OS issues we had with it over several years on one hand- and we've had even better success with Flatcar so far. Calling CoreOS unsuccessful is a massive misunderstanding of the market.

Your company running on it (and not flatcar, heh) is what shows a massive misunderstanding of the market. Just because people buy snake oil, it doesn't make it a good product.

We've been running CoreOS since before Flatcar _existed_. And we seem to be doing just fine on the OS.
Post reply on HN