Live data from Hacker News

Reclaiming the lost art of Linux server administration

pietrorea.com

341–350 of 485 posts

Re: Reclaiming the lost art of Linux server administration

#341
post #167

Earlier quoted context omitted.

People who have been woodworking for decades can own very expensive tools so that they can create very complicated things. People who are experts in cars can own very expensive cars and tools to tune them. People who have been working in music can have very expensive instruments and expensive headphones, microphones, sequencers, etc. We seem to be looking down on experienced "computer experts" and wanting to take the…

bruh nobody's looking down on people running they're own metal Server hardware is fun but it's not trivial to manage, buy or run. So when someone talks about how they've managed servers for 2 decades, own a house where they can install a 42 rack and how much better it is than a hosted solution. A lot of people rightly point out that this is hardly feasible for most people

No post body was provided.

Re: Reclaiming the lost art of Linux server administration

#342
post #160

Earlier quoted context omitted.

Here's what the bare metal server didn't come with: API access for managing configuration, version updates/rollbacks, and ACL. A solution for unlimited scheduled snapshots without affecting performance. Close to immediate replacement of identical setup within seconds of failure. API-managed VPC/VPN built in. No underlying OS management. (Probably forgot a few...) I get that going bare metal is a good solution for som…

> Here's what the bare metal server didn't come with: [bunch of stuff I don't need] Exactly. Imagine paying for all that when all you need is bare metal. Now imagine paying for all that just because you've read on the Internet that it's best practice and that's what the big guys do. Way back the best practice was what Microsoft, Oracle or Cisco wanted you to buy. Now it's what Amazon wants you to buy. Buy what you ne…

Sure and you could buy a digital ocean droplet or Amazon lightsail instance for 5$/month. Both include 1TB of free data transfer.

Re: Reclaiming the lost art of Linux server administration

#343

Earlier quoted context omitted.

> Blame the folks demonizing/shaming having "pet" servers and pushing immutable infrastructure. Linux server administration is quite enjoyable, and with how well apps these days can scale vertically, it really takes a special kind of workload to need (and actually saturate) fleets of servers. You don't need pet servers. Puppet or Ansible make your baremetal cattle.

I think most folks would argue "cattle" means using imaging to manage/replace your fleet. Using something like puppet or ansible against a fresh install implies a level of individualism towards each system as they "may" have minute details based on when puppet/ansible ran, even if they're part of a dynamic inventory of some sort.

I disagree. The 'cattle' vs 'pet' distinction is about replicability and replaceability. If you can spin up a new baremetal server quickly and easily in the configuration you need, then you're doing it right. The technology you choose is not relevant.

Re: Reclaiming the lost art of Linux server administration

#344

Honestly after discovering NixOS I have a new found joy of administering Linux servers. It's easy and painless, everything is declarative and versioned, and new machines can be set up for new projects or scaling in a matter of minutes. This "cattle not pets" mentality doesn't make sense for everything and is highly inefficient if the OS itself seamlessly supports immutable workloads and configuration.

"cattle not pets" doesn't mean "use cloud images for everything." It means to do essentially what you're saying you do with nix. Hosts should be easy to make and easy to drop.

Re: Reclaiming the lost art of Linux server administration

#345

Earlier quoted context omitted.

> Entirely eliminating "system administration" from my duties... ... and adding "cloud administration". What is it with people doing completely one-sided analysis even when they experiment the thing by themselves? Is cloud administration less time consuming than system administration? That's not my experience, so I'm quite interested on how it got so.

> Is cloud administration less time consuming than system administration? Infinitely, and if you look at it from a startup lens it only makes sense. One needs to point only at the recent log4j incident. This is obviously a gigantic black swan event, but even just ongoing security patching at the OS level can be a full-time gig. There is absolutely no substitution for being able to ship code to a platform that just ru…

> but even just ongoing security patching at the OS level can be a full-time gig

Oh boy, if that's what your server admin guys told you before you went full cloud then I'm sorry, I've got some bad news for you.

Re: Reclaiming the lost art of Linux server administration

#346
post #157

Earlier quoted context omitted.

Not a fair comparison and you know it. Now add to the $200 month bare metal server, the yearly salary of the 3 admins you need to manage it. One as backup, one for day time, one for the night plus a weekend rotation. Add to the admin salaries social security, insurance and a margin of safety if one is unavailable due to sickness.

> Now add to the $200 month bare metal server, the yearly salary of the 3 admins you need to manage it. That's the line the clouds, MSPs, and service contract providers sell you on, but it's never true. When you need help you're getting some minimum wage slave who hired on 6 months ago, and is struggling to juggle the dozens of other clients competing with you for his time, too. When you don't need help, they get to…

> Cut out the middle man and you're far better off.

All that knowledge is now in-house, and you have access to it at a moment’s notice.

Re: Reclaiming the lost art of Linux server administration

#347
post #137

Earlier quoted context omitted.

Who would have guessed having local, low latency, high iops drives would be better than VM using iSCSI-attached drive for storage, right? ;-)

Tbf iscsi and especially it’s future demise - nvmeof can be made quite fast. The problem that in gp2/3, gce-pd cases it’s probably not some native storage but another layer of software-based distributed store which makes stuff slow/expensive

But also less prone to single drive failures. And easily auto expanding storage sizes. Everything is a trade off.

Re: Reclaiming the lost art of Linux server administration

#348
post #86
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

Yes but you have talent and a lifetime of experience, plus space for a noisy 42u rack full of servers, but not everybody does...

I can do the same thing with the 15x8x20cm Synology sitting above my bathroom (don’t ask, that’s where the internet enters te house :P).

Your main point still stands though, not everyone can or wants to do that.

Re: Reclaiming the lost art of Linux server administration

#349

Earlier quoted context omitted.

I love Apache, do not love Nginx and won't be looking at Caddy (it's written in Golang). Apache (even Nginx) are easy to set up reverse proxy and can simultaneously serve static content and serve as the https certificate main, as well as a few dozen other things like load balancing and rate limiting, etc.

>> (it's written in Golang) That means no memory related security bugs, which is a huge plus.

Not really. JVM is similar in this regard and one of the thing we wrote in our basic Java course was the code that produces memory leak. And when you have memory leak you also may end up with other fun things like applications behaving unexpectedly in low memory conditions.

Re: Reclaiming the lost art of Linux server administration

#350
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

I assume/hope you have good, tested, off-site backups if the important data & config… I run my stuff from home too, though it is smaller scale than yours currently. Off-site & soft-offline backups are on encrypted volumes on servers & VMs elsewhere.

The backup server in my home is the offsite backup. The primary data lives in the datacenter.
Post reply on HN