Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reclaiming the lost art of Linux server administration

pietrorea.com

Post reply on HN
Print view View original

321–330 of 485 posts

Re: Reclaiming the lost art of Linux server administration

#321

Post by willjp »

LoveGracePeace wrote: post #153
TacticalCoder wrote: post #74

A very weird thread that degenerated into: "PaaS vs self-hosted/self-owned hardware" . I'm pretty sure most people sysadmin'ing their Linux servers are actually doing it with rented dedicated servers. TFA btw specifically mentions: "don't manage physical hardware" . Big companies like Hetzner and OVH have hundreds of thousands of servers and they're not the only players in that space. They don't take care of "everyth…

I generally agree, I have a cheap AWS Lightsail VPS (mainly for email hosting since my ISP blocks port 25 because I'm a "consumer" and they want to protect the world for consumer spammers) but also for flexibility. I like that the Internet is not at my doorstep (no open ports at home). So, cheap VPS, Wireguard and my home machines to serve whatever I want. I don't pay extra if I use a ton of CPU or disk IO, for examp…

This is exactly the solution I landed on as well. It has served me well for a few years now :).

Re: Reclaiming the lost art of Linux server administration

#322

Post by willjp »

solatic wrote: post #102

Why can't people just pick the right tool for the job? The truth behind these managed services is that, for the correct usecases, they are VERY cheap. And for the wrong usecases, they are RIDICULOUSLY expensive. Most businesses have nightly cronjobs generating some kind of report that is then emailed to stakeholders. Why on Earth would you run a dedicated Linux box for that anymore? Glue a nightly trigger to AWS Lamb…

agreed.

Re: Reclaiming the lost art of Linux server administration

#323

Post by cosmin800 »

Tragic and comic in the same time. In half generation we will be losing all technological skills. There is a shortage ... "The world is facing a shortage of nuclear specialists because of a lack of training programmes and students to replace those about to retire, according to Nobel Peace Prize holder and former director-general of the International Atomic Energy Agency, Dr Mohamed ElBaradei." apparently the same thing is happening in aviation.

Re: Reclaiming the lost art of Linux server administration

#324

Post by ayushnix »

madjam002 wrote: post #95

Honestly after discovering NixOS I have a new found joy of administering Linux servers. It's easy and painless, everything is declarative and versioned, and new machines can be set up for new projects or scaling in a matter of minutes. This "cattle not pets" mentality doesn't make sense for everything and is highly inefficient if the OS itself seamlessly supports immutable workloads and configuration.

> It's easy and painless

It was probably the most difficult thing I tried to, unsuccessfully, use on my desktop. I imagine learning how to use Vim/Emacs as a complete beginner would probably be several magnitudes of order easier than learning the Nix DSL and the Nix way of doing things. And from what I've read about the experience of other people who do use NixOS and talk about both the good AND the bad, using it seems like an unhealthy relationship.

Not to mention that the Nix package manager feels slow as hell and reminds me of my unpleasant hours spent using rpm and dnf.

Re: Reclaiming the lost art of Linux server administration

#325

Post by NikolaeVarius »

MonaroVXR wrote: post #311
NikolaeVarius wrote: post #185

Earlier quoted context omitted.

I find people dont know how amazing a "immutable" server fleet is until you've experienced it. It was so trivial to terminate and restart dozens of servers at any given time since unless there was a mistake in the cloud-init, we could bootstrap our entire infrastructure from scratch within an hour. It was amazing, never had to deal with something missing on a server or a config being wrong in a special case. Dozens o…

"It was so trivial to terminate and restart dozens of servers at any given time since unless there was a mistake in the cloud-init, we could bootstrap our entire infrastructure from scratch within an hour." Got any tips in this regard? Ansible, Puppet, Python, Terraform, Openstack Pulumi springs to my mind. Still need to cover a PXE server. (Maybe with Ansible)

We use Salt Masterless with Packer to build AMIs.Terraform defines our infra. No matter what happens we ensure any service can boot by itself with 0 intervention

Re: Reclaiming the lost art of Linux server administration

#326

Post by Nextgrid »

dragonsky wrote: post #291
senko wrote: post #160

Earlier quoted context omitted.

> Here's what the bare metal server didn't come with: [bunch of stuff I don't need] Exactly. Imagine paying for all that when all you need is bare metal. Now imagine paying for all that just because you've read on the Internet that it's best practice and that's what the big guys do. Way back the best practice was what Microsoft, Oracle or Cisco wanted you to buy. Now it's what Amazon wants you to buy. Buy what you ne…

Your point is well made... The other thing you need to think about is that all those extra services can come with a reliability cost if your provider is not 100%. Most outages we've encountered have been because the HA infrastructure has decided to have problems rather than the underlying hardware or actual service we need to run. Having all that "best practice" service is great if it works well, but when it becomes…

It’s great if you need that reliability in practice - and not just that you think you need it.

For every company or startup that thinks they need 100% uptime the reality is that they can not only get away with much less but in practice will end up with much less anyway because the effort and moving parts (load balancing, distributed database, etc) will typically result in something failing anyway even if the underlying hardware is indeed 100% up, and somewhat surprisingly to quite a bit of people here, manage to survive and thrive despite that (the recent AWS outages took out a lot of services and products and they still seem to be around somehow).

Re: Reclaiming the lost art of Linux server administration

#327

Post by lamontcg »

chasil wrote: post #96
lamontcg wrote: post #83

Earlier quoted context omitted.

If you're not publishing your scripts and you're running your own infrastructure, you probably don't care about portability at all.

None of the BSDs use bash in their base. Apple recently switched from bash to zsh. OpenBSD uses a descendent of pdksh. Another major user of a pdksh descendent is Android (mksh), with a truly massive install base. Some of the bash problem, besides portability, is GPLv3. That was a major factor for Apple. I don't want my script portability linked to corporate patent issues. For this and other reasons, I don't use bash…

Yeah I've written and maintained scripts for the past 10 years that have to run on stock Solaris, AIX and *BSD. When they started out they had to work on the Solaris 9 /bin/sh, which is particularly elderly.

You should feel free to continue to write portable scripts.

That is horrible advice for anyone who is just starting out though and they should stick with #!/bin/bash and not worry about it until they actually want to consider platforms other than Linux. Portability is the wrong thing for people who are learning to focus on.

It is also bad advice for most system administrators at most businesses since they won't have those alternative *nixes either, and its a best practice to minimize how many different operating systems you have to care about.

Re: Reclaiming the lost art of Linux server administration

#328

Post by nextos »

MonaroVXR wrote: post #286
0xCMP wrote: post #94

Earlier quoted context omitted.

In my experience Pet servers are a good starting point (you really should _graduate_ from Pet servers into all the various immutable/cattle stuff), but it can quickly require discipline from the Admins. They can't be doing one-off undocumented config, package, and network/firewall changes which make it impossible to setup another server reliably. At $company I moved us to Terraform+Packer (to get them used to immutab…

Do you have any tips in auto installing an OS on a server/desktop? I'm completely missing. I have searched arpund and I have some solutions, but back in my head, some people have something else. The part before Ansible or puppet to kick in.

NixOS.

Re: Reclaiming the lost art of Linux server administration

#329

Post by hackerfromthefu »

cerved wrote: post #167
sseagull wrote: post #93

Earlier quoted context omitted.

People who have been woodworking for decades can own very expensive tools so that they can create very complicated things. People who are experts in cars can own very expensive cars and tools to tune them. People who have been working in music can have very expensive instruments and expensive headphones, microphones, sequencers, etc. We seem to be looking down on experienced "computer experts" and wanting to take the…

bruh nobody's looking down on people running they're own metal Server hardware is fun but it's not trivial to manage, buy or run. So when someone talks about how they've managed servers for 2 decades, own a house where they can install a 42 rack and how much better it is than a hosted solution. A lot of people rightly point out that this is hardly feasible for most people

Got an old laptop laying around?

Re: Reclaiming the lost art of Linux server administration

#330

Post by chrismarlow9 »

StillBored wrote: post #85
deepspace wrote: post #56

Earlier quoted context omitted.

> I can invest in a $3000 PowerEdge server with much better hardware And when some component of the server fails, your app is unavailable until you can repair it. So you need another server for redundancy. And a load balancer. And a UPS. And a second internet connection. If your app is at all critical, you need to replicate all of this at a disaster recovery site. And buy/run/administer DR software. And hardware has…

So, just buy another and leave it as a hot (or cold) standby in a different data-center. Or use AWS as the DR site an spin it up only if the local HW fails. This sounds expensive if your talking one server and vs a year of AWS charges, but is a tiny bump if it turns out you need to buy a dozen servers to replace a large AWS bill. Plus, I think most people underestimate how reliable server grade hardware is. Most of i…

The original intended use for AWS was to be additional capacity for existing non cloud setups during spikes. Things like investment firms and algorithmic trading setups that cost money during downtime.

They found a larger market in being the full infrastructure, so they started down the road to cloud appliances making it easy for any average Joe to spin up infra.

This was in ec2 classic era before VPC was a thing. So your active/passive setup description is actually using AWS appropriately based on it's original ideas.

Post reply on HN