Live data from Hacker News

Reclaiming the lost art of Linux server administration

pietrorea.com

141–150 of 485 posts

Re: Reclaiming the lost art of Linux server administration

#141

Time is money, and the more time I spend on infrastructure, the less time I spend on product. And thus is born the incredible demand of infrastructure as a service. Thankfully one person's cloud is another person's on prem infrastructure so sysadmin skills will always be in demand. From my perspective in enterprise computing, I now see people taking 2 paths. One where they become super deep sysadmins and work on infr…

That's when it clicked for me.. comparing my hourly salary rate vs. the cost of running these services "in the cloud." Entirely eliminating "system administration" from my duties was absolutely a net win for me and our team.

[deleted]

Re: Reclaiming the lost art of Linux server administration

#142
post #44

Earlier quoted context omitted.

Do you have a static IP? I have a homelab too but getting “enterprise grade” service from comcast seems to be my biggest barrier to scaling without leaning on aws.

Rent a $5 VPS, run a VPN tunnel from your lab to that box, and run a reverse proxy on it. You'll get some additional latency, but that's about it.

Caution: you may end up with your packets blackholing on the way for unknown reasons after temporary loss of connectivity.

I think it might have something to do with the NAT forgetting about my UDP "connection", but haven't found the culprit yet.

Re: Reclaiming the lost art of Linux server administration

#143
post #77

Full disclaimer, I'm very much not a sysadmin or devops guy. However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all. I couldn't figure out AWS. But I could figure out how to host sites on a linux VPS. So what's the story here - is serverless something that only makes sense at a certai…

Caddyserver is Apache/Nginx killer and we will talk about Caddy in a couple of years as if it's been always the default ("why did we kept fighting with apache/nginx all those years, silly us"). Seriously. It's just a completely different way to think about web servers and automation. I'm just amazed it took all these years to emerge.

I love Apache, do not love Nginx and won't be looking at Caddy (it's written in Golang). Apache (even Nginx) are easy to set up reverse proxy and can simultaneously serve static content and serve as the https certificate main, as well as a few dozen other things like load balancing and rate limiting, etc.

Re: Reclaiming the lost art of Linux server administration

#144
post #87
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

I remember how surprised people were when I demoed a $200/month bare metal server outperforming by a huge margin RDS MySQL instance that they were paying something upwards of 16k/month. IIRC we ended up using it as a disposable replica for some non-real time but heavy operations.

Just out of curiousity but what did you get out of it? I mean if they get to pick it, get the responsibility of AWS on you, but do you also get to get paid those money after saving them money? I mean the way I see it, is that 16k/month don't only pay for the hardware, but also to keep headaches away and to have someone to blame

Re: Reclaiming the lost art of Linux server administration

#145

Since the post pretty much says “go out and do it!” Does anyone have a good source of learning that is comprehensive and practical? I’m talking about a good guided book/tutorial on how to administer a server properly and what things one should know how to fix, not just how to set up Wordpress.

I usually provide this as an intro:

http://www.linuxcommand.org/tlcl.php/

From there picking up configuration management should be pretty straightforward.

Re: Reclaiming the lost art of Linux server administration

#146
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

What services would you be paying “several hundred a month” for - in what currency? In US dollars my personal cloud services don’t run more than $20 a month.

Re: Reclaiming the lost art of Linux server administration

#147
post #87

Earlier quoted context omitted.

I remember how surprised people were when I demoed a $200/month bare metal server outperforming by a huge margin RDS MySQL instance that they were paying something upwards of 16k/month. IIRC we ended up using it as a disposable replica for some non-real time but heavy operations.

Here's what the bare metal server didn't come with: API access for managing configuration, version updates/rollbacks, and ACL. A solution for unlimited scheduled snapshots without affecting performance. Close to immediate replacement of identical setup within seconds of failure. API-managed VPC/VPN built in. No underlying OS management. (Probably forgot a few...) I get that going bare metal is a good solution for som…

Of course there are a lot of benefits of using hosted databases. I like hosted databases and use them for both work and personal projects.

What I have a problem with is:

- the premium over bare metal is just silly

- maximum vertical scaling being a rather small fraction of what you could get with bare metal

- when you pay for a hot standby you can't use it as a read only replica (true for AWS and GCP, idk about Azure and others)

Re: Reclaiming the lost art of Linux server administration

#148
post #87

Earlier quoted context omitted.

I remember how surprised people were when I demoed a $200/month bare metal server outperforming by a huge margin RDS MySQL instance that they were paying something upwards of 16k/month. IIRC we ended up using it as a disposable replica for some non-real time but heavy operations.

Here's what the bare metal server didn't come with: API access for managing configuration, version updates/rollbacks, and ACL. A solution for unlimited scheduled snapshots without affecting performance. Close to immediate replacement of identical setup within seconds of failure. API-managed VPC/VPN built in. No underlying OS management. (Probably forgot a few...) I get that going bare metal is a good solution for som…

fully agree with this, in my free time I would fully go for the baremetal, but if I have to save money to a company, by placing all the headaches that are solved by AWS then I just say goodbye

Re: Reclaiming the lost art of Linux server administration

#149
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

I'm curious how much you're paying for business fiber. Do you have IP transit in your home, or is it just a business connection?

Centurylink gigabit business fiber is about $140/mo, and a block of 8 static IPv4s is $25/mo

Re: Reclaiming the lost art of Linux server administration

#150

Earlier quoted context omitted.

> Blame the folks demonizing/shaming having "pet" servers and pushing immutable infrastructure. Linux server administration is quite enjoyable, and with how well apps these days can scale vertically, it really takes a special kind of workload to need (and actually saturate) fleets of servers. You don't need pet servers. Puppet or Ansible make your baremetal cattle.

I think most folks would argue "cattle" means using imaging to manage/replace your fleet. Using something like puppet or ansible against a fresh install implies a level of individualism towards each system as they "may" have minute details based on when puppet/ansible ran, even if they're part of a dynamic inventory of some sort.

[deleted]
Post reply on HN