Reclaiming the lost art of Linux server administration • Hacker News Board

Re: Reclaiming the lost art of Linux server administration

#411

Post by nix23 » Sat, Jan 29, 2022, 10:39 AM UTC

viraptor wrote: post #139

c2h5oh wrote: post #87

Earlier quoted context omitted.

I remember how surprised people were when I demoed a $200/month bare metal server outperforming by a huge margin RDS MySQL instance that they were paying something upwards of 16k/month. IIRC we ended up using it as a disposable replica for some non-real time but heavy operations.

Here's what the bare metal server didn't come with: API access for managing configuration, version updates/rollbacks, and ACL. A solution for unlimited scheduled snapshots without affecting performance. Close to immediate replacement of identical setup within seconds of failure. API-managed VPC/VPN built in. No underlying OS management. (Probably forgot a few...) I get that going bare metal is a good solution for som…

> Here's what the bare metal server didn't come with:

It's called software, you put that on top of a server, it's not some kind of magic.

Re: Reclaiming the lost art of Linux server administration

#412

Post by pid-1 » Sat, Jan 29, 2022, 10:44 AM UTC

LAC-Tech wrote: post #36

Full disclaimer, I'm very much not a sysadmin or devops guy. However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all. I couldn't figure out AWS. But I could figure out how to host sites on a linux VPS. So what's the story here - is serverless something that only makes sense at a certai…

I use serverless for internal systems.

- Small API that is used by a couple people everyday? Lambda.

- Need to store some data? DynamoDB.

- Need to store some files? S3.

- Cron? Step Functions.

- Need services to communicate with each other? SQS.

Those end up being either free or very cheap.

If you have high traffic, serverless is actually really expensive. It is only worth it if you have high scale but unpredictable / bursty traffic.

> However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all.

I understand AWS so others don't need to and I'm making the money of my life. All 3 big cloud providers have really terrible developer experience and I feel really sorry for folks who just want to get their shit done.

I used to think some other player would come in and offer something much easier and simpler to take over this space, but I'm not really seeing any serious contender. Just low code / no code platforms and managed k8s stuff.

Re: Reclaiming the lost art of Linux server administration

#413

Post by ozim » Sat, Jan 29, 2022, 10:47 AM UTC

warent wrote: post #34

When my SaaS app started scaling, I saw how badly cloud can be priced if you have even slightly unusual use-cases. It occurred to me that instead of spending ~$600/mo on GCP, I can invest in a $3000 PowerEdge server with much better hardware, run it out of my home office, and it pays for itself in less than a year. Running your own server is an investment that doesn't make sense for everyone. If you can get it, it is…

When I read small/medium business administrators horror stories on reddit and other forums I think IaaS is a god send.

Especially when someone else is making money decisions. Of course in horror stories admins have to run their servers in broom closets or in sheds because business owner is too cheap to get a proper space for something that whole company is running.

Re: Reclaiming the lost art of Linux server administration

#414

Post by belter » Sat, Jan 29, 2022, 10:52 AM UTC

selfmodruntime wrote: post #303

section_me wrote: post #260

Earlier quoted context omitted.

I think this is valid in many cases. A decent sysadmin in the US is (speculating) 200K USD, a good one in Europe is 120K+ euro. You have cheaper but they probably can't explaint a UDP datagram or know how ECMP over BGP works. It's the understanding of all of the underling tech which make them worth there weight in gold.

No Sysadmin in Europe is being paid 120k Euro. It‘s learned trade and you‘d be lucky to pull 70k.

The calculation from the management side of 120K Euro is correct. And you are also correct that they are lucky in Europe to get paid 70K.

To pay them around 70K, you need to budget for 120K. It will go into social security, taxes, training, overheard of management, HR team, other costs centers like a margin to account for sickness, absenteeism, people who are trying to "find themselves", personal crisis etc...

Re: Reclaiming the lost art of Linux server administration

#415

Post by goodpoint » Sat, Jan 29, 2022, 11:04 AM UTC

primax wrote: post #290

viraptor wrote: post #139

Earlier quoted context omitted.

Here's what the bare metal server didn't come with: API access for managing configuration, version updates/rollbacks, and ACL. A solution for unlimited scheduled snapshots without affecting performance. Close to immediate replacement of identical setup within seconds of failure. API-managed VPC/VPN built in. No underlying OS management. (Probably forgot a few...) I get that going bare metal is a good solution for som…

Also physical security, power, physical rack cost, networking, salaries to manage everything, and all those other hidden costs

Hidden costs? I've been in companies with their own datacenters from tiny to enormous.

The costs were very clearly spelled out and always lower that SaaS.

Why do people here think that cloud companies are charities? The cost of their hardware leasing, personnel, electricity, building management, insurance... it is all paid by the customer.

Plus a beefy profit margin. All paid by you.

Re: Reclaiming the lost art of Linux server administration

#416

Post by cblconfederate » Sat, Jan 29, 2022, 11:11 AM UTC

Lamad123 wrote: post #333

TacticalCoder wrote: post #74

A very weird thread that degenerated into: "PaaS vs self-hosted/self-owned hardware" . I'm pretty sure most people sysadmin'ing their Linux servers are actually doing it with rented dedicated servers. TFA btw specifically mentions: "don't manage physical hardware" . Big companies like Hetzner and OVH have hundreds of thousands of servers and they're not the only players in that space. They don't take care of "everyth…

How good is their 50 euro package? I wanna try it just for fun and dicking around.

for fun try https://www.hetzner.com/sb

Their prices have spiked because of electricity costs, the entry level used to be around 20 euros. They come with 1Gbit

Re: Reclaiming the lost art of Linux server administration

#417

Post by lambdaba » Sat, Jan 29, 2022, 11:11 AM UTC

trabant00 wrote: post #406

lambdaba wrote: post #242

Earlier quoted context omitted.

Hey I'm super interested in such sysadmin work if you could describe it here or share it on another channel (email in bio). Many thanks.

What exactly are you interested in finding out? What I do every day? How you can find such work? Or?

Yes, basically if I could peek into your activity for a day it would be awesome (I'm not asking that, although I would like it :P :)). I'm just curious of what a day's work looks like.

Re: Reclaiming the lost art of Linux server administration

#418

Post by goodpoint » Sat, Jan 29, 2022, 11:28 AM UTC

solatic wrote: post #102

Why can't people just pick the right tool for the job? The truth behind these managed services is that, for the correct usecases, they are VERY cheap. And for the wrong usecases, they are RIDICULOUSLY expensive. Most businesses have nightly cronjobs generating some kind of report that is then emailed to stakeholders. Why on Earth would you run a dedicated Linux box for that anymore? Glue a nightly trigger to AWS Lamb…

> Glue a nightly trigger to AWS Lambda

You are forgetting the future cost of losing knowledge and control over your infrastructure.

Re: Reclaiming the lost art of Linux server administration

#419

Post by superleo2020 » Sat, Jan 29, 2022, 11:36 AM UTC

christophilus wrote: post #313

kirubakaran wrote: post #275

Earlier quoted context omitted.

R820s idle at ~200W and consume ~500W at 100% CPU load on all 80 virtual cores.

So, if we say it draws around 350W, that makes it around $32 /mo power (rough calculation based on a 12.5cent (US) kWh rate, right? Not bad for all that beef.

And for Western Europe readers as a comparison...

UK UK >=Apr2022 = £76.16 ($102 USD) £0.3022 likely new price cap rate

Thus the reason r/homelab users in Europe tend not to have Craigslist blade servers of any sort!

Re: Reclaiming the lost art of Linux server administration

#420

Post by southerntofu » Sat, Jan 29, 2022, 11:48 AM UTC

nix23 wrote: post #384

akireu wrote: post #119

Earlier quoted context omitted.

It's 2022 and we're about to rediscover something we know for 40 years already: mainframes are freaking expensive.

You misread something, it's 2022 and we discover that mainframes can be cheaper then a AWS-cloud-machine-park ;)

I thought AWS was the mainframe in the parent comment?!