Live data from Hacker News

Reclaiming the lost art of Linux server administration

pietrorea.com

101–110 of 485 posts

Re: Reclaiming the lost art of Linux server administration

#101
post #34

When my SaaS app started scaling, I saw how badly cloud can be priced if you have even slightly unusual use-cases. It occurred to me that instead of spending ~$600/mo on GCP, I can invest in a $3000 PowerEdge server with much better hardware, run it out of my home office, and it pays for itself in less than a year. Running your own server is an investment that doesn't make sense for everyone. If you can get it, it is…

> I can invest in a $3000 PowerEdge server with much better hardware And when some component of the server fails, your app is unavailable until you can repair it. So you need another server for redundancy. And a load balancer. And a UPS. And a second internet connection. If your app is at all critical, you need to replicate all of this at a disaster recovery site. And buy/run/administer DR software. And hardware has…

I am not the guy you replied to, but I also self host my web apps. I think every project is different and not all projects demand near 100% uptime. I certainly strive for HA for my projects but at the appropriate budget and my users understand.

If you are trying to go commercial you might have a different attitude but for those of us who do this mostly for fun and for some donations on the side, over complicating our setups to ensure we add a 10th of a percent to our uptime stats just isn't worth it.

Re: Reclaiming the lost art of Linux server administration

#102
Why can't people just pick the right tool for the job? The truth behind these managed services is that, for the correct usecases, they are VERY cheap. And for the wrong usecases, they are RIDICULOUSLY expensive.

Most businesses have nightly cronjobs generating some kind of report that is then emailed to stakeholders. Why on Earth would you run a dedicated Linux box for that anymore? Glue a nightly trigger to AWS Lambda, send the report via AWS SES, and it's free. Literally, because it fits quite easily within the free plan. No $5/month VPS box, no patching, no firewalling, no phone calls from execs at 6 AM wondering why their report isn't in their inbox and you track it down to the server being down when the cronjob was supposed to fire.

With that said, if you come to me and tell me what you want to add a new business feature to stream video for our customers off AWS, I'll first ask you why didn't you tell me you won the lottery, then I'll berate you for being stupid enough to spend your lottery winnings on the AWS bill.

Pick the right tool for the job.

Re: Reclaiming the lost art of Linux server administration

#103
post #89
post #44

Earlier quoted context omitted.

Do you have a static IP? I have a homelab too but getting “enterprise grade” service from comcast seems to be my biggest barrier to scaling without leaning on aws.

Dynamic DNS (dDNS) works here[0]. You have free services like no-ip, and also most paid domain registrars support this. I know both Namecheap and AWS Route 53 support it if you want it at your own domain. Essentially, it's a cron curling with an API key from the host machine, that's it. Works great in my experience.

Keep in mind you will have a small downtime until the new IP is registered. Also the cache TTL of your domain will be very low, so your site will have a small loading time penalty from time to time.

Re: Reclaiming the lost art of Linux server administration

#104
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

I'm curious how much you're paying for business fiber. Do you have IP transit in your home, or is it just a business connection?

100mb FiOS Business is on the order of $70.

Re: Reclaiming the lost art of Linux server administration

#105
post #34

When my SaaS app started scaling, I saw how badly cloud can be priced if you have even slightly unusual use-cases. It occurred to me that instead of spending ~$600/mo on GCP, I can invest in a $3000 PowerEdge server with much better hardware, run it out of my home office, and it pays for itself in less than a year. Running your own server is an investment that doesn't make sense for everyone. If you can get it, it is…

I'm just curious, because to me it seems a little bit unrealistic. How do you handle traffic spikes, especially from the networking point of view? What kind of connection do you have? How do you make your service as fast for all customers around the world (saying you have a succesful Saas). How do you prevent a local blackout from taking down your service? Where do you store your backups, in case your building gets f…

I've been doing my own hosting for 20 years, this just doesn't happen often enough to concern myself with it.

You need to disassociate yourself from the start-up mindset when you DIY a side project app or site. Having said that, there are ways to cache and improve your write performance and maintain HA on a budget. The only thing that's hard to replicate in self-hosting is a high performance global presence.

Re: Reclaiming the lost art of Linux server administration

#106
post #63

Earlier quoted context omitted.

> I can invest in a $3000 PowerEdge server with much better hardware And when some component of the server fails, your app is unavailable until you can repair it. So you need another server for redundancy. And a load balancer. And a UPS. And a second internet connection. If your app is at all critical, you need to replicate all of this at a disaster recovery site. And buy/run/administer DR software. And hardware has…

You need to replicate in Cloud too, most people tend not to because they think the cloud is magic, but it's computers and computers can fail- even if they're someone else's. Also "if some component fails or the app is critical" has a lot of nuance, I agree with your sentiment but you should know: 1) Component failures in hardware are much rarer than you think 2) Component failures in hardware can be mitigated (dead r…

I don't know what they call that logical fallacy cloud fanatics use when they say "if blah blah just make build your own datacenter".

Re: Reclaiming the lost art of Linux server administration

#107
post #47
post #44

Earlier quoted context omitted.

Do you have a static IP? I have a homelab too but getting “enterprise grade” service from comcast seems to be my biggest barrier to scaling without leaning on aws.

Hey, you actually have a few options, notably, doing nothing! Comcast doesn't actually change your public IP address between DHCP renewals and thus it's effectively static. The only time that it'll change is when the modem is powered off for an amount of time, or the upstream DOCSIS concentrator is powered off for maintenance or otherwise.

I would be more worried about violating the ISP's terms of service. Running a business based on that seems pretty precarious.

Re: Reclaiming the lost art of Linux server administration

#108
post #44
post #34

When my SaaS app started scaling, I saw how badly cloud can be priced if you have even slightly unusual use-cases. It occurred to me that instead of spending ~$600/mo on GCP, I can invest in a $3000 PowerEdge server with much better hardware, run it out of my home office, and it pays for itself in less than a year. Running your own server is an investment that doesn't make sense for everyone. If you can get it, it is…

Do you have a static IP? I have a homelab too but getting “enterprise grade” service from comcast seems to be my biggest barrier to scaling without leaning on aws.

I wrote a script that basically hits an offsite DNS with my latest IP. It's worked quite well. I think in the past 4 years I have had Comcast, they haven't changed my IP once.

Re: Reclaiming the lost art of Linux server administration

#109

Why can't people just pick the right tool for the job? The truth behind these managed services is that, for the correct usecases, they are VERY cheap. And for the wrong usecases, they are RIDICULOUSLY expensive. Most businesses have nightly cronjobs generating some kind of report that is then emailed to stakeholders. Why on Earth would you run a dedicated Linux box for that anymore? Glue a nightly trigger to AWS Lamb…

This is the real truth. People complain about certain services and theirs costs (Lambda being one I've heard) but I have a full side project that runs on lambda with extremely bursty traffic and it couldn't be more perfect. If I had sustained activity I might look into something else but it really does come down to picking the right tool for the job.

Re: Reclaiming the lost art of Linux server administration

#110
post #86
post #61

I have over 20 years of Linux/FreeBSD sysadmin experience ranging from universities to major silicon valley companies in both cloud and on-prem. When it comes to companies I mostly support cloud these days but when it comes to me and my family I accept every downside and host as almost all of our digital lives in a 42u rack in a gutted closet in our house with static IPs and business fiber. I know where our data live…

Yes but you have talent and a lifetime of experience, plus space for a noisy 42u rack full of servers, but not everybody does...

You don't need a 42u rack. You can run a cluster of Raspberry Pi's hidden in your basement ceiling rafters like me.
Post reply on HN