181–190 of 241 posts
I, too, have a server running 16.04 that I'm afraid to update. It currently has an uptime of 1281 days... at this point I'd feel bad rebooting it
What is there to be afraid of? Don't you have backups? Also, debian/ubuntu systems can easily be setup to auto update and reboot on a regular basis, leaving you manual maintenance only for the larger version upgrades.
Earlier quoted context omitted.
There's threats on the internet, so don't spin up servers? Idk am I reading into that unfairly? That seems pretty fear mongering to me. Lots of engineering goes into making things safe for engineers to build on. Of course you can also just use squarespace and not worry about it at all. Perhaps my security posture is just not as intense as yours but I'm really just not super concerned my blog is going to get pwned. If…
Servers are work, including security overhead, so yes, don't spin them up if there is an alternative solution that is superior in every way except for not being able to churn digital butter.
Earlier quoted context omitted.
"The biggest mistake I made was high uptime" Quite. I'm old enough to remember machine uptime being a badge of honour. However, being older and not really wiser, I look for service uptime these days. Yes we did have similar back in the day, that's why MX and the like DNS records exist. Old school clusters were pretty esoteric but the lessons were learned (split brain n that) and that's why we still argue the toss wit…
There is something like live patching. One reason mainframes and micros are still around us, is that you can change almost everything between hardware and software without downtime. It is also available in commercial surviving UNIXes, and as paid for feature in some Linux distros, although not to the extent that those grandparent systems are capable of.
You can build that way cheaper with 2-3 proper clustered load balancer units, 2-3 application servers behind those and those using persistent storage (databases,ldap, files) which allow writing multiple nodes simultaneously.
I used to work uni that we had few services from 2012 to 2025 my retirement with zero downtime. One time my manager with tech background tried to add PBR in hurry using WebUI and did not understand cli syntax and caused close to require reboot, but I was able to fix it from cli rolling back previous config and rebooting one unit at time. Upgrading software major version up to each unit supported level wasn't hard, upgrade node it joins back cluster, upgrade another node and it joins cluster, all done. Few times I had to fix manually config for some less important test backend servers that I had forgotten to change before upgrade. No big deal. No major outages during all that 13 years time happened. Some redirecting policy and action syntax was first hard to understand and learn like GeoIP, but I was very surprised how darn reliable and nice they to use and maintain.
The LB's were (Citrix) Netscalers in clustering mode (all nodes process traffic concurrently), which allowed live update one node at time without losing any connectivity through them. That wouldn't have been possible devices in just HA mode.
We had just 2 beefy units which worked very well for us, but you can have 2-32 of them in cluster and managing thousands of servers behind them if you need that. Netscalers are FreeBSD derived where quite a bit of the TCP/IP stack was rewritten adding support many some quite odd features std FreeBSD doesn't have. Much of that is IP/ethernet multicast features, PBR's, Traffic Domains (VRF's) and of many service and monitoring processes which sync cluster (or HA) and if node fails another can continue straight from there without any loss of traffic to clients being proxied.
Though I think most people in this forum are familiar with with haproxy, pound and web-server software provided reverse proxying.
A car analogy if previous were your fancy sport sedan Netscaler and F5 BigIP are formula F1 class cars ie. quite different beasts altogether.
e: And proper LB's are not just for HTTPS etc. but very nice proxying many other protocols were they TCP, UDP or something else. We did done VPN's and something like Cisco AP'S CAPWAP (DTLS ie SSL over UDP). e: typo.
Earlier quoted context omitted.
There is something like live patching. One reason mainframes and micros are still around us, is that you can change almost everything between hardware and software without downtime. It is also available in commercial surviving UNIXes, and as paid for feature in some Linux distros, although not to the extent that those grandparent systems are capable of.
You should't need mainframe for 100% (or five nines if that's fine) service uptime. You can build that way cheaper with 2-3 proper clustered load balancer units, 2-3 application servers behind those and those using persistent storage (databases,ldap, files) which allow writing multiple nodes simultaneously. I used to work uni that we had few services from 2012 to 2025 my retirement with zero downtime. One time my man…
Hence my second paragraph.
Thanks for sharing the story.
Earlier quoted context omitted.
What is there to be afraid of? Don't you have backups? Also, debian/ubuntu systems can easily be setup to auto update and reboot on a regular basis, leaving you manual maintenance only for the larger version upgrades.
Backups are not going to help if a reboot breaks something.
A reboot doesn't break anything. Bugs do.
Any time I had a regression after a kernel update on a linux distro I could boot it on a prior version from the grub menu. Any time I had a regression with a software package I could rollback to a prior version. Rolling back updates is a problem that has been solved for decades, at least on linux systems.
The key with unattended upgrade is you want to have decent monitoring to make sure you never run out of disc space and do not figure it out weeks later if you have had an issue.
Earlier quoted context omitted.
What is there to be afraid of? Don't you have backups? Also, debian/ubuntu systems can easily be setup to auto update and reboot on a regular basis, leaving you manual maintenance only for the larger version upgrades.
You do not reboot systems for regular updates. Only in case of critical kernel updates do you consider it
In the end it is easier to schedule a weekly reboot window if packages have been updated. You aren't running a single server if you are interested in 99.99999990% of uptime anyway.
Imho a regular reboot is good practice: you are more likely to remember what you did a week earlier if an app/service fail to restart after you tweaked a config file than if it happens months later.
There is no reason to be afraid of reboot when they happen on a regular basis.
I enjoyed my foray into trying FreeBSD for my personal server. There's something cool, clean, simple and "punk rock" about it. But I gave up as my main pain points were: - PM2 was buggy on FreeBSD, which I used to manage my processes - An alternative, using `rc.d` to run daemons was just so hard to get logs working. - The firewall required too much self configuration to get it right with all the best security practic…
Earlier quoted context omitted.
The problem with live patching is twofold. First, you might not reload everything in memory, so it will be patched on disk but not in process. Second, you have not tested that the system can boot to a functional system. Say you have done live patching for 5 years and never rebooted, and then you have a power loss or hardware failure/upgrade that takes the system down. When you try to bring it back up, it doesn't work…
Which is moot, because of the system is important enough you'll have an automatic failover to another system running on standby All this "we must reboot to test" is bullshit excuses by unqualified workers
I enjoyed my foray into trying FreeBSD for my personal server. There's something cool, clean, simple and "punk rock" about it. But I gave up as my main pain points were: - PM2 was buggy on FreeBSD, which I used to manage my processes - An alternative, using `rc.d` to run daemons was just so hard to get logs working. - The firewall required too much self configuration to get it right with all the best security practic…
My main pain points were that it doesn't survive power hits. If your power goes out, it will reboot and ask you to manually fsck the filesystem.
I was running Ubuntu 16.04; migrated to FreeBSD and I'm all in. Between 16.04 and the current version of Linux; the ecosystem shifted. It's values shifted in ways that did not align with me. This mis-alignment is what motivated me to boot-up FreeBSD. I'm glad I discovered it. I found my happy place again. It's an incredible journey to take--whether you stick with it or not. Migrating to FreeBSD gives you new eyes int…