You don't
need two servers rented, even for 99.99% uptime.
That 99.99% uptime is achievable with a single dedicated server, plus cloud as a cold backup and some monitoring to trigger spinning it up.
You will spin up the cloud server when the dedicate server is down. It will cost, but only for brief periods of time.
That's sometimes cheaper than two dedicated servers.
I prefer more than 1 dedicated server, but it's definitely possible to get the 99.99% uptime with just 1.
And in some ways the extra diversity of having a different kind of backup is useful, e.g. for burst scaling as well, and for peace of mind not relying on a single provider.
Even better, you can (and should) run the applications in VMs, or in containers inside VMs. They perform well these days, and even a severe application crash won't take down the host.
Then regular application upgrades don't need a separate host server. Instead you're spinning up and shutting down VMs.
You can also migrate VMs between hosts, and this can be done live if you want. It takes some setting up to have working distributed storage and movable networking, but it's possible.
However, you might decide not to bother with that, as starting and stopping instances and having fleets of instances managed by something like K8S is modern practice for other comforting reasons anyway.
Then the only "big" upgrades are limited to host server upgrades and major network reconfigurations, both of which you will do rarely. You might do host server security upgrades (kernel etc) more often, but that costs the time of a reboot unless it goes wrong.
If those are planned or automated, those "big" upgrades can be done with zero user-facing downtime except in the event of failure, in which case it's limited to the cold-spare startup time
So provided you have the backups in place, maybe on the cloud as a cold-spare so you're not paying for it except when it's needed, you can still have that 99.99% uptime for approximately the cost of one server.
You do not need a separate load balancer device for uptime specifically; only if load balancing is something you want anyway for other reasons. For seamless transitions between backend servers, if you have multiple servers running at a particular time, there are other methods.
On occasions you might instantiate a cloud load balancer temporarily to cover downtime during a planned host upgrade that can't be routed around another way. Like with a cloud cold-spare server, you don't pay for it except when it's temporarily instantiated.