Self-Hosting Dozens of Web Applications and Services on a Single Server • Hacker News Board

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#251

Post by gmfawcett » Sat, Jan 01, 2022, 5:08 AM UTC

rajasimon wrote: post #212

rudasn wrote: post #3

Great post with lots of details. I'm sure I'll try some of tools mentioned. We do something similar, multiple django sites and static sites on a single digital ocean droplet. We use docker for everything though, and have a single nginx listening on 80/443 that routes traffic to the sites by domain. Each site has a separate pg db and everything is glued together / configured / deployed via ansible.

Do you think I can make this without Docker? I afraid to use docker I don't get it. Why not just systemctrl restart webserivce1 or 2 ?

Of course you can. That approach makes different tradeoffs, of course, but it's entirely possible. I have a "legacy" server with ~20 small/medium, non-containerized applications running on it (Django, older Python frameworks, some Lisp & Scheme stuff, etc.), all running as system services and sitting behind an Nginx front-end.

But I can see the merits of containers, and might containerize these apps & services some day.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#252

Post by kasra85 » Sat, Jan 01, 2022, 5:34 AM UTC

Aeolun wrote: post #73

tweetle_beetle wrote: post #20

Really good write up and always nice to see the honesty about past mistakes. For those without time to get hands quite as dirty, like me, I've found that CapRover[1] gives just enough UI to fiddle while having sensible defaults. [1] https://caprover.com/

The only negative about caprover I’ve found is the distinction between containers with persistent storage and not. I don’t want to recreate my whole app just to add persistent storage. Otherwise I’ve been using it without issue for months now.

Kasra from CapRover. Feel free to open a Feature Request on Github. There is no architectural design limitation that prevents this. It can certainly be added.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#253

Post by SturgeonsLaw » Sat, Jan 01, 2022, 6:17 AM UTC

ornornor wrote: post #239

progx wrote: post #13

Easy setup with: - traefik (nginx proxy with auto letscencrypt) - portainer (docker container management) - fail2ban (basic security) - logwatch (server / security stats by mail) - munin (server stats) - restic (cloud backup) - unattended-upgrades (auto install security updates) - apticron (weekly info) - n8n (automatisation for e.g. quick info via telegram, if something not work) Run every app that you want in your…

How do you deal with backing up the data/“disaster” recovery? (I out disaster in quotes because we’re talking a about home servers so not mission critical data but still pretty annoying to lose.

If you have OP's level of control over the hardware (ie; you have a dedicated server on which you can access the hypervisor), then taking incremental backups of the entire VM is the best way to ensure you can hit the big undo button if anything goes wrong.

The most important, and neglected, part of backup & restore, is the restore. If resources permit, this is where I like to use a battle tested solution instead of rolling my own with scripts. For my self hosted servers I use Veeam, but there are many good alternatives.

It's nice having the option to restore the entire VM to a specific state, and to also be able to mount backups and pull files if only a few need to be restored. It's also handy to be able to spin up a backup into its own virtual machine.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#254

Post by devoutsalsa » Sat, Jan 01, 2022, 8:12 AM UTC

vangelis wrote: post #209

t-3 wrote: post #64

Earlier quoted context omitted.

"Easy" is probably because it boils down to, install X, Y, Z, edit config for X, Y, Z, start daemons X, Y, Z. There's no complex math or thinking involved, just plugging stuff in.

Rebuilding a transmission is easy in the same way, undo some bolts, replace some parts, press in some seals, and screw some bolts back in!

It takes practice. I just started making smoothies in a blender, and there are a bunch of little things to know to make it a little easier on yourself. It’s not just “throw all your shit in a blender and push button”, at least not with the cheap blender I have access to.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#255

Post by jmatthews » Sat, Jan 01, 2022, 8:31 AM UTC

Thanks for the write up. I use roughly the same set up minus docker. My deploy is usually log in to ssh, git pull and restart the systemctl service.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#256

Post by BrandoElFollito » Sat, Jan 01, 2022, 9:13 AM UTC

I have been self hosting on and off since about 2000. When I got my fiber link (4 or so years ago) I also started to do home automation and the self hosting got more serious ("national security infrastructure" as my wife calls it, her to-go question is "what will happen with that shit if you die tomorrow?").

I run everything in docker containers, except Wireguard and the backup.

I had an idea to "document everything" but the realized there would be no-one to maintain the services (neither my wife not my children are interested in IT). I then decided to document how to untangle themselves from the self-hosting (moving to well-established services, dismantle the home automation, ...). Thi sis going to make more sense in the long term.

Since I do not plan to die tomorrow but I do not know for my server, I tried to do a disaster recovery from scratch. It was surprisingly easy - after starting a VM on my laptop (simulating the bare-metal rebuild of my server), I was up and running in about an hour. Docker is simply fantastic for that.

I took some notes about the disaster recovery, which I promptly lost so I will have to run this again soon (maybe next week). Actually I will copy the encryption key for my remote backup right away as this is the only thing I won't be able to recover. EDIT: done :)

TL;DR:

- use docker containers for your services, it is a life-saver and makes everything easier

- use a docker-compose script to have everything on file

- document either your maintenance or exit strategy

- do a quick DRP test on a VM

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#257

Post by forty » Sat, Jan 01, 2022, 9:19 AM UTC

FpUser wrote: post #129

throwaway984393 wrote: post #90

Earlier quoted context omitted.

The business benefits of the cloud are ability to scale, an API for everything, cheap automation, cheap managed services for everything, and good customer support. Like you say for your model none of those things may be important, but they're killer features for others

>"ability to scale" I write C++ business API servers. On the type of hardware I rent from Hetzner /OVH they're capable of processing thousands of requests per second. This would cover probably 90+ percent of real world businesses. >"an API for everything" Not sure what you mean here. API for what in particular? My servers talk to partner systems of real businesses. We consume their APIs and they consume ours. Integra…

Of course there are use cases where not using cloud facilities is cheaper, but unless your business is low margin the difference might still be irrelevant and having the managed services available easily can be handy if you ever need them.

There are many use cases where hosting fee is not the right thing to optimize for.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#258

Post by diordiderot » Sat, Jan 01, 2022, 10:18 AM UTC

exitheone wrote: post #24

padde wrote: post #9

Nice write-up :) I'll probably adopt some of these fancy new tools like nginx or munin (that you call old) some day... still running good old Apache + php, postfix, dovecot, irssi, ... I think my box (in its various reincarnations, first as dedicated server, then as VM on a bigger dedicated server that I shared with friends, and now as cloud VM) has been running since ~2005. Now I feel old ;)

Same here. It's amazing how many services you can easily host if they are on PHP and low traffic. The beauty of PHP is that more services consume no additional resources until they are used. This lets you run many many services without worrying about resource usage. In stark contrast to running everything in containers where you have at least a process per idle service.

Why does php not need to idle?

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#259

Post by belter » Sat, Jan 01, 2022, 1:00 PM UTC

galaxyLogic wrote: post #192

This all seems quite complicated. Wouldn't it be easier to use Amazon Lambda or similar service. Spend your time creating and optimizing the applications instead of the infrastructure. I've known people who built their own boat and also built the engine for the boat. And it's a great hobby which gives you a great sense of accomplishment. But, if you just need to get some fishing done, buy the boat with the engine alr…

Uhmmm...but then why go fishing when you can go to the supermarket?

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#260

Post by southerntofu » Sat, Jan 01, 2022, 3:59 PM UTC

remram wrote: post #143

southerntofu wrote: post #47

Thanks for this article, it's great to see people caring for their server (does it have a name?) and not defaulting to the serverless craze. Here's a few thoughts :) > there is some small downtime when I deploy new versions of things since I don't have any load balancing or rolling deployments It's entirely possible to achieve, depending on your stack. `nginx -s reload` will reload the entire config without killing e…

> there is some small downtime when I deploy new versions of things Some time ago I was looking for an easier way to fix this. It seemed to me that a good way would be to have the reverse proxy (e.g. nginx or similar) hold the requests into the app restarts. Fit the user, this would mean a ~10s hiccup rather than 504 errors. I didn't find an easy way to do it with nginx though and was sort of disappointed. Maybe othe…

> [edit: one app I host can only be run single-instance as it keeps some state in-process. It can outsource it to Redis, but that seems overkill if it's only needed during upgrades, 10s/week or so]

Not exactly the same, but couldn't you serve a 425 Too Early with a meta refresh of 10s to achieve exactly the same goal? So change your nginx config to serve this "updating, your browser will refresh automatically" page, reload nginx, update the app, revert nginx config and reload nginx. Would that not address your needs, albeit in a more convoluted way?