Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Hosting Dozens of Web Applications and Services on a Single Server

cprimozic.net

Post reply on HN
Print view View original

151–160 of 285 posts

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#151

Post by whoevercares »

unclebucknasty wrote: post #140
nicoburns wrote: post #72

Earlier quoted context omitted.

The problem with AWS is that you can’t really do anything without understanding 1. IAM, and 2. VPCs/networking. And these are probably the two most complicated parts. For DIY you’re probably best off avoiding AWS

I had this feeling when I first started with AWS years ago. It was hard to find a good overview and all of the Amazon doc on individual services seemed to start in the middle. So, a lot of my initial understanding came through intuition, and trial and error. For many scenarios, you can completely ignore IAM, but it's definitely not advisable. On the VPC side, it's actually fairly straightforward, but you may need to…

But how can we trust the DIY stuff meet compliance and hold the right security bar? It’s much easier to do with AWS.

Or maybe as a startup, to-C website you don’t really care

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#152

Post by FpUser »

dspillett wrote: post #136
FpUser wrote: post #129

Earlier quoted context omitted.

>"ability to scale" I write C++ business API servers. On the type of hardware I rent from Hetzner /OVH they're capable of processing thousands of requests per second. This would cover probably 90+ percent of real world businesses. >"an API for everything" Not sure what you mean here. API for what in particular? My servers talk to partner systems of real businesses. We consume their APIs and they consume ours. Integra…

> > "an API for everything" > Not sure what you mean here. I assume referring to APIs for controlling and monitoring the infrastructure, not anything about APIs you may provide in your application or consume from external sources.

What infrastructure? Thanks to my primitive approach I hardly have any.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#153

Post by dspillett »

vaxman wrote: post #42
tpetry wrote: post #28

Earlier quoted context omitted.

Raid-1 is doing exactly what you recommend without any effort. A perfect replica of the disk. And if the other one dies, who cares, the beauty of raid-1 is you don‘t need the other one to have a full copy.

Again, both NVMe modules are likely to fail simultaneously when used in a RAID-1 mirror on the same chassis, controller and PSU, under the same workload, especially if they are the same model and age.

I'm not sure this issue is significantly worse for SSDs compared to other drive types, except once they get really old and are close to EOL as defined by the amount written, though I'm happy to be proven wrong if you have some references where the theory has been tested.

If you are really worried about that, perhaps artificially stress one of the drives for some days before building the array so it is more likely to go first by enough time to replace and bring the array back from a degraded state?

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#154

Post by alohahola »

lucb1e wrote: post #21

I wonder what the environmental impact of a 24/7 running system with such high-end specs is. Desktops are worse with a graphics card (just yesterday I noticed my system's power consumption doubles when the GPU turns on: 20W with 100% CPU on all cores and WiFi stressed; 37W with additionally primusrun glxgears running), but desktops only run on demand. Dedicated non-mobile hardware doesn't scale to demand that well an…

I run a home-server with pretty high spec and how much extra juice it has left over has been bothering me. It feels wasteful.

My server isn't bound by CPU almost at all - the most intensive thing is the occasional trans-coding for jellyfin, however it does use quite a bit of ram for a couple of minecraft servers and ZFS. I'd really like some sort of preferably arm or maybe risc-v based server (in the future) that can take sata and a good bit of ram but most of the SBCs I see would require that my drives work over USB which is annoying and usually dont have more than 4/8gb of ram.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#155

Post by throwaway984393 »

BenjiWiebe wrote: post #113
throwaway984393 wrote: post #78

Earlier quoted context omitted.

The fact that it's a dedicated physical host means that any of a dozen components could fail and bring the box down permanently. The hosting company would have to reassign or build a new server for the customer, which could take from an hour to a week (or more in a pandemic with scarce parts). It depends on their excess capacity and on the machine's specifications. If it was one virtual machine I'd say, sure, just us…

It's true that it could fail in 10 weeks, but it's pretty likely that it lasts the whole 10 years. Computer hardware is pretty good like that. Especially if you've got power conditioning and a UPS so it never even reboots (except when you want it to). At home I'm running an old desktop PC with Linux for my router. It's running on >10 year old hardware. I've got several other old PCs running, too, that's just the olde…

'Pretty likely' isn't a measurement, it's just a way to dismiss thinking about the problem. Most hard drives, for example, are only reliable for 3-5 years. Other components have varying lifespans. And that's just an average; in a hot, vibration-filled building with heavy carts banging into server racks, it gets more dodgy. And I wouldn't bet on the datacenter even having a UPS or multi-phase power set up correctly.

Assuming your one giant server will be as reliable as a 10 year old box sitting on your desktop is a gamble. All I'm saying is be prepared to lose.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#156

Post by bryanrasmussen »

t-3 wrote: post #64
bryanrasmussen wrote: post #46

Earlier quoted context omitted.

I think just the typical evaluation of someone with expertise in doing something, it's easy if you're a real devops kind, then just put together a bunch of things, do some config files, write a make or two, take two or three hours to do something that the rest of use take a week to do. in the same way that I might set up a service to scrape XML files and fill up my ElasticSearch search instance, and take a couple hou…

"Easy" is probably because it boils down to, install X, Y, Z, edit config for X, Y, Z, start daemons X, Y, Z. There's no complex math or thinking involved, just plugging stuff in.

sure but you have to spend lots of time reading documentation and know stuff to figure out you need to put these things together.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#157

Post by whoevercares »

FpUser wrote: post #77
exhaze wrote: post #66

Three independent, but somewhat related thoughts on this topic: 1). On HOWTO articles about infra (1/2): I'd like to see more articles that lead with requirements, rather than setups that then justify the setup with requirements. Like, congrats, you managed to host a bunch of web applications via containers on a dedicated server. It's really nice for a super personal project and I'm sure it helped OP gain a lot of op…

There is a life beyond AWS. I host applications (including ones responsible for 10s of millions in revenue) on dedicated servers rented from Hetzner and OVH. Do not even do containers as my normal deployment is server per particular business (with standby) and the cost of renting a dedicated server comparatively to revenue is microscopic. CI/CD / setup from scratch / backup / restore is handled by a single bash scrip…

Playing devil’s delegate with a normal process how we launch new service in big tech:

1. Have you done a security review and ideally pen-testing from a 3rd party? Are you fully TLS from end to end, are you aware of any exploitable vector from your API/UI and how you mitigate them

2. How do you handle software patching and vulnerabilities?

3. Do you consider your app Operational ready? Can you rollback, do you audit changes, do you need multi-AZ resilience, do you need cell based arch. Did someone other than yourself or outside your team look at your monitoring dashboard? Do you need to be paged at 3am if some of your dependencies or underlying node degrades. We have to answer a 50 questions template and review it multiple times..

4. Did you calculate the cost and forecast it with how the service/app may grow in the next 12-36 months?

While you still need to do all this when using a Cloud providers, you probably _should_ do much more if you manage the bare metal

If you have already done all those, Kudos to you, but I still find it hard to trust everyone who DIY

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#158

Post by kakoni »

PascalW wrote: post #27

I'm doing something similar though I've opted specifically to _do_ use Kubernetes via k0s [0]. It works wonderfully well and allows me to use most things that are available in the k8s ecosystem like cert-manager [1] and external-dns [2]. All configuration is stored in Git and the server itself is basically disposable as it only runs k0s and some very basic stuff like iptables rules. I see this sentiment quite a lot t…

Do you have any estimates how resource hungry k0s is? Ran few resource constrained k3s clusters, where 25% of cpu was always spent on running k3s itself.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#159

Post by gogs »

We kind of moved away from this monolithic setup tho. Oh you can scale-up your hardware like we used to, it will work, until some point. And the day you will need to split your services will come quickly.

This is a perfect solution for early stage projects if you're in a startup or for a personel project, but monoliths are....well....monoliths.

Re: Self-Hosting Dozens of Web Applications and Services on a Single Server

#160

Post by justin_oaks »

kilroy123 wrote: post #25
progx wrote: post #13

Easy setup with: - traefik (nginx proxy with auto letscencrypt) - portainer (docker container management) - fail2ban (basic security) - logwatch (server / security stats by mail) - munin (server stats) - restic (cloud backup) - unattended-upgrades (auto install security updates) - apticron (weekly info) - n8n (automatisation for e.g. quick info via telegram, if something not work) Run every app that you want in your…

This is _very_ close to my exact setup. Though, I never used portainer (just manage a big docker-compose file) is it worth it?

Portainer does make certain tasks much easier. I use Portainer to clean up any Docker volumes, containers, and images I no longer need. Doing that for a lot of Docker resources using the command line gets old really fast.

I say this as someone who uses docker and docker-compose extensively and I'm very comfortable with a CLI.

Post reply on HN