Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Solo-preneurs, how do you DevOps to save time?

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 328 posts

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#231

Post by blablablub »

For a couple of persons team it is important to keep the changes small, so errors can easily identified. That means daily or bi-daily upgrades. It also helps if you focus on a certain small project and finish that, before moving on to the next project.

SVN checkout development, meld into testing. Copy production db into testing. Run a db upgrade script in testing. Test the new function for a day. If all ok stop the production HAproxy at 3am. run production DB backup, Checkout testing into the production server. mount the new dir into the NGINX path. Upgrade+start the DB, start production HAproxy.

Except for the meld and testing part, this is all automated.

A rollback on the app servers is pretty simple, just remount to the previous location. Even though there is a backup of the DB, a rollback there is not really feasible.

Everything is running in LXC containers, which I treat as cattle. Creating new ones is either automatic based on time/demand/failover or a few clicks in a custom web interface. HAproxy automatically pics them all up via DNS.

SVN and production DB are always backed up nightly and copied to a S3 compatible storage. Everything older than 6 months gets deleted. All the free space on the servers is globbed up with glusterfs to give a decent amount of fast,free storage.

Using Bash as scripting language, there is literary no problem in devops which hasn't been solved with bash and the solution is somewhere in the internet.

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#232

Post by bredren »

Solo founder, sold my last SaaS I had set up using "spooky arcane oldhat sysadmin deploy techniques" h/t to @Fiahil for that shade.

On that it was a Django product with deploy scripts for staging and production, i.e. `./deploy_production` ran:

   #!/bin/bash
   ssh  web@[production ip]
Where `pg.sh` pulled from `master` ran static and migration management commands and restarted the web server.

Last summer I moved away from the spookies and now do all deployment using Github Actions.

Commits to either a feature branch or `develop` run tests and all necessary deployment steps to a staging instance. (my stuff so far does not require k8s) and all containers are put up using docker compose. Commits to `main` (or `master` depending on the project)

I blended a few guides written by Michael Herman to build my GA-based CI, and it took several weeks to work on the scripts and learn Github Actions. But it was time absolutely worth investing because my DevOps is modern enough now.

No more bare metal stuff. No more scripts. Nice visual display of deployment success, "free" VMs for the deployment and CI UI integrated alongside my issues and code all on Github.

Importantly, I still use git flow. I tried moving to trunk based, but the macros inherint in git flow (including the pycharm plugin) make this production / staging // master / devops or feature/ work very well (I'm not making PRs to myself)

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#233

Post by xrd »

GitLab and Dokku.

You can run almost ANYTHING inside dokku. I use it to run a private/public docker registry, minio (amazon S3 clone), analytics via matomo, and a ton of mini apps that share a bunch of databases. It's incredible.

With dokku, deploying your app is just "git push dokku main"

GitLab plus CI is amazing. And, other than hosting, I don't pay a penny for it. If you self-host, you can put it behind a wireguard VPN. It's incredible. You can run as many runners as you want on your personal laptops, or on the cloud, and bring them up and down when needed.

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#234

Post by whitepoplar »

strzibny wrote: post #176
nkzd wrote: post #127

Earlier quoted context omitted.

How do you host your DB? Do you use managed DB service?

If someone wants to know how to host a database for the first time, I have a full chapter on PostgreSQL (and one on Redis) in my new book https://deploymentfromscratch.com/ . There is also a demo that sets up an independent database server (which you can treat as your own managed db). It's easier than you think if you stick to your distributions' packages. Just need to know how to 1, do auth right 2, enable SSL 3, ru…

Any chance you have an HN discount code? :-)

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#235

Post by NotAnOtter »

malwrar wrote: post #13

Not a solo founder, but I do a lot of ops professionally. Unless you have a specific reason, I say it’s best to avoid complicated tooling until later. k8s, ci, etc are all really useful and solve a lot of hard problems, but god are they a bitch to set up and then monitor, fix when something weird happens, secure from nasty people outside, patch when some update becomes necessary, resist the urge to “improve”, etc. Th…

Yeesh

I cannot imagine not setting up at least a basic CI. Even if you remove all unit/integration test alarms, a CI is still very useful. The 20 minutes of build -> upload -> deploy script -> check prod vs a 1/2 day to set up a CI and forget it.

I'm not sure you save much time but you save a lot of headache

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#236

Post by pirate787 »

jon-wood wrote: post #190
bingohbangoh wrote: post #188

Earlier quoted context omitted.

Heroku is by far the best for this but just to throw in some alternatives: * Digital Ocean Apps -- somewhat finicky but works very similar to Heroku, it gets closer to bare metal so I prefer it * render.com * AWS Elastic Beanstalk -- though setting this up is non-trivial, it is very similar to Heroku with its "set it and forget it" * AWS Container Services -- if you're using Docker * Google App Engine * Supabase -- g…

I would advise against Elastic Beanstalk, last time I used it the product had the stink of being on life support - they've not really added any new features to it in several years, and even before that it was incredibly flakey.

I use Elastic Beanstalk daily it is maintained and supports everything on AWS, they've added ALB and WAP for example. It runs on Cloud Formation. Never experienced anything "flakey"

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#237

Post by davidgl »

Fiahil wrote: post #151

Because everyone here is hell bent on "spooky arcane oldhat sysadmin deploy techniques", I'll share my setup using k8s. - CI on github actions - Project management on post-its (short) and READMEs (long term) - deployment is done by a github action trigger on the `main` branch - Hosting is using AKS, GKE, or on-prem k3s on a raspberry pi. Want to restart a service ? just kill the pods. - Devops took about 2 days of wo…

> Note: this is highly effective because I already had significant K8s experience, so, if you're still learning about what Ingress-Controller to choose for your OIDC proxy, then don't go that route.

What would you recommend instead ?

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#238

Post by emidln »

NotAnOtter wrote: post #235
malwrar wrote: post #13

Not a solo founder, but I do a lot of ops professionally. Unless you have a specific reason, I say it’s best to avoid complicated tooling until later. k8s, ci, etc are all really useful and solve a lot of hard problems, but god are they a bitch to set up and then monitor, fix when something weird happens, secure from nasty people outside, patch when some update becomes necessary, resist the urge to “improve”, etc. Th…

Yeesh I cannot imagine not setting up at least a basic CI. Even if you remove all unit/integration test alarms, a CI is still very useful. The 20 minutes of build -> upload -> deploy script -> check prod vs a 1/2 day to set up a CI and forget it. I'm not sure you save much time but you save a lot of headache

I think you mean automation is very useful. I've worked in a lot of places where we had a single `make deploy` that could compile, test, upload the artifacts, deploy the artifacts to hosts, start the new services, and slowly transition the load balancer to point at the new instances while automatically backing out if error metrics started lighting up. This is just automation, it doesn't have to happen on every commit. Especially when working solo or on a small team, I might not want to maintain a lot of VCS branches and instead, just deploy when I feel it's ready. You have to do all of that stuff anyway, so if you later decide to hook it up to a VCS trigger, you're all set.

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#239

Post by sumtechguy »

diiq wrote: post #22

Lots and lots of best practice exists specifically to help teams , and especially teams with some normal amount of turnover. The problems of a solo dev are very different than a dev on a team. Knowledge silos don't exist. Distributed expertise doesn't exist. There's no one to mentor, no shared vision to maintain, no intertia to combat. I consult on big complicated team projects. I also manage multiple solo projects.…

Probably the best tool you have is somewhere to write your notes down. What you did, why you did it, how to reproduce it. That saves you time in a year or two when you go 'hmm I did something like this before have to do it again because some CPU ate itself'. My old notes have saved me a lot of re-work over the years. Because I had written down what I did so I do not have to rebuild it. I can just skip along the notes and be most of the way there.

This bit seems to hold true across any size team for me. If I have notes I can spiffy them up and make cheat sheets out of them and bootstrap other developers faster. Or if it is just me, my feeble brain will forget odd details that I needed for something.

Re: Ask HN: Solo-preneurs, how do you DevOps to save time?

#240

Post by throwaway2016a »

dsgrillo wrote: post #230
throwaway2016a wrote: post #211

Earlier quoted context omitted.

Good question and I have an answer for you. It depends on your risk tolerance and resiliency. If you compare it to a single machine with multiple tenants and no replication, it doesn't make sense. But that comes with risks and given the OPs question in the context of a k8s cluster (which for HA is a minimum of 5 servers), I figured they are prioritizing time savings over money. Losing customer data, downtime, and dat…

You bring up good points, fair enough. I might being naive here, still: - losing customer data: the backup must be done frequently enough. So I'm not risking losing data. - downtime: I'd expect that RDS could still go south as well, maybe less frequently, but still. In my case I start another VM, run again the container and apply the backup. - data breaches: Of course one can misconfigure something here, but not sure…

True. You have downtime risk with RDS and managed databases as well. I have taken down RDS instances with a bad query and indexes once or twice.

$50 total would be tough with fully managed AND high availability. I agree. But to digress from the tech side a bit and talk my own experiences, the question I ask myself is: as a solo-prenuer can I use that time to be doing something that will make me more than $X per month? If the answer is "yes, immediately!" I go with the managed server. If the answer is, "it would take me years to break even"... I may roll my own.

To address your points, thought: I think managed databases do help here. Data breaches and outages are usually configuration issues. Since they start with a good known configuration, managed services do help. Also, I see not being able to SSH in as a positive in this scenario.

For example: Say you misconfigured a database an break backups (happens all the time even to good DBs). That is much harder to do on a managed database.

If you're managing just one machine and being down for 10-30 minutes while your backup is being restored is fine, you probably don't gain much by using a managed service. In that case I'd take frequent backups, upload them immediately off the server (probably S3), and containerize the database. In my architectures I always run at least 3 servers.

With most database engines, if you have three servers you need to lose 2 to cause an outage. But a 3 server cluster is much harder to manage manually, tilting the scales more towards a managed database.

Also, to your point, would blow $50 out of the water.

If I can, I use a managed serverless database like DynamoDB (there is also others) that are pay as you go. You can easily run a service on just DynamoDB free tier if your use case is well suited to that kind of key/value stored based DB.

You can also, always start with your own and switch to managed later as you add machines to the cluster.

Just, whichever you choose... but "especially" when rolling your own... make sure you run fire drills. I.e test your backup and restore process regularly.

Post reply on HN