Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 96 posts

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#71

Post by KaiserPro »

bjackman wrote: post #45
KaiserPro wrote: post #27

There are a lot of ways of doing things, and its really easy to make mistakes. my advice would be: Separate your build from your infra. Whilst its nice to have your cloud be spun up with CI, its really not a great use case, and means your CI has loads of power that can be abused. Gitlab with local runners is a good place to start for CI. its relatively simple and your personal runners can be shared between projects (…

> You are unlikely to change cloud providers, so choose one and stick to it. Use their managed features. I am curious about this because I see opposing views expressed by different people. I have never personally been in a position where the decision has been relevant. I work at a cloud provider an I'm told that a big slice of our revenue comes from customers who are already load-balancing across multiple clouds, so…

> I am curious about this because I see opposing views expressed by different people.

My sample size is me and my immediate friends. I suspect that if you have a resilient multi-cloud deployment, then you'll use it to hunt for a metric you want to hit (speed/price/latency)

However, the engineering cost to get there is pretty high, and almost negates the point of having the cloud (unless you have a scaling requirement where you need 10x at short notice.)

I worked at a large news company, and it was decided that it was cheaper to just pay for the hosted services than pay for the people to run them (think RDS vs home grown DB) RDS is what 2x the cost of a normal instance, but thats still cheaper than the three engineers + oncall to manage a custom deployment and manage the backups and migrations. (along with conway's law of having DBAs)

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#72

Post by sofixa »

sph wrote: post #69
sofixa wrote: post #65

Earlier quoted context omitted.

> That these days there are only two serious choices: deploy on a single-node Docker/podman machine, or Kubernetes. I disagree - there's cloud PaaSes like Fargate or Cloud Run, or when self-managing, a third option which is much less known but quite easier while also being more flexible - HashiCorp Nomad. Disclaimer time: I work at HashiCorp, but I've had this opinion for years before joining - https://atodorov.me/20…

This is silly but I will not use any Hashicorp product ever again, for two reasons: one is Terraform (it is a bad product with a bad DSL), the second is that I bought one service from them that advertised a clear 30-day money back guarantee, found a bug in one hour of running, a multiple-year old Github issue they had been ignoring, and to make it worse, they completely ghosted me when I went to ask for a refund. Bot…

Nah, it's not silly - you had bad experiences with a company, it's normal not to want to interact with them further. I'm personally not a fan of Microsoft because I spent my formative tech years installing Windows on family friends' PCs and had so much inconsistencies and issues that I have a very negative association with Windows and Microsoft.

I disagree with the DSL part, but it's in big part subjective so people have all sorts of different experiences and opinions.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#73

Post by verelo »

djboz wrote: post #40
verelo wrote: post #32

Earlier quoted context omitted.

I inherited a project on fly.io but before that I’ve used ECS as my primary platform. While ECS is kubernetes lite, fly is ECS but a little more dev friendly in the realm of not dealing with the complexities of IAM etc. It has other downfalls, but it’s the sweet spot for my early stage products right low…although i do expect I’ll end up on ECS if this is going well a year from now.

If the rest of your infra is on AWS(e.g., S3/RDS/Dynamo), why not just start with ECS? Egress fee's would seem to take a toll after a bit using Fly.io

Mostly just because it’s there already, and after getting familiar with it i really like how fast it is to get a project started. Ecs is great, but fly is 10x easier to deploy to.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#74

Post by imglorp »

bionhoward wrote: post #13

I’d argue the obvious answer is address the lack of great answers for declarative schema migration in PostgreSQL. There is Skeema https://github.com/skeema/skeema but it doesn’t support Postgres and Prisma iirc forces you into an ORM, atlas looks perfect but has a nonstandard license.

We are using Hasura for schema migration. You don't have to use the graphql stuff: it will just follow your tables and generate .up and .down migrators for you.

Then you can just use pg like normal, directly.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#75

Post by Ologn »

I would say be cognizant that a major cloud provider might always just contact you out of the blue and say they after a few years of usage they blew away your VPS, and have no recovery for it, and you should rebuild it from scratch off your last backup.

I already was using more than one service but I cancelled Rackspace when this happened, although was not making enough yet to be fully and automatically redundant. It was a pain to suddenly have to drop everything and rebuild the service, as it broke the entire service I was offering. Actually I rebuilt everything on my existing service at Linode (now Akamai), and then got a VPS at Ramnode as my backup. So it was a lot of work suddenly thrown at me that I didn't need. Luckily, while my backup practices are not completely ideal, I do follow the 3-2-1 backup rule enough that it wasn't catastrophic.

Here's the message I got from Rackspace in 2019:

This message is a follow-up to our previous notifications regarding cloud server, "c0defeed". Your cloud server could not be recovered due to the failure of its host.

Please reference this ID if you need to contact support: "CSHD-5a271828"

You have the option to rebuild your server from your most recent server image or from a stock image. Once you have verified that your rebuilt server is online, you must (1) Copy your data from your most recent backup or attach your data device if you use Cloud Block Storage and (2) Adjust your DNS to reflect the new server’s IP address for any affected domains. When you have verified that your rebuilt server is online, with your data intact, you will need to delete the impacted server from your account to prevent any further billing associated with the impacted device.

We apologize for any inconvenience this may have caused you.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#76

Post by icedchai »

infecto wrote: post #17
cjk2 wrote: post #11

Killer one that has shafted us is tying everything into popular deployment and CI tooling. Your product should be buildable, deployable and tested on a standalone workstation at any time. We have lost literally days fucking around with broken products which get their tentacles into that shit. I'd use AWS + ECR + ECS personally. Stay away from kubernetes until you have a fairly large deployment as it requires huge adm…

ECS is the way imo. I dislike the complexity in auth around cloud services but ECS is a really nice way to build scalable systems cheaply. I am a big fan of using the underlying Fargate spot machines within ECS.

On GCP, Cloud Run is also a nice solution with minimal complexity. It's very similar to Fargate / ECS.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#77

Post by pimsn3000 »

sph wrote: post #19
supriyo-biswas wrote: post #18

Earlier quoted context omitted.

There’s also Docker swarm for containerized workloads. Personally I like VMs, and I’ve been toying around with the idea of having a LXD cluster on dedicated servers where each LXD container hosts some of my workloads.

I set up Docker Swarm at my previous company, but it was a dead and stale project 5 years ago when I last used it. I honestly cannot recommend it in 2024.

I just set up my first docker swarm cluster. It's not in production yet, just a stage environment, but it is working very well so far, and I like it very much. From my experience so far I can very much recommend it, and I hope it will get more attention again. Because it does fill the gap which is described in the original post.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#78

Post by margorczynski »

evanelias wrote: post #57
margorczynski wrote: post #37

Earlier quoted context omitted.

I've used Flyway and it works great, can also recommend Alembic from the Python world - https://alembic.sqlalchemy.org/en/latest/

Flyway and Alembic are imperative, not declarative. Imperative tools express schema changes as a filesystem of incremental "migrations", whereas declarative tools use a filesystem of CREATE statements and can generate DDL diffs to transition between states. The declarative flow has a lot of benefits [1]. It is much closer to how source code is typically managed/reviewed/deployed, which is especially important for sto…

Yeah, I was talking more in general as to what parent said suggesting good, battle-tested migration tools like Flyway. Declarative/imperative wasn't the focus.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#79

Post by matt-p »

tj1516 wrote: post #6
matt-p wrote: post #3

I think without aws it would be painful to be honest. Use code pipeline/cloud build then either container registry and ecs or beanstalk. You get everything you mentioned for free with either set up. Seriously it takes moments to write terraform or even do it by hand.

Understood. But what if you have a multi-cloud setup or you want to migrate to a different cloud to efficiently use credits?

99% of the time it's just a docker container that connects to a database, an object store and redis.

If we need to move it then we move it... not exactly difficult or time consuming. There are always ways of half moving things too, for example we could move the container deployment to another cloud but keep our build process and object store at the original vendor and so on.

In reality we are not going to be shifting production load between cloud vendors every 2 weeks.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#80

Post by niteshade »

Hortinstein wrote: post #30
sph wrote: post #19

Earlier quoted context omitted.

I set up Docker Swarm at my previous company, but it was a dead and stale project 5 years ago when I last used it. I honestly cannot recommend it in 2024.

I don't think there has been a ton of iteration on it, but did you run into any specific problems or bugs or is this lack of recommendation based off caution against adopting something that is not being iterated on? Just asking because while I haven't used it in years, it's been my go-to for small projects in the past, it seemed to do what it advertised very well. I hope that someone picks up the swarm torch, I reall…

Not GP, but I recently attempted to migrate a single-node Docker Compose setup to Docker Swarm and ran into the following issues:

- No ability to use secrets as environment variables, and no plans to change this

- Cannot use `network_mode` to specify a service to use for network connections a la Docker Compose

There were a few other minor issues which resulted in ditching Docker Swarm completely and moving to a Nomad + Consul stack instead.

Post reply on HN