Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 96 posts

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#81

Post by rjst01 »

mdaniel wrote: post #56
rjst01 wrote: post #14

A year ago I left my job at a 500-employee SaaS business working on the team that maintains the devops infrastructure, to found a startup. For me the biggest pain point is going from nothing to a sufficiently flexible devops setup. There are a lot of great tools out there, but making them play well together is an exercise for the reader. There are also a lot of preference-based choices you need to make in how you wan…

> but there's some amount of extra stuff deployed (ingress controller, for example) that is documented in a text file Out of curiosity, why just the "readmeware" for those components? I can't think of a single thing that requires clickops in a modern k8s setup, so much so that in the beginning we used to bring up the full stack from nothing based on a single CFN template - roles, load balancer, auto-scaling group, co…

> Out of curiosity, why just the "readmeware" for those components?

Because there are so few of them. Our setup has an ingress controller and a certificate manager, and then some bookkeeping like copying the container registry credentials into every namespace

> I can't think of a single thing that requires clickops in a modern k8s setup

Absolutely agree.

> The whole process took about 8 minutes from go

How long to do the development and testing of the template, and what size is your team?

Don't get me wrong, I'm not happy about this situation. As well as the DR concern you raise, we can't quickly spin up short lived clones of our infra for testing complex changes, so we test them in our staging environment and have to block prod deploys until we're either happy with the change or decide to roll it back. At a larger org this would be a major headache but at our current size it does not matter.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#82

Post by tj1516 »

rjst01 wrote: post #14

A year ago I left my job at a 500-employee SaaS business working on the team that maintains the devops infrastructure, to found a startup. For me the biggest pain point is going from nothing to a sufficiently flexible devops setup. There are a lot of great tools out there, but making them play well together is an exercise for the reader. There are also a lot of preference-based choices you need to make in how you wan…

> We don't need to deploy environments super often, so just do it manually and update documentation in the process if any variations are needed.

So you don't need to deploy multiple times or you don't do it because the system is stable when you deploy less often? I mean is it by choice or because of some tool or expertise limitation?

> Also, for architecture stored in text files - does that cause any problems for you?

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#85

Post by Cowsheek »

cebert wrote: post #22
tj1516 wrote: post #6

Earlier quoted context omitted.

Understood. But what if you have a multi-cloud setup or you want to migrate to a different cloud to efficiently use credits?

I personally think multi-cloud is over hyped and the complexity isn’t worth it for most organizations. The apps I am responsible for are one AWS region / 3 AZ and we have sufficient uptime. We could be multi-region but the added cost wasn’t worth the benefit and complexity. Multi-cloud makes zero sense.

[dead]

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#86

Post by rjst01 »

tj1516 wrote: post #82
rjst01 wrote: post #14

A year ago I left my job at a 500-employee SaaS business working on the team that maintains the devops infrastructure, to found a startup. For me the biggest pain point is going from nothing to a sufficiently flexible devops setup. There are a lot of great tools out there, but making them play well together is an exercise for the reader. There are also a lot of preference-based choices you need to make in how you wan…

> We don't need to deploy environments super often, so just do it manually and update documentation in the process if any variations are needed. So you don't need to deploy multiple times or you don't do it because the system is stable when you deploy less often? I mean is it by choice or because of some tool or expertise limitation? > Also, for architecture stored in text files - does that cause any problems for you…

We deploy our services on every merge (multiple times a day) from CI assuming the tests pass. But we don’t re-create our Kubernetes infra when we do that. I set up 3 Kubernetes clusters (staging, prod, internal apps) when I went full-time and have barely touched them other than to apply updates.

> does that cause any problems for you?

Nope. Our Kubernetes setup is just about as simple as it is possible to have a Kubernetes setup. I entertained the idea of going with something else, because we definitely don’t need all Kubernetes has to offer. But I settled on it because it’s what I know best and the overhead+risk of something new would have exceeded the cost of the unnecessary for us baggage that Kubernetes brings.

If our requirements were different or if I was making regular changes, we would be in a very different spot. But as it stands today it is just not a priority.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#87

Post by icedchai »

sph wrote: post #69
sofixa wrote: post #65

Earlier quoted context omitted.

> That these days there are only two serious choices: deploy on a single-node Docker/podman machine, or Kubernetes. I disagree - there's cloud PaaSes like Fargate or Cloud Run, or when self-managing, a third option which is much less known but quite easier while also being more flexible - HashiCorp Nomad. Disclaimer time: I work at HashiCorp, but I've had this opinion for years before joining - https://atodorov.me/20…

This is silly but I will not use any Hashicorp product ever again, for two reasons: one is Terraform (it is a bad product with a bad DSL), the second is that I bought one service from them that advertised a clear 30-day money back guarantee, found a bug in one hour of running, a multiple-year old Github issue they had been ignoring, and to make it worse, they completely ghosted me when I went to ask for a refund. Bot…

Terraform may seem bad, but cloud-specific solutions like Cloud Formation are even worse...

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#88

Post by icedchai »

dig1 wrote: post #42

My main challenge is people, especially "experts," who come, preach whatever popular cloud stuff is today, and then leave. This leaves me with a lot of shi* to deal with. We should keep things simple, KISS. My few key points for the future myself: 1. Be cloud agnostic. Everyone operates on margins and a slight increase in a cloud provider's fees could be a death sentence for your business. Remember, cloud providers a…

The "experts" will happily lead you down the path of cloud lock-in, building solutions around proprietary tech like DynamoDB and Lambda. You might even wind up in a state where you can't develop locally. It's totally non-productive and I've witnessed it first hand.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#89

Post by oldandboring »

efxhoy wrote: post #46

I work at a 60 dev shop so not a startup per se. Our biggest problem is feature environments, or actual integration tests where multiple services have to change. Because infra is in its own repo in terraform and the apps have their own repo we don’t have a good way of creating infra-identical environments for testing code changes that affect multiple services. We always end up with some hack and manual tweaks in stag…

> Our biggest problem is feature environments

I'm always baffled to see how many shops claim to either not have this problem, or sidestep it. Every project I've ever worked on has had multiple enhancements/fixes in flight at the same time, they need to be tested and deployed independently of each other, on their own timelines. For this we need story branches and a fast way of deploying different story branches to different test environments. If you're merging everything into kitchen-sink "dev", "staging", "test" branches because someone drank the Gitflow kool-aid, your confidence goes way down that a specific story branch is "ready to go" and that production will behave exactly like dev and test. And, as you mention, accomplishing the story-branch approach across multiple repos (assuming the change is large enough to affect multiple repos) sounds like swimming with crocodiles.

Re: Ask HN: Startup Devs -What's your biggest pain while managing cloud deployments?

#90

Post by pravanjanc »

tj1516 wrote: post #6
matt-p wrote: post #3

I think without aws it would be painful to be honest. Use code pipeline/cloud build then either container registry and ecs or beanstalk. You get everything you mentioned for free with either set up. Seriously it takes moments to write terraform or even do it by hand.

Understood. But what if you have a multi-cloud setup or you want to migrate to a different cloud to efficiently use credits?

> Understood. But what if you have a multi-cloud setup or you want to migrate to a different cloud to efficiently use credits?

So yeah, one of the clients we were working with had to move from AWS to GCP (mostly for utilizing cloud credits). The setup was small but the estimated time was nearly 2-3 months. It's really a pain, and I think a lot of organizations don't think of doing it because when they weigh rewards and the effort/time needed, it doesn't make sense.

Post reply on HN