41–50 of 116 posts
None of this is simple. We're drowning in YAML files. It's difficult to maintain. Jinja + ansible + helm... it's a joke. There must be a better way of doing all this.
Helm uses Go templates which are awesome. It is what Hugo is built off. The main issue is that you still have to manage indentation with Helm. YAML is easier to read than JSON and TOML. If you don't like it, then what do you suggest is better?
None of this is simple. We're drowning in YAML files. It's difficult to maintain. Jinja + ansible + helm... it's a joke. There must be a better way of doing all this.
I'm working on a command-line deployment tool that deploys to DigitalOcean and AWS LightSail (to start with). This is based on my experience deploying apps. I expect to finish the remaining work in the next few weeks. Can I contact you to try it out? (My email is on my profile)
Earlier quoted context omitted.
I mostly agree with you but we definitely reduced reliance on AWS specific functionality after starting to offer on-premise software installations. Most of them don’t gain you THAT much and then the software really isn’t very portable.
I question this logic - there seem to be fundamental differences in even the basic services between various cloud providers. For example you can resize a compute-attached disk size live in GCP while you can't in AWS. Many intricacies like this are annoying, but at the same time advantageous if you know and use them. If you are primarily trying to just offer another layer on top of these cloud services, (like snowflak…
Earlier quoted context omitted.
Could you comment on why you are moving away from elastic beanstalk? We manage a fairly simple EB deployment but it seems perfect for our infra needs, and I'm not able to imagine why we might need to scale out of it - if any other services are required, I'm tempted to only launch multiple EB deployments instead of going the kubernetes/fargate route.
Largely I don't think it's worth moving off EB except in specific scenarios. I'm a Rails monolith and DB migrations are not well suited in ElasticBeanstalk when running docker containers. There's no way to run a single container that 'completes' and then returns whether it was successful or not. Currently I have a separate environment that holds a single EC2 (lots of idle time) to run migrations. It fires and the dep…
1) having our app containers include the DB migration logic 2) on container startup, “check and run migrations” before app startup, 3) the trick: acquire a lock in postgres as part of step 2, so that only one node at a time can run migrations.
Migrations are run inside of a begin/commit, so with the lock we have reliable guarantees that 1) exactly one container at a time can try to run the migration; 2) the migration either completes or fails.
This setup has the benefit too that, if the migration fails for whatever reason, no new app containers will start in production. That is: we can basically trigger a production deploy and if it fails, the deploy halts and the previous app version remains up and serving traffic.
There are better ways of handling this, but for us at our scale, this has been both very simple and very reliable, which makes me happy. :)
I created [Qovery](https://www.qovery.com) to address this problem of simplifying the Cloud. Allow streamlining the use of the Cloud for any developer. Today traditional players stack layers to simplify what they have created but without going out of their way of seeing things.
Qovery makes application deployment very pragmatic for developers because we put ourselves in their shoes. How does a developer think? He simply thinks about his code and wants to focus on his mission, which is to address business issues and not all the plumbing around it.
None of this is simple. We're drowning in YAML files. It's difficult to maintain. Jinja + ansible + helm... it's a joke. There must be a better way of doing all this.
None of this is simple. We're drowning in YAML files. It's difficult to maintain. Jinja + ansible + helm... it's a joke. There must be a better way of doing all this.
All of this is endlessly pushed by AWS, Google, Docker, and anyone else with a foot in the "Snag as much cash from DEVs" crowd.
Other old timers will explain how they ran thousands of hits/second on 6 or 7 bare metal servers, 15 years ago, without CDNs, using LAMP, ajax/js frontends, and more. The key was code optimization, SQL queries without inane and poorly written MVC SQL logic, and the list goes on.
I am relentlessly gobsmacked at how people are spending quite literally 100 to 1000 times the cash to host on AWS, using microservices. And often amused how new devs just can't get it through their head, that yes, this is 100% factually true.
Docker replaced something that was already done ; identical PROD and DEV environments. All DEVs I had working with me, were issued VMs with 100% replicated PROD. VMs auto-build using debootstrap + SVN checkout.
Rollbacks in prod? Handled by SVN rollbacks.
When I look at the insane complexity being displayed by containers on top of VMs on top of baremetal, the MASSIVE loss in performance (yup, it's there.. especially for IO)... I just don't get it.
Many DEVs have all been sold a pack of complete and total lies.
I get called in again, and again by clients to reduce cost, optimize resource usage. I literally bring 1000x performance boosts to the SQL layer with minimal improvements.
Anyhow. Yes, this was a rant. Sorry it's a reply to you specifically.
I am in the process of migrating my stack from Elastic Beanstalk multi-container to Fargate so this looked like an interesting thing I could 'pick up'. This does potentially unify the container definitions between compose and ContainerDefinitions in the task definitions, but for my self, that's not a super helpful tool. Much of the complexity of running Fargate is outside of Fargate, wiring everything up so Route 53…
Could you comment on why you are moving away from elastic beanstalk? We manage a fairly simple EB deployment but it seems perfect for our infra needs, and I'm not able to imagine why we might need to scale out of it - if any other services are required, I'm tempted to only launch multiple EB deployments instead of going the kubernetes/fargate route.
Then there's issues with when it goes wrong. Troubleshooting is really hard, and knowing what actually went wrong requires a ton of digging around, and god forbid the environment gets stuck in a warning state. You can only download logs when it's "OK", so you're pretty much SOL if this happens. There's also the issue where you can't be sure what's running anymore. If a rollout didn't finish, what actually got updated? There's just no good way to find out, which is absolutely terrible for a deployment system.
All in all, the drawbacks + lack of options/control make EB useless for a lot of companies. Personally, I find it best to avoid EB and just go with tried and tested methods without it. That doesn't mean rolling out K8S or using fargate either. You can if you want, but there are a whole lot of other options.
Earlier quoted context omitted.
Largely I don't think it's worth moving off EB except in specific scenarios. I'm a Rails monolith and DB migrations are not well suited in ElasticBeanstalk when running docker containers. There's no way to run a single container that 'completes' and then returns whether it was successful or not. Currently I have a separate environment that holds a single EC2 (lots of idle time) to run migrations. It fires and the dep…
I might not know what I'm talking about (our deployments don't do automatic migrations, for eg), but if I understand correctly, your problem is to synchronize migrations with deployments, is that accurate? Couldn't you use something like GitHub Actions to run migrations and then deploy to EB?
Github Actions is CI/CD and it can be used to kick off any deploy step and manage it, but it's not one to actually perform the migration. That would be like Github Actions actually running the webserver directly.
In this example Github Actions needs to kick off a `eb deploy`, which it can totally do (I personally use CircleCI) but `eb deploy` (elastic beanstalk in general) is not designed to run a short lived script on a single container AND wait for it to return.
You can run a short lived script no problem, but you have no idea if it was successful or not because once the container becomes 'healthy' the command completes.
What I'm really waiting for is the container.lastStatus:'STOPPED' and exitCode: 0. Can't do that on EB.