Live data from Hacker News

I returned to AWS and was reminded why I left

fourlightyears.blogspot.com

471–480 of 684 posts

Re: I returned to AWS and was reminded why I left

#471
Can report that exiting from Lambda to something more sane (like, say, a django task or api endpoint or something) is now pronounced "hey copilot, look in that directory and implement precisely the same functionality over here". Or thereabouts. A whole lot of things suddenly look a lot less locked in.

Re: I returned to AWS and was reminded why I left

#472
post #439

Earlier quoted context omitted.

Doesn't Amazon engineering culture have a very engineer-led product culture? Meaning, devs are often responsible for the UX and flow. I remember many years ago we hired a junior developer who just finished his internship at AWS and he showed me the dashboard he shipped all by himself in the summer with no product or designer help. It looked horrible. Some devs have a good product/UX sense but the vast majority are ho…

> Some devs have a good product/UX sense but the vast majority are horrendously bad at UX. I think the problem is that nobody understands the size of the problem. For most tasks, the accomplishment is getting something to work. That takes 90% of the time. But the UI requires polish, working things out, backing out and trying again, and takes the OTHER 90% of the time. I remember talking to a friend who worked with ap…

Prior to AWS at Amazon hosts were provisioned as “host classes” and typically operated on in that way. We were encouraged to make them “touchless”, which meant the infrastructure team could replace that host without contacting the team first. The deployment tool deployed to host classes (though you could put an individual server there if you wanted). EC2 wasn’t quite the same, but not very foreign either. We didn’t originally even use the AWS interface (at the team level). They were managed by a team working on the transition.

Re: I returned to AWS and was reminded why I left

#473

These arguments against AWS are boring. 99% of the negative comments are along the line of "so i have a dead simple product, I dont know anything about AWS, I logged in and it was super complicated and it seemed pricey". Well guess what, if you have a CRUD website and 100 users you're just not the target. Move on. Some days ago I wanted to sketch a 3D model of my TV remote. I opened blender and what a mess of complic…

I agree, this is a common story and your point stands for some significant percentage of the complaints. It should be made clear though, that some of us helped spend many millions in obviously wasteful on-prem infra in the nineties, bought into AWS wholeheartedly when it came out, fought through the ignorance, developed the ability to deliver highly scaled applications on the platform over many years and at least som…

This is borne out by the fact that there are alternatives that are:

- dramatically simpler

- cheaper

- easier to budget

while retaining the scale-on-demand and hide-the-actual-hardware properties that the industry jumped for joy at. What they don't have is the nobody-got-fired-for-rearchitecting-to-aws bit.

Re: I returned to AWS and was reminded why I left

#474

Earlier quoted context omitted.

I agree with you at some degree, but I would like to point out that AWS pricing is much more complicated that you can calculate how much will you pay from a static number showing up on the UI. If it bothers you that you need to open two tabs for cross-checking the costs, you may want to avoid every cloud provider, not just AWS. Once you have NAT gateways, CloudFront, S3, auto scaling, loadbalancers, etc, calculating…

I don't think your comment hits like you think it does. I think your intent with "cheap" implies some level of being lesser. In my experience that is not the reality. Similar to opp I migrated a startup from 5x cost in AWS to DO years ago. In fact that "cheap" competitor was able to give them better performance, more reliability and more features for a lot less. AWS is almost never required and almost never the best…

> Finally, AWS isn't a silver bullet. For anyone in us-east-1, you know [0].

I probably should have commented on the original article here, but I pulled all of my company's production infra out of that AZ back in 2019 because AWS dragged its feet for too long deploying 5th gen hardware there.

I assumed the racks were full or something. I still don't know if they ever did get newer hardware in that AZ—I just avoid it like the plague.

I had a light chuckle this week when I discovered the work I did out of sheer frustration saved us from a partial outage seven years later.

Re: I returned to AWS and was reminded why I left

#475

Earlier quoted context omitted.

> that's because you have a service principle in your IAM trust relationship that allowed us access That’s why it’s so complicated!!! I don’t understand how I should evaluate trust for your internal EBS org versus your internal ALB org. I kinda just expect it to be all “AWS” trust. And it’s all garbage anyway. There’s no way I can prevent the hypothetically untrustworthy EBS team from surreptitiously adding charges t…

3 things to untangle here. 1. It's about trust and auditability, while you may not want or need it, there are a lot of customer that are either interested or legally obligated to know who have accessed certain data. 2. It's about dogfooding - how would you trust an identity and access system when the company does not even use it internally? 3. In general, there are quick buttons and template to do it if you don't wan…

1. It's about imposing worst-case complexity on the 99% of people who will never benefit. 2. Some of that complexity only arises because of the dogfooding 3. No it doesn't get easier, because you still need to understand what those things actually do to know if they're right for your use case, and besides if you're driving everything from terraform then having a "quick button" is precisely useless.

We had an AWS rep try to sell us on an AI tool to help with predicting the IAM permissions that our infrastructure code needs. My response was, essentially, "why have you built a deterministic system so complicated that it needs an AI to configure correctly?" I have not had an answer.

Re: I returned to AWS and was reminded why I left

#476

Imo lambdas are super cool, and the best way to have a no-headache fast-iteration time deployment service. What most people realize, that you don't have to go microservice or fragment your code to a billion little repos, you could take a standard webserver, and move it to lambda, as long as you don't expect requests to be able to share on-server state.

What is lambda adding in this situation that throwing a docker image up somewhere isn't?

Well, complexity and iteration time for example. For docker, you need something that runs it (like your own cluster of EC2 or ECS), a private registry you push to, a separate user that provisions the actual server, while iteration (code changes, or something as simple as changing an env var) - involves the cycle of upload new image, shut down the old containers, try to start up the new ones, with all sort of weird failure cases, like if your container depends on a dockerhub image like alpine, you can run into a ratelimit scenario with dockerhub,as AWS is too cheap to pay for dockerhub access, and doesnt have their own mirror, so your containers may fail to start unless you explicitly mirror your base image.

Then you have to take care to update your images etc.

All this stuff, like ECS also lives in a subnet, so you have to manage routing, and public accessibility and stuff - its legit crazy amout of work compared to either lambda or just running stuff on a virtual machine.

Imo it's the worst of all worlds.

Re: I returned to AWS and was reminded why I left

#477

Earlier quoted context omitted.

A lot of these projects work on a business model where they open-source their core product, and provide advanced services, installation, maintenance or fully-managed services around their product. AWS was bypassing them by providing fully-managed services. On this, I am on the side of the people behind the projects. Basically AWS was eating their lunch. They had no choice but to change the licenses.

They have a problem with their business model, then. License changes to a formerly open source project are costly. The community reacts very strongly when license terms change after they've come to depend on a product, and they should. Why do we apply this standard to MongoDB but not to Apache, Linux, Postgres, or MariaDB? One purpose of an open source license is to allow many providers to provide the service. As I'v…

"They have a problem with their business model, then"

Ok, then don't be surprised when the most popular license becomes the FairSource license. Under this license, you have no rights, no ability to fork and no ability to modify, no ability to legally change the software in any way, but hey...you can see the source right. I feel like you don't understand the tragedy of the commons somehow.

Re: I returned to AWS and was reminded why I left

#478

Earlier quoted context omitted.

A core part of an engineers job is including thinking about cost in what they do.

Right - nobody who’s had a formal education in engineering would think that way, because cost considerations are part of the curriculum from the start.

I don't think a lot of formal education places teach AWS's resource pricing structure, which can be incredibly confusing, but can be boiled down to: if you want to be as cheap as possible, just use EC2 for everything and maybe S3 for storage.

Re: I returned to AWS and was reminded why I left

#479
post #323

Earlier quoted context omitted.

IAM is unnecessarily bad. I recently had to set a trivial policy, and was doing it correctly. The console kept warning me that I was giving root AWS access to my external application because they want people to use the locked in AWS path, and I was running off cloud. On top of that, they break copy paste on the web console, so you can’t just ctrl-c ctrl-v and then ask Claude to explain their WTF-ery. Instead, you hav…

You think that you're complaining about IAM, but really you're complaining about the web ui. I rarely use the web console, I use terraform or the cli. I'd you're vibe coding your infra with Claude, point it at the cli / terraform. Skip the ui.

With terraform you get the amazing experience of having to iterate, one at a time, through the five hundred and thirty seven new permissions you need to grant having decided that a lambda configuration needs to be ever so slightly different than it was yesterday, because there's no documentation linking terraform creation of resources and the IAM permissions required to successfully make the AWS API call behind the scenes. Or those for updating a resource, which are different, so you get to do it all again tomorrow. Or deleting - different again. Fun for the day after.
Post reply on HN