Live data from Hacker News

I returned to AWS and was reminded why I left

fourlightyears.blogspot.com

241–250 of 684 posts

Re: I returned to AWS and was reminded why I left

#241
post #49

Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months. They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship. The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs. I had to have the two…

Doesn't Amazon engineering culture have a very engineer-led product culture? Meaning, devs are often responsible for the UX and flow. I remember many years ago we hired a junior developer who just finished his internship at AWS and he showed me the dashboard he shipped all by himself in the summer with no product or designer help. It looked horrible. Some devs have a good product/UX sense but the vast majority are ho…

I get why it's like this.

Some background. I work at an Amazon sub. This is a good UI for the way we work. We don't spin up a single machine pretty much ever unless it's a cloud dev machine, at which point the price is listed at startup on a custom internal UI. They should consider putting that UI in the ec2 console.

When I spin up machines I pick an instance class by looking through specs and the price chart and set it via AI into a cdk construct. Usually pick a relatively normal machine type digging through all the ilvarious enterprise discounts (which are not reflectedin the prices in the console). Then as I roll out or when I get resource limit alarms on the fleet I adjust the instance types. Or when accounting asks me about price. In those cases I usually look if it's worth it to optimize.

The enterprise discounts are a big consideration. Every year new hires make bad decisions because they don't know about the discounts. They wildly affect total cost. Some things are more expensive (lambda first few years), and others are very cheap so we dog food. The console price in no way reflects reality.

In 15 years we've had about 1k services stood up, around 700 are active. 2000 or total counting tutorials and tests. That means out of an eng org of 500, we've made those decisions maybe 10k times total.

That's how Amazon thinks about it as well. So yeah I agree that the UI isn't meant to be like one where your spinning up a host. I haven't spun up a single host in like 5 years, but I've made many clusters.

But that doesn't mean it shouldn't be better to work for a wider audience. Customer obsession and all

Re: I returned to AWS and was reminded why I left

#242
I was such a fan of it that I ended up working there for 4 years. Now I avoid it and encourage others to do the same.

AWS used to have a nifty tool called "policy analyzer" or something that monitored for permissions used by a role so you could scope it down. The other day I had the need for it and when I went to use it, found out they charge something like $9/resource. So I would pay $45/month for metadata monitoring on just 5 things? Nuts. If they knew how to build truly delightful products, they would make something like a role that starts with broad permissions and automatically scopes itself down after some point. And it would be free or at least really cheap.

DDB is hardly a database. The only reason I can think of to use it is for massive amounts of data whose schema and query patterns are guaranteed to almost never change, which is very rare. Need to sort data on a field? Then you have to create a 'secondary index', which is a copy of the table that they charge you for and that is not strongly consistent. Schema change? Good luck with that. And don't you dare ask to use a nice ORM library. But hey it's serverless.

Here's a good one: you stop an EC2 instance and its volume keeps running and you pay for that. If you detach the volume, you still pay. There is no way to 'archive' an instance. And the only way I found out about that was I got hit with a big bill for those volumes with the charge labeled 'EC2 - Other' lol. Not very 'customer obsessed' to me.

My gripes are clearly not important to them because this is old stuff. So all I can do is go somewhere else, which is fine with me

Re: I returned to AWS and was reminded why I left

#244
post #214

Earlier quoted context omitted.

You’re not wrong but if people didn’t, all our companies would be using Oracle and Microsoft SQL Server and paying Larry Ellison instead today.

PostgreSQL is doing fine even with AWS having a multitude of hosted offerings. Maybe the business model / community-governance model does matter after all...

PostgreSQL doesn't have a pro/officially supported version though.

Re: I returned to AWS and was reminded why I left

#246
post #49

Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months. They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship. The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs. I had to have the two…

Doesn't Amazon engineering culture have a very engineer-led product culture? Meaning, devs are often responsible for the UX and flow. I remember many years ago we hired a junior developer who just finished his internship at AWS and he showed me the dashboard he shipped all by himself in the summer with no product or designer help. It looked horrible. Some devs have a good product/UX sense but the vast majority are ho…

> My point is that maybe it was intentional, but just bad UX culture.

This may be valid, but even if it is someone (or a group of people) at Amazon are violating one of their core leadership principles - Customer Obsession

https://www.amazon.jobs/content/en/our-workplace/leadership-...

A useful (and hopefully delightful) UX is key to showing customer obsession.

That being said, I personally feel the UX at Amazon sucks overall, not just for pricing/packaging but even getting basic shit done. So perhaps Amazon (or at least AWS) doesn't think a good UX is a key ingredient to demonstrating Customer Obsession.

Re: I returned to AWS and was reminded why I left

#247
post #85

Earlier quoted context omitted.

t4g.nano is $3/m; a similar spec-ed fargate on ecs (just any docker container) is $10/m

This sentence beautifully encapsulates my point. I know that this is just ordinary jargon, but wow that's a lot all at once. And it does seem like something I need to know before I start.

[deleted]

Re: I returned to AWS and was reminded why I left

#248
I always smile at posts like this. They're right and wrong at the same time. Systems should be "as simple as possible, but no simpler". And thinking that you can gloss over the detail is just going to create more hassle later on.

IAM is just complex. I can't think of any implementation of "users, groups, roles, policies, identity providers, oidc" that is truly simple.

I'm reminded of a guy I worked with, who fought against Kubernetes adoption because it was "too complex", only to slowly reinvent Kubernetes badly, adhoc, out of vault, consul, systemd, nomad, iscsi, ansible, jenkins, puppet, bash, spit, glue... making lots of mistakes along the way. You think you don't need to implement some feature until you do.

Another thing I'll say about AWS (having been the sole infra guy at a few startups) is that it's well within most people's abilities to learn it. And you can usually avoid the shitty stuff. You think lambdas stuck? Don't use them! You could use EKS, ECS or bare EC2.

Re: I returned to AWS and was reminded why I left

#249

I always smile at posts like this. They're right and wrong at the same time. Systems should be "as simple as possible, but no simpler". And thinking that you can gloss over the detail is just going to create more hassle later on. IAM is just complex. I can't think of any implementation of "users, groups, roles, policies, identity providers, oidc" that is truly simple. I'm reminded of a guy I worked with, who fought a…

Another point is that while it can be more expensive than self hosting, the savings are dwarfed by the engineering costs. A decent infrastructure engineer working for 2 man months on your “money saving” ovh setup costs you more than you can possibly save by not just using fargate or rds whatever.

Re: I returned to AWS and was reminded why I left

#250
post #56

Earlier quoted context omitted.

Competition would mean Amazon creating their own software. Taking software others made and using your monopoly eco-system and scale to drive the original creator out of the game kills the product. Many support breaking up Amazon so others could compete not killing small entities and growing Amazon.

That's a risk they knowingly chose to accept when they opted for FOSS licensing. It's not as if people hadn't asked " Well, what if another party tries to fork our open source code for profit? " all the way back when FOSS was starting to gain traction in the 1990s.

OSS licensing.

Free Software was designed to avoid this, and has become stricter as the technology changed. Open Source was deliberately designed to thwart this. The entire intention of it was to allow businesses to resell work that was done for free. When you fork Free Software, your fork is also Free Software.

Post reply on HN