Live data from Hacker News

I returned to AWS and was reminded why I left

fourlightyears.blogspot.com

381–390 of 684 posts

Re: I returned to AWS and was reminded why I left

#381
We invested in colocation 2022-2024 for non-prod (log aggregation, Gitlab, warehouse databases, analytics loads, etc). Didn't know what kind of savings we accidentally set ourselves up for. Investing 3 months DO and AWS Bills permanently cut our spending, and since then has never seen an increase. If these systems go offline, it's an inconvenience but not a show stopper.

We intentionally engineer prod so it doesn't rely on any system in the colo (so nothing like 'store our config in git and the apps pull it on startup' type party tricks).

With memory prices right now it's harder to recommend expanding colocation but it's something every company needs to do (eventually). Not every system you have has equal production value.

Re: I returned to AWS and was reminded why I left

#382

Earlier quoted context omitted.

Some internal perspective - IAM has maybe thousands of options but fundamentally it is "what does this role have access to doing (action + resource)" + "who has access to this role". That is really it from a 10k foot level. IAM is great because it applies internally just like it does externally. The internal AWS team don't get more access than you do, and if we get access to do certain thing on your account to perfor…

> that's because you have a service principle in your IAM trust relationship that allowed us access That’s why it’s so complicated!!! I don’t understand how I should evaluate trust for your internal EBS org versus your internal ALB org. I kinda just expect it to be all “AWS” trust. And it’s all garbage anyway. There’s no way I can prevent the hypothetically untrustworthy EBS team from surreptitiously adding charges t…

3 things to untangle here.

1. It's about trust and auditability, while you may not want or need it, there are a lot of customer that are either interested or legally obligated to know who have accessed certain data.

2. It's about dogfooding - how would you trust an identity and access system when the company does not even use it internally?

3. In general, there are quick buttons and template to do it if you don't want to worry about it, in the LLM age, this gets easier. Personally I prefer this because I intensely dislike "magic". This allow you to control, to the maximum degree possible, what is actually going on, despite not owning any of the physical aspect of the data center.

Re: I returned to AWS and was reminded why I left

#383
post #302

Earlier quoted context omitted.

> if you have a choice I just read: > If I had learned one thing from my past life was that if you see the signs of an abusive relationship, you have the option to walk out, and you don't, all that follows is your own fault. so... :)

Regardless of whether the metaphor stands up, this is a horrible thing to say. Abuse victims are not responsible for the abuse they receive.

I had the same thought at first, but in context I think the quoted text refers to business relationships. Which makes all the difference.

Re: I returned to AWS and was reminded why I left

#384

Imo lambdas are super cool, and the best way to have a no-headache fast-iteration time deployment service. What most people realize, that you don't have to go microservice or fragment your code to a billion little repos, you could take a standard webserver, and move it to lambda, as long as you don't expect requests to be able to share on-server state.

What is lambda adding in this situation that throwing a docker image up somewhere isn't?

Re: I returned to AWS and was reminded why I left

#385
post #49

Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months. They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship. The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs. I had to have the two…

>> The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs. This is false. The price shows up right away when you select a machine. I dont work for AWS...

I do see prices when I look at the instance selection box. "On-demand linux base pricing: 0.0104 USD per hour" for t3.micro (matching the published pricing). it does not show the full price (based on any additional volumes or other configuration details).

It gets far more complicated when you have reserved instances, and combine reserved instances with RAM sharing when working in a larger org.

Re: I returned to AWS and was reminded why I left

#386
post #238

Earlier quoted context omitted.

Not really. It's cheaper than hiring an IT admin and sysadmin for a while. Those tend to be tricky hires on the small end since you tend to want jack-of-all-trades who either demands a premium salary or doesn't exist. When you have 10 software engineers, having 1 dedicate 10-20% of their time is cheaper than hiring 1-2 FTEs that aren't writing code.

Unless we're talking actual PaaS (Heroku, Render, Railway, etc), the cloud also needs a dedicated skillset, so "cloud" doesn't remove the need from a sysadmin. If you can get (and trust they do it right) developers to do AWS or Kubernetes, you should be able to trust them to do conventional Linux sysadmin on a bunch of dedicated boxes.

I suspect you're either severely underestimating what the cloud offers or thinking of a very narrow set of software businesses.

A full stack/backed dev is more than capable of learning both, but one of those has way more foot guns than the other.

Re: I returned to AWS and was reminded why I left

#387
post #49

Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months. They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship. The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs. I had to have the two…

> The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs.

I'm sorry, what? I just tried the EC2 launch wizard, and the price is listed right in the dropdown with the instance types. Or you can open a table with comparisons and enable the price there, along with ~20 other instance type properties.

Yeah, the AWS UI is not great. But they go out of way to make pricing predictable and public.

Re: I returned to AWS and was reminded why I left

#388

You can accomplish a lot by just having a basic knowledge of Linux sysadmin. I was clueless and then learned some systemd-and-curl-fu. Will never forget the "holy sh*t, this is deceptively simple" moment. A bit more research and I found that beyond convenience and specialty APIs, you really just don't need a lot of this stuff to run a healthy system (since reducing absolute cloud dependence, my reliability has gone t…

Just wait until you learn about system tools like perf, gdb, bpf -- the amount of low-level detailed information you can get about running processes means you'll reduce the amount of guesswork involved with troubleshooting or performance optimization to a minimum.

Re: I returned to AWS and was reminded why I left

#389
post #172

Earlier quoted context omitted.

...and AWS does it too. I can go right into an account and see an estimated cost per hour, and even pre-pay at a fixed discount for longer terms if I want to. They tell me right there what it will cost. They do this for everything that is reasonably a "fixed cost", like CPU time. They cannot predict what my bandwidth consumption will be, or other such variable costs. For those, they tell you rates.

No it's pretty bad. They show you the cost after all the resources are set up. Even setting up an ec2 instance, a really basic use case that has a fixed cost based on size, you have to go Google it and find their ec2 pricing table. It would take no space to just put the price per hour in the drop-down as you're picking an instance. But no, they obscure it on purpose. That's just for ec2. Everything is like this. Supe…

> Even setting up an ec2 instance, a really basic use case that has a fixed cost based on size, you have to go Google it and find their ec2 pricing table

This is the "Comparison Table" from the EC2 launch wizard: https://imgur.com/a/YjFhkzb

The pricing is right there, along with filtering and sorting.

Re: I returned to AWS and was reminded why I left

#390

Earlier quoted context omitted.

That's one of the things I like about Azure, they don't overwhelm you with listing prices beside every individual item as you're creating it, but they seem to always present a price on things that could be expensive. It's a good balance, I have yet to be surprised by a charge.

Using Azure in 2026 should be a firing offense. How many cross-tenant incidents are enough for you? In 20 years of existence of AWS ( since 2006 and S3 ) show me ONE with AWS ... and I will publicly eat my hat here... "Azure’s Security Vulnerabilities Are Out of Control" - https://www.lastweekinaws.com/blog/azures_vulnerabilities_ar...

Azure is a compliance requirement for us.

Haven't had anything impacting in GovCloud, but if you're not there yet I'm sure there's shenanigans in the consumer version.

Post reply on HN