What would make AWS even better • Hacker News Board

Re: What would make AWS even better

#51

Post by philliphaydon » Sat, Sep 03, 2022, 2:29 AM UTC

paranoidrobot wrote: post #22

Network Load Balancers supporting security groups. I've gone through a bunch of audits, and automated scans, and I constantly have to explain this shit, even to AWS Employees. How it works with ALBs, which do support security groups: You want to receive traffic on port :443, and allow it to be accessible to the world. You have EC2 instances, and they are listening on the VPC at port :1234 So, you create: - ALB my_alb…

I don’t think that’s right. You wouldn’t use 0.0.0.0/32 you would use the private subnet range so they have no direct access to the internet and use a nat gateway to give them access to the outside world but no inbound access.

Edit: in the first example the servers should be in a private subnet. And not have a public ip allocated. They would require ssh hopping via a bastion or a vpn.

Re: What would make AWS even better

#52

Post by [deleted] » Sat, Sep 03, 2022, 2:44 AM UTC

philliphaydon wrote: post #50

datalopers wrote: post #4

Lambda is insanely expensive is why they don’t allow long-running jobs. A 1GB allocation is $43/mo. And most lambda users are running 1 single task/process per lambda invocation.

It sounds like you’re trying to use lambda for the wrong things and complaining.

[deleted]

Re: What would make AWS even better

#53

Post by scarface74 » Sat, Sep 03, 2022, 3:23 AM UTC

killyourcar wrote: post #48

scarface74 wrote: post #43

Earlier quoted context omitted.

https://www.amazon.jobs/en/landing_pages/pto-overview-us 15 PTO days. 6 personal days Yes I work at AWS. I’m never on call and I haven’t worked for more than 40 hours unless I’m learning something new trying to figure out. I control my own calendar and I manage expectations for my projects. I do work in ProServe though…

I work in tech in Seattle. AWS is notably worse then its peers. And even still, your experience is not typical for AWS.

Have you thought that of a company had over 1.5 million employees, everyone’s experience wouldn’t be the same?

Besides, I purposefully put myself in a position that I wouldn’t have to relocate to a high cost of living area. I knew that Azure, AWS, and GCP had a Professional Services department that may require a lot of travel. But no relocation, no on call, etc.

Then a worldwide pandemic happened that reduced travel…

Re: What would make AWS even better

#54

Post by moltar » Sat, Sep 03, 2022, 3:55 AM UTC

pid-1 wrote: post #10

Long running lambdas would be sick for infrequent / low concurrency data pipelines. My wet dream is "bidirectional IaC". Let me make changes using the GUI, commit to repo automatically.

Take a look at former2 and cdk-dasm.

Re: What would make AWS even better

#55

Post by moltar » Sat, Sep 03, 2022, 3:58 AM UTC

social_quotient wrote: post #27

We have a process where we strip out the text of PDFs and shove in to elastic. The lambda starts by counting the pages and if it’s 250 or less it handles the job. If it’s larger than that we make the lambda kick the job to a temp ec2 instance which takes over the job. Our cutoff is around 250 pages but it’s highly dependent on text density. It would be great if the lambda could handle running long. Id probably even b…

Why not count the pages and then create a separate task for batches of N pages? You could orchestrate the batch thru parallel map in step function and bring together the results on the final step.

Re: What would make AWS even better

#56

Post by nathants » Sat, Sep 03, 2022, 4:37 AM UTC

a good pattern is using lambda to boot and then monitor ec2 spot. you the flexibility of lambda and the power of ec2 spot.

some external event triggers the boot lambda.

1 minute schedule triggers the monitor lambda.

Re: What would make AWS even better

#57

Post by thiht » Sat, Sep 03, 2022, 4:41 AM UTC

Environment variables on lambda aliases would be a good thing to implement, to make the thing somewhat usable.

Re: What would make AWS even better

#58

Post by killyourcar » Sat, Sep 03, 2022, 5:57 AM UTC

throwaway_4ever wrote: post #26

killyourcar wrote: post #15

What about, "treat AWS workers better"? Pay your people for their on call hours! Let them work on side projects and games in their spare time! Give them more than seven paid holidays. Give them more than two weeks vacation! Only six weeks of paid parental leave? I would absolutely be willing to pay more for AWS if I knew that amount was going to treating the poor folks who built it all better.

Amazon is a terrible place to work for your well-being generally (personal experience and data-based). But, 2 weeks vacation only applies to 1st year employees outside of CA. Seattle dev: 1st year -> 2 weeks, 2-4th year -> 3 weeks, 5th+ year -> 4 weeks. California dev: 1st year -> 3 weeks, 2-4th year -> 4 weeks, 5th+ year -> 5 weeks.

If the average Amazonian washes out in under two years, I think it's pretty fair to say most amazonians only get two weeks.

Re: What would make AWS even better

#59

Post by paranoidrobot » Sat, Sep 03, 2022, 7:29 AM UTC

philliphaydon wrote: post #51

paranoidrobot wrote: post #22

Network Load Balancers supporting security groups. I've gone through a bunch of audits, and automated scans, and I constantly have to explain this shit, even to AWS Employees. How it works with ALBs, which do support security groups: You want to receive traffic on port :443, and allow it to be accessible to the world. You have EC2 instances, and they are listening on the VPC at port :1234 So, you create: - ALB my_alb…

I don’t think that’s right. You wouldn’t use 0.0.0.0/32 you would use the private subnet range so they have no direct access to the internet and use a nat gateway to give them access to the outside world but no inbound access. Edit: in the first example the servers should be in a private subnet. And not have a public ip allocated. They would require ssh hopping via a bastion or a vpn.

If your traffic is coming through a NLB, then the instances attached to the target group need to have 0.0.0.0/0 as the permitted source. (Assuming you want traffic from anywhere)

If you dont believe me, try setting it up yourself.

As for instances being on a private VPC/non-public IPs thats deployment specific.

In any case, everything then complains about listening on strange ports with 0.0.0.0/0

Re: What would make AWS even better

#60

Post by benmmurphy » Sat, Sep 03, 2022, 8:24 AM UTC

paranoidrobot wrote: post #22

Network Load Balancers supporting security groups. I've gone through a bunch of audits, and automated scans, and I constantly have to explain this shit, even to AWS Employees. How it works with ALBs, which do support security groups: You want to receive traffic on port :443, and allow it to be accessible to the world. You have EC2 instances, and they are listening on the VPC at port :1234 So, you create: - ALB my_alb…

If you run the NLB in ‘ip’ mode instead of instance mode you don’t have this problem. You still can’t setup security groups. However, you can put the NLB in a small subnet then whitelist the subnet.

‘ip’ mode is very different from instance mode and might not be what you want. Though, ‘instance’ mode is subtly bugged if you are using cross zone load balancing.