101–110 of 284 posts
> You don’t have to randomize the first part of your object keys to ensure they get spread around and avoid hotspots. As of when? According to internal support, this is still required as of 1.5 years ago.
He's not talking about the prefix, just the beginning of the object key.
Earlier quoted context omitted.
I just stick CloudFront in front of those buckets. You don't need to expose the bucket at all then and can point it at a canonical hostname in your DNS.
For the sake of understanding, can you explain why putting CloudFront in front of the buckets helps?
- signed url's in case you want a session base files download
- default public files, for e.g. a static site.
You can also map a domain (sub-domain) to Cloudfront with a CNAME record and serve the files via your own domain.
Cloudfront distributions are also CDN based. This way you serve files local to the users location, thus increasing the speed of your site.
For lower to mid range traffic, cloudfront with s3 is cheaper as the network cost of cloudfront is cheaper. But for large network traffic, cloudfront cost can balloon very fast. But in those scenarios S3 costs are prohibitive too!
> You don’t have to randomize the first part of your object keys to ensure they get spread around and avoid hotspots. Not strictly true.
Earlier quoted context omitted.
VPC sharing is the sleeper here. You can do cross account networking all in the same VPC and skip all the expensive stuff.
as long as your VPCs aren't too big, yea.
Want to set up MFA ... login required to request device.
Yes, I know, they warned us far ahead of time. But not being able to request one of their MFA devices without a login is ... sucky.
I have a preempt-able workload for which I could use Spot instances or Savings Plans. Does anyone have experience running Spot in 2025? If you were to start over, would you keep using Spot? - I observe with pricing that Spot is cheaper - I am running on three different architectures, which should limit Spot unavailability - I've been running about 50 Spot EC2 instances for a month without issue. I'm debating turning…
1. Spot with autoscaling to adjust to demand and a savings plan that covers the ~75th percentile scale
2. On-demand with RIs (RIs will definitely die some day)
3. On-demand with savings-plans (More flexible but more expensive than RIs)
3. Spot
4. On-demand
I definitely recommend spot instances. If you're greenfielding a new service and you're not tied to AWS, some other providers have hilariously cheap spot markets - see http://spot.rackspace.com/. If you're using AWS, definitely auto-scaling spot with savings plans are the way to go. If you're using Kubernetes, the AWS Karpenter project (https://karpenter.sh/) has mechanisms for determining the cheapest spot price among a set of requirements.
Overall tho, in my experience, ec2 is always pretty far down the list of AWS costs. S3, RDS, Redshift, etc wind up being a bigger bill in almost all past-early-stage startups.
Earlier quoted context omitted.
>S3 is the right choice for causal users. It's so simple for storing and serving a static website. Are there good and cheap alternatives?
Yeah, your classic web hoster. Just today I uploaded a static website to one via FTP.
BTW: Is GitHub Page still free for custom domains? (I don't know the EULA)
I had a theory (based on no evidence I'm aware of except knowing how Amazon operates) that the original Glacier service operated out of an Amazon fulfillment center somewhere. When you put it a request for your data, a picker would go to a shelf, pick up some removable media, take it back, and slot it into a drive in a rack.
This, BTW, is how tape backups on timesharing machines used to work once upon a time. You'd put in a request for a tape and the operator in the machine room would have to go get it from a shelf and mount it on the tape drive.
Earlier quoted context omitted.
Gateway endpoints are free. Network endpoints (which are basically AWS-managed ENIs that can tunnel through VPC boundaries) are not free. S3 can use either, and we recommend establishing VPC Gateway endpoints by default whenever you need S3 access. (Disclaimer: I work for AWS, opinions are my own.)
That's fascinating! I hadn't found that in the documentation; everything seems to steer people towards PrivateLink, not gateway endpoints. Would you recommend using VPC Gateway even on a public VPC that has an Internet gateway (note: not a NAT gateway)? Or only on a private VPC or one with a NAT gateway?
Gateway endpoints only work for some things.
Earlier quoted context omitted.
Fine for when you have no NAT gateway and have a subnet with truly no egress allowed. But if you're adding a NAT gateway, it's crazy that you need to setup the gateway endpoint for S3/DDB separately. And even crazier that you have to pay for private links per AWS service endpoint.
There's very real differences between NAT gateways and VPC Gateway Endpoints. NAT gateways are not purely hands-off, you can attach additional IP addresses to NAT gateways to help them scale to supporting more instances behind the NAT gateway, which is a fundamental part of how NAT gateways work in network architectures, because of the limit on the number of ports that can be opened through a single IP address. When…