41–50 of 51 posts
This is great and all, but I'd love to see a community wiki / discussion area for AWS tips, tricks, and gotchas. Something like Quora crossed with Wikipedia, just for AWS. Is anyone aware of such a thing? If not, any advice for starting one?
http://aws.amazon.com/forums
Earlier quoted context omitted.
That's the AWS CLI (Command Line Interface), available at http://aws.amazon.com/cli/ .
Is there a huge difference with s3cmd?
Also the 's3cmd sync' and 'aws s3 sync' commands don't have exactly the same heuristic so switching caused a one-time resync of everything.
"9) Use Virtual Private Cloud (VPC) from the start" This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go. The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet. Setting up a bastion is reasonably straight-forward. There is an AM…
Why not use a vpn on the bastion?
Description of noisy neighbor problem #6 lacks some depth. AWS noisy neighbors problem is very often misunderstood. CPU steal time under linux does NOT mean that somebody is stealing your CPU. It simply means that you wanted to use CPU and hypervisor has given it to another instance. This may happen because you have exceeded your quota or scheduling algorithm selected another pending instance at this very moment and…
Given that distributed network rely heavily on network latency and capacity, I found it very interesting to see the effect of busy CPU propagate to slower network IO.
"9) Use Virtual Private Cloud (VPC) from the start" This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go. The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet. Setting up a bastion is reasonably straight-forward. There is an AM…
"9) Use Virtual Private Cloud (VPC) from the start" This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go. The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet. Setting up a bastion is reasonably straight-forward. There is an AM…
Can you point to a blog post / article that describes this in greater detail?
Earlier quoted context omitted.
Why does this happen even when the instance isn't using all of its CPU?
>Why does this happen even when the instance isn't using all of its CPU? Probably hypervisor does not assign CPU every time it is requested but it still manages to assign as much as needed because in the end there is some idle time left.
But rebuilding the instance brings them all in line.
Earlier quoted context omitted.
One thing to note with EMR: you still pay 25% of the ondemand price as overhead to use EMR. If you're bringing up and turning off clusters all the time, it's probably worth it, but you might want to look into using Whirr instead.
You've a good point about the EMR charge - that's easy to overlook. I took a look at Whirr [1] but I don't see how having a cloud-agnostic platform helps - are there really alternatives to EMR out there? Can they give me 500+ cc2.8xlarge equivalent machines on-demand but at spot prices? [1] Assuming this is the Whirr to which you refer: https://whirr.apache.org/