Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things You Should Know About AWS

highscalability.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Things You Should Know About AWS

#31

Post by falcolas »

DrJ wrote: post #29
falcolas wrote: post #16

> Stripe your RDS disks for better performance This is a fun hack to perform, but it opens you up to another problem: latency on any of the striped EBS volumes will lag out the entire striped array. Attempts to mitigate this problem (including setting up raid 10) work in the short term, but it really is easier to just purchase a guaranteed iops volume if you want to run a database on EC2.

RDS is the hosted sql database solution. When RDS stripes your disks (I have seen it start when I jumped from 100GB to 300GB) you benefit from faster IOPS (I have seen 700 IOPS improve to 2500 IOPS. PIOPS (Provisioned IOPS) is the way to go in the long run though.

> you benefit from faster IOPS

Yes, you do.

However, if a single EBS that backs the RDS instance lags out (as EBS volumes in AWS are wont to do), you lose the iops.

The key point is that as the number of EBS volumes you depend on goes up, the chances of this happening goes up correspondingly.

Striping EBS volumes works, but I would not depend on it for a production environment.

Re: Things You Should Know About AWS

#32

Post by miles932 »

ideaoverload wrote: post #17

Description of noisy neighbor problem #6 lacks some depth. AWS noisy neighbors problem is very often misunderstood. CPU steal time under linux does NOT mean that somebody is stealing your CPU. It simply means that you wanted to use CPU and hypervisor has given it to another instance. This may happen because you have exceeded your quota or scheduling algorithm selected another pending instance at this very moment and…

"You are guaranteed to avoid noisy neighbors CPU problem by using AWS dedicated instances" This is incorrect. You simply ensure that the only ec2 instances that can be a noisy neighbor are other instances of yours. Since most users investing in dedicated instances tend to put their instances to work, this sometimes has the net effect of reducing performance.

Re: Things You Should Know About AWS

#33

Post by miles932 »

vosper wrote: post #4

This is great and all, but I'd love to see a community wiki / discussion area for AWS tips, tricks, and gotchas. Something like Quora crossed with Wikipedia, just for AWS. Is anyone aware of such a thing? If not, any advice for starting one?

http://aws.amazon.com/forums

Re: Things You Should Know About AWS

#34

Post by rpedela »

falcolas wrote: post #16

> Stripe your RDS disks for better performance This is a fun hack to perform, but it opens you up to another problem: latency on any of the striped EBS volumes will lag out the entire striped array. Attempts to mitigate this problem (including setting up raid 10) work in the short term, but it really is easier to just purchase a guaranteed iops volume if you want to run a database on EC2.

Yeah that is one reason we just use instance storage with database replication. Instance storage is generally faster and cheaper than EBS especially if you have a "very high I/o" instance.

Re: Things You Should Know About AWS

#35

Post by mdellabitta »

vosper wrote: post #5
anan0s wrote: post #3

Actually I was wondering if there is any experience/benchmarks about Amazon high-performance instances. anyone already used this ?

I am using a number of high-performance instances - cc2.8xlarge machines for EMR jobs, and cr1.8xlarge machines for analytics databases. From my testing: - Running EMR jobs on cc2.8xlarge machines as spot instances is a great way to get a LOT of computer power very cheaply. Because our jobs are periodic we run both Core and Task as spots and simply retry the job if our machines get terminated. I did a lot of benchmar…

One thing to note with EMR: you still pay 25% of the ondemand price as overhead to use EMR. If you're bringing up and turning off clusters all the time, it's probably worth it, but you might want to look into using Whirr instead.

Re: Things You Should Know About AWS

#36

Post by jwilliams »

"9) Use Virtual Private Cloud (VPC) from the start"

This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go.

The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet.

Setting up a bastion is reasonably straight-forward. There is an AMI that will do it all for you. Just make sure you've got "src/dst check" turned off in the EC2 panel for that server. Just that tip can save you hours of hair-tearing.

Outbound is via the bastion, which you can lock down to certain protocols via VPC security groups. I limit to HTTPS (no HTTP!).

The bastion should use ssh keys only (no username/password). Put in place fail2ban on the bastion. You can also add a firewall rule that backs off multiple fails SSH attempts. This all but nukes brute-force attacks. Also, make sure you patch regularly.

I go as far as to have separate keys for the bastion and then the hosts, but sensible policies should apply here (e.g. passphrase on your keys please).

Keep the bastion superuser login to a very select group. You can quickly remove employees' access by taking them out of the bastion. If you're in panic mode, you can turn off the bastion and isolate your network until you can regroup (similarly with the ELB for web-based attacks).

This is a pretty sound foundation for a secure setup.

Re: Things You Should Know About AWS

#37

Post by joevandyk »

jwilliams wrote: post #36

"9) Use Virtual Private Cloud (VPC) from the start" This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go. The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet. Setting up a bastion is reasonably straight-forward. There is an AM…

Why not use a vpn on the bastion?

Re: Things You Should Know About AWS

#38

Post by ademarre »

I don't hear enough people talking about vendor lock-in with AWS, or any other cloud provider for that matter. Too many of us are building our company's infrastructure on AWS with little regard for the cost of switching.

I try to stick with Amazon's IaaS offerings, only employing PaaS products when they offer considerable advantages over anything I could roll economically on bare infrastructure.

Ask yourself, "in terms of time or money, what would it cost me if this cloud product were discontinued without notice?"

AWS might lose a patent battle, get shut down by the government, or get crippled by hackers or a long-term service interruption. If that happened, how long would you be down while you struggle to get up and running at another provider? What if Amazon raises their prices to the point where you want to change cloud providers? Would the cost of switching be prohibitive?

Re: Things You Should Know About AWS

#39

Post by bcoates »

joevandyk wrote: post #37
jwilliams wrote: post #36

"9) Use Virtual Private Cloud (VPC) from the start" This is now a no-brainer. New registrations in certain zones will kick you into a basic VPC from the get-go. The only inbound should be via an ELB (HTTP/HTTPS) and an non-DNS-resolvable SSH bastion/NAT host (m1.small is more than enough). Your bastion is the only host that is on the public Internet. Setting up a bastion is reasonably straight-forward. There is an AM…

Why not use a vpn on the bastion?

Isn't a VPN a much bigger hole than ad-hoc port-to-port SSH tunnels?

Re: Things You Should Know About AWS

#40

Post by vosper »

mdellabitta wrote: post #35
vosper wrote: post #5

Earlier quoted context omitted.

I am using a number of high-performance instances - cc2.8xlarge machines for EMR jobs, and cr1.8xlarge machines for analytics databases. From my testing: - Running EMR jobs on cc2.8xlarge machines as spot instances is a great way to get a LOT of computer power very cheaply. Because our jobs are periodic we run both Core and Task as spots and simply retry the job if our machines get terminated. I did a lot of benchmar…

One thing to note with EMR: you still pay 25% of the ondemand price as overhead to use EMR. If you're bringing up and turning off clusters all the time, it's probably worth it, but you might want to look into using Whirr instead.

You've a good point about the EMR charge - that's easy to overlook.

I took a look at Whirr [1] but I don't see how having a cloud-agnostic platform helps - are there really alternatives to EMR out there? Can they give me 500+ cc2.8xlarge equivalent machines on-demand but at spot prices?

[1] Assuming this is the Whirr to which you refer: https://whirr.apache.org/

Post reply on HN