Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Tips I Wish I'd Known Before I Started

wblinks.com

Post reply on HN
Print view View original

101–110 of 152 posts

Re: AWS Tips I Wish I'd Known Before I Started

#101

Post by milkshakes »

richadams wrote: post #92
michaelmior wrote: post #50

Disabling SSH is an interesting tip. I guess the OP doesn't do any automation via SSH.

Just disabling inbound SSH connections, the servers can still SSH out to other systems to pull in files, configurations, clone git repos, etc. It's just a way to stop yourself from cheating and SSHing in just to fix that one thing, instead of automating it.

except that some automation frameworks rely on inbound ssh access to the machines. ansible would be an example of such a framework, in its default configuration at least.

Re: AWS Tips I Wish I'd Known Before I Started

#102

Post by mvleming »

AaronBBrown wrote: post #43
bearbin wrote: post #33

Earlier quoted context omitted.

DigitalOcean - Same or better speed, 1/6 or less of the price of ECC.

DO has a very limited API, no ability to add additional storage without resizing your droplets, and has no firewall protection without iptables being enabled. DO has its uses (I'm migrating my personal server there presently), but it's not even close to being in the same market/caliber as EC2. For some businesses, the huge AWS feature set (RDS, EBS, ELB, security groups, VPC, ELB, EIPs, etc, etc, etc) is more valuabl…

Hey, I've started to learn about devops and systems administration recently and I've learned a ton in this thread and this article, so thanks for that and thanks to everyone else.

But do you know of any good resources to learn about those two things? And I'm taking about basic devops, before you even start worrying about automating, and the things you would actually automate–because I don't know what it is I should be doing in the first place.

Things like what you should be doing right after you SSH into your server, how to make your server secure, how to use nginx, chmod'ing permissions of files correctly, and things I don't even know about.

Is there a One Month Rails or Michael Hartl's RoR Tutorial for devops/sys admin?

Regardless, thanks for taking the time to read this :)

Re: AWS Tips I Wish I'd Known Before I Started

#103

Post by samplonius »

freerobby wrote: post #55
pavel_lishin wrote: post #37

Earlier quoted context omitted.

Based on the article, it seems it's there to make sure that you're automating everything, instead of logging in to do that one little thing by hand.

Thanks. Does anybody else here agree with this mentality? This seems a major mispractice to me. I've worked at companies with as few as two people to as many as 50,000 people. None of them have had production systems that are entirely self-maintaining. Most startups are better off being pragmatic than investing man-years of time handling rare error cases like what to do if you get an S3 upload error while nearly out…

Facebook has dozens of sysadmins working 24x7, but they also have 200,000 servers.

While diagnosing issues may require logins, you aren't going to be fix anything on 200,000 servers without automation. At large scale, all problems becomes programming problems.

It also means the death of jobs for sysadmins that only know how to go in tweak *.conf files and reboot servers. So, I guess that sucks for you.

Re: AWS Tips I Wish I'd Known Before I Started

#104

Post by osteele »

NathanKP wrote: post #60
mbreese wrote: post #11

I'd also add to the list - make sure that AWS is right for your workload. If you don't have an elastic workload and are keeping all of your servers online 24/7, then you should investigate dedicated hardware from another provider. AWS really only makes sense ($$) when you can take advantage of the ability to spin up and spin down your instances as needed.

The startup I'm working for has minimal scaling required but we still use AWS despite the higher cost for EC2 because the broad ecosystem of AWS products make it easier to develop interesting things quickly and efficiently. If we went with all of our own dedicated hardware, or cheaper instances from a different cloud provider then we'd miss out on ELB, have slower and more expensive communication to and from S3, not…

I think of AWS as a tool for prototyping and early-stage outsourcing of your infrastructure. Use it when you're fighting past market and technology risk with a single-digit team; consider dropping it in order to optimize costs when you've got more people (including fractional people) to evaluate, configure, and operate alternatives.

Re: AWS Tips I Wish I'd Known Before I Started

#105

Post by freerobby »

samplonius wrote: post #103
freerobby wrote: post #55

Earlier quoted context omitted.

Thanks. Does anybody else here agree with this mentality? This seems a major mispractice to me. I've worked at companies with as few as two people to as many as 50,000 people. None of them have had production systems that are entirely self-maintaining. Most startups are better off being pragmatic than investing man-years of time handling rare error cases like what to do if you get an S3 upload error while nearly out…

Facebook has dozens of sysadmins working 24x7, but they also have 200,000 servers. While diagnosing issues may require logins, you aren't going to be fix anything on 200,000 servers without automation. At large scale, all problems becomes programming problems. It also means the death of jobs for sysadmins that only know how to go in tweak *.conf files and reboot servers. So, I guess that sucks for you.

Agreed, but two important notes:

1) Automation doesn't have to be complete automation. I can use Chef tools like knife-ssh to run a single command on every one of our boxes in near-real time. This might not be automated to the extent that the OP is referring but it's 99.8% more efficient than logging into 500 boxes to do it (or 200,000 in Facebook's case). If you can get 99.8% with ease, it may not be worth pre-automating for that final 0.2%.

2) Knowing what is worth automating comes from experience. If I have to do something nasty once, it might be a total waste of time to automate it. There are lots of one-off things I type at a command prompt that are quicker to run than to automate. I think being so dogmatic about automation as to say you should never run things at a command line requires you to spend peple's time non-optimally.

Re: AWS Tips I Wish I'd Known Before I Started

#106

Post by AaronBBrown »

banachtarski wrote: post #94
AaronBBrown wrote: post #76

Earlier quoted context omitted.

A bunch of Chef cookbooks does not AWS make. Configuration management tools are of course a necessity in AWS but do not replace their offerings. I'm very impressed that you were able to build in 3 weeks time a low-latency multi-data center application with master/slave database failover, robust fault tolerant load balancing, and backups that can be restored in minutes with an API to control all of those services. Tha…

had master slave, had fault tolerant load balancing, had backup scripts, tested restore procedures. also had node upgrade procedures, and more. it was more than just chef obviously but chef + any bare metal host environment gets you a large percentage of the way there. Tacking on specific aws services like route53 when necessary works too.

That's a great accomplishment in such a short period of time. I'm glad that you were able to save time and money by using another service. Thankfully, there are tons of options out there to suit every business' use cases.

Re: AWS Tips I Wish I'd Known Before I Started

#109

Post by carlio »

novaleaf wrote: post #81

i'm a devops noob. what tools should i use to log / monitor all my servers? i don't want to learn some complex stuff like cheff/puppet btw.... anything SIMPLE?

Though I haven't tried it, people tell me that ansible is pretty simple - http://www.ansible.com/home

For logging, try logstash? http://logstash.net/

Monitoring... well that's a large and complicated topic!

Re: AWS Tips I Wish I'd Known Before I Started

#110

Post by themartorana »

mbreese wrote: post #90
joevandyk wrote: post #69

Earlier quoted context omitted.

AWS is PCI compliant. http://aws.amazon.com/compliance/pci-dss-level-1-faqs/ DigitalOcean and most others are not. AWS's security and API is lightyears ahead of everyone else.

Yes, this is all true. But the question remains - how much is that worth to you? For some it will be mandatory. For others (particularly startups), not as much.

If said startup is storing any personal data, it better be important.
Post reply on HN