i'm a devops noob. what tools should i use to log / monitor all my servers?
i don't want to learn some complex stuff like cheff/puppet btw.... anything SIMPLE?
AWS Tips I Wish I'd Known Before I Started
81–90 of 152 posts
Re: AWS Tips I Wish I'd Known Before I Started
#82Earlier quoted context omitted.
You can specify CORS headers for S3, or you can just use a standard form POST.
You still need a stub API for generating the signature to sign the upload requests to S3, correct?
Not technically, but generally in practice. You can open things up permissions-wise but run the risk of folks uploading lots of large files. Keeping permissions locked down and doing a signature allows things like file size, location, etc. restrictions.
Re: AWS Tips I Wish I'd Known Before I Started
#83Good article, but I think it touches too little about persistence. The trade-off of EBS vs ephemeral storage, for example, is not mentioned at all. Getting your application server up and running is the easiest part in operation, whether you do it by hand via SSH, or automate and autoscale everything with ansible/chef/puppet/salt/whatever. Persistence is the hard part.
Good point. We're struggling to see the benefits of EBS for Cassandra that has its own replication strategy (ie data is not lost if an instance is lost), voiding the "only store temporary data on ephemeral stores" argument.
How do you handle entire datacenter outages with ephemeral only setup? You can replicate to another datacenter, but if power is lost to both do you just accept that you'll have to restore from a snapshotted backup?
Re: AWS Tips I Wish I'd Known Before I Started
#84Earlier quoted context omitted.
Are you really going tolerate 10+ ms of latency between a database, and an app server? I have never personally tried it, but it seems like it would be a disaster.
Heh, heck my db and app live in the same data center, and the latency is more than 30-40 MS.
Really? That seems enormously too slow. Even over the internet, I'm within 5-10ms of most things in the bay area.
If you have a dozen slow (100us) switches, three slow (2ms) routers, and 500 kilometers of fibre (2ms), you should still be under 10ms each way. And that's enough gear to cross some countries.
I'm curious how you can do much worse than this in a data center. Maybe some pokey "application firewall" or massive buffer bloat?
Re: AWS Tips I Wish I'd Known Before I Started
#85Lots of very useful tips there! There's one that I think could be improved on a little: Uploads should go direct to S3 (don't store on local filesystem and have another process move to S3 for example). You could even use a temporary URL[0,1] and have the user upload directly to S3! [0]: http://stackoverflow.com/questions/10044151/how-to-generate-... [1]: http://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlU...
Wow, didn't know about pre-signed URLs, very useful. I've added this info to my article, thanks!
Re: AWS Tips I Wish I'd Known Before I Started
#86Earlier quoted context omitted.
True, but outside of S3 and Route53, how much under the AWS umbrella is much use without using at least one EC2 instance? I can see a lot of benefit to using S3 without EC2, but after that, I'm not sure what else would be possible. Care to elaborate more? Can you use their queues and database tools w/o using EC2? (If you are using a VPC, maybe?)
You're moving the goalposts. First you said: AWS really only makes sense ($$) when you can take advantage of the ability to spin up and spin down your instances as needed. After it's pointed out that other facilities besides EC2 motivate people to use AWS, you can't turn around and complain that those other facilities are useless without EC2. Even if that were true (it's not), such people would be using EC2 not for i…
Who's moving goal posts? After it was (correctly) pointed out that AWS!=EC2 (even though most of the original post was talking about EC2), I asked a simple question - how useful are the rest of the services outside of EC2? It is a legitimate question.
I'm not sure how useful everything in the AWS umbrella is outside of EC2. I was hoping to learn something, not provoke anyone.
Re: AWS Tips I Wish I'd Known Before I Started
#87Earlier quoted context omitted.
True, but outside of S3 and Route53, how much under the AWS umbrella is much use without using at least one EC2 instance? I can see a lot of benefit to using S3 without EC2, but after that, I'm not sure what else would be possible. Care to elaborate more? Can you use their queues and database tools w/o using EC2? (If you are using a VPC, maybe?)
The problem here is that you're classifying everything AWS does as "EC2", which then causes confusion when someone else comes along and says "vendor X does everything EC2 does for half the price", and they mean solely the VM-related stuff, not the additional array of services.
That's a problem with the comments for this whole post. AWS means a lot of different things to different people. While I focused on EC2, others focused on RDS or S3 or ELB, etc... it's easy to get off on a tangent.
When everything is covered under the AWS umbrella, it's easy to argue from any side you want.
I still think that it's important to consider whether or not AWS (for all values of AWS) correctly fits your workload. You may not need the ability to spin up servers within minutes, or have multi-datacenter redundancy in a database, or have virtually unlimited storage, or a robust queuing system. A lot of great engineering has gone into all of the AWS products, and for many instances it is probably overkill. And that can cost you a lot if you don't know what you're doing.
Re: AWS Tips I Wish I'd Known Before I Started
#88Can anyone explain how disabling ssh has anything to do with automation? We automate all our deployments through ssh and I was not aware of another way of doing.
ssh is handy if you're creating instances and then setting them up. However, if you're doing that on a regular basis, you might ought to use custom AMIs instead. Then (with proper "user data" management) you can just roll out instances that are already set up how you want.
Re: AWS Tips I Wish I'd Known Before I Started
#89Earlier quoted context omitted.
AWS makes a lot of sense in a 24/7 environment, particularly when you are a new startup and don't have enough information (or capital!) to make educated server purchases.
With the additional caveat that you have tons of money sure. With 24/7 load, you'd probably pay a 10x premium to use AWS.
Reserved Instances really drive the cost down quite a bit, but yes, it's expensive.
Re: AWS Tips I Wish I'd Known Before I Started
#90Earlier quoted context omitted.
DigitalOcean - Same or better speed, 1/6 or less of the price of ECC.
AWS is PCI compliant. http://aws.amazon.com/compliance/pci-dss-level-1-faqs/ DigitalOcean and most others are not. AWS's security and API is lightyears ahead of everyone else.
Yes, this is all true. But the question remains - how much is that worth to you? For some it will be mandatory. For others (particularly startups), not as much.