Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS mistakes to avoid

cloudonaut.io

Post reply on HN
Print view View original

211–220 of 276 posts

Re: AWS mistakes to avoid

#211

Post by obulpathi »

peterwaller wrote: post #16

I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor! It seems like the assembly of the AWS ecosystem. Does anyone else have a favourite hammer for this particular nail? I'd love to have something better than our home-baked solution, but I'm yet to find anything which doesn't introduce other flaws, such as an inc…

CloudFormation is not very elegant. Things become complicated with the concept region, zonal resources (same AMI is represented by a different id in different region etc). Try Google Clouds's Deployment Manager. Functionally similar, but Google Cloud Deployment Manager is far easier (everything is a global resource), Jinja based templates (you get to write for's and if's, evaluate lists, dictionaries inside templates .. )

Re: AWS mistakes to avoid

#212

Post by tie_ »

zerotosixty wrote: post #7

You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…

As much as I agree with your point about being hard, I'd also like to add out that there isn't anything even remotely similar to documentation on how a pure dev person could do this outside of AWS, with the lack of managed cognito, API gateway + lambda, and dynamodb.

Making those notions and technologies easy and cheap to access AWS suddenly gave devs the idea that they can roll out complex infrastructure on their own, similar to copy-pasting a piece of code. Well, it is still a bit harder than that, and if you are that kind of dev (which I'd applaud), you'd better dedicate some time to learning those technologies.

Re: AWS mistakes to avoid

#213

Post by draw_down »

ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

I can't believe this is the top comment. This is supposed to be about AWS advice, not your personal feelings about AWS. This comment is bad and you should feel bad and everyone who upvoted it should feel bad.

Re: AWS mistakes to avoid

#214

Post by mgkimsal »

bfung wrote: post #194
comex wrote: post #173

Earlier quoted context omitted.

$20k/month is $240k/year, which is at least two developer salaries. Do you really need two full time devops/whatever to manage the open source equivalents to the AWS services? Aren't there tools that can provide a similarly high level/seamless/polished experience running on your own computer, using your choice of cloud provider as the backend? If not, why not? (I wouldn't know - I don't even run a website, and if I d…

From a management standpoint, it takes at least 2 developers to run something equivalent. Perhaps not full time, but it doesn't scale out that much - not more than 10 highly available services per team. A good example of this is roughly a 3 instance RabbitMQ cluster used by ~100 people. 1. The "Bus Factor" - https://en.wikipedia.org/wiki/Bus_factor 2. Context switching - if you have a developer that also works on cod…

I'd think you'd need those staff (or approximate) regardless of whether you used AWS or not, no? It ain't the marketing team that orchestrates all that amazon stuff.

At scale, the arguments about needing 10 admins vs needing 15, for example, start making sense.

Re: AWS mistakes to avoid

#215

Post by BogusIKnow »

tie_ wrote: post #208
BogusIKnow wrote: post #199

Earlier quoted context omitted.

I'm working at a medium, growing startup and wonder when/why to use AWS? We currently use a mixture of root servers (DB) and VPS (App, HAProxy, NGinx) at a hoster. * We use a Docker/Consul/Nomad/SaltStack setup to install new servers for replacing broken ones and growth * App setup is Redis/Disque, Postgres(DB, BI) and app servers * New dedicated servers we get in around 1h, VPS in around 1min (Only VPS over API) * W…

0.2 Full-time employees to build all the automation, scale up (and down!) the stack as needed, monitor, operate and maintain it at an HA level? That is a huge feat, and I'd love to read about how you pulled it off. Please consider sharing your story, and perhaps your Salt recipes ;) With all those things that you have in place, AWS would not be a big win for you, though you still might find some use of AWS services l…

It takes some weeks to get it working, I'd say two to four, depending how often you've done it. Then for weeks there is nearly no Ops at all (security updates only mostly while the servers are humming) with releases done with Docker push by developers. Adding and provisioning a new VPS takes some seconds (and some seconds later checking), with Consul the stuff running on the VPS usually integrates itself). Sometimes hardware breaks down, then it's either setup a new VPS (setup_server ) which takes around 5sec or mail the hoster if it's a DB server with a (mirrored RAID) SSD crash, which takes around 2h to fix all in all. So on average per year I'd guess 0,2 Ops FTE (which equals 10 weeks a year, which is a lot if you're not Facebook and don't need to do releases of new app versions). Today ops is relativly easy with a well trodden path, it's not 1999.

The Salt is of the shelf, with some bash scripts on top to auto generate some config and make installation of a new server one script call.

We surely could put endless hours into Ops, playing around with stuff etc. but the benefit would be marginal.

Our DNS setup is simple, not much to say, we use DNS Made Easy for failover, we use S3 though I do not consider it "AWS" (S3 is comparativly cheap).

I've wondered about Mesosphere with autoscaling etc. but with adding a new VPS in seconds setting up Mesosphere/Kubernetes taking two weeks would take quite some time to amortize.

Re: AWS mistakes to avoid

#216

Post by Jhsto »

obulpathi wrote: post #206
jasode wrote: post #197

Earlier quoted context omitted.

The "alternatives" depend on your situation. Picture a continuum between brain-dead simple websites and business-critical complex websites: simple: static website, WordPress blog moderate: small business CMS, etc complex: Netflix, AirBNB If you're running a simple WordPress blog, the AWS prices are absurdly overkill. For this use-case, there are a zillion alternatives. Linode, Digital Ocean, Rackspace bare metal, etc…

If you look at the speed at which Google Cloud has been adding products for past 6 months, you will be amazed. Google Cloud is now feature complete with AWS and 90% of its products are far better than AWS.

I've found Google Compute Engine decent, but with App Engine I ran into difficulties with vendor lock-in. I had to first modify my code to get the app run on my local computer. After that I never got it deployed because GAE requires me to bind their own TCP context for outbound connections. This was a shame because I would have wanted to have my app on GAE and database on Heroku.

Just saying that calling 90% of Google Cloud's products to be far better than AWS (or any equivalent provider for that matter) is bold.

Re: AWS mistakes to avoid

#217

Post by chucky_z »

latch wrote: post #150
chucky_z wrote: post #140

Earlier quoted context omitted.

I simply don't believe the $250 - $20,000 month comparison. Could you please provide some real-world numbers/hardware/stack information? I just did this migration (non-AWS -> AWS), and we saw a ~20% reduction in monthly prices with 1-year contract at AWS vs. physical hosting (admittedly we were getting terrible pricing with our physical hosts, but still). This will be reduced with the addition of t2.nano servers as w…

Your parent is [rightfully] including bandwidth. If he was able to get 1gbps unmetered, then the math is in the right ballpark (that's a huge if though). Still, I agree with the parent's general sentiment re price. The gap used to be much worse, but it's still anywhere from 2x-20x depending on what you're doing. - EC2 prices are simply high and exclude things like bandwidth. - The CPUs are usually 1-3 generations beh…

I think in this case I could have interchanged "terrible" with "enterprise" and it would've had a more ""correct"" meaning (seriously I wish I could keep adding quotes to all of that...). I had to fight and argue with the previous host (Rackspace) to get them to lower our prices. It also took weeks to deploy a server. Now it takes 5-25 seconds with a prebuilt AMI that I've cooked together, deployed on exactly the hardware that I want, with the network configuration that I setup, and the security rules (bi-directional! Rackspace firewall only offers inbound at the level we had) that I want/need. On top of this, to get lower prices, I open a ticket and say "Hello, I'll be needing this server for 1 year, lower prices please." No phone calls involved. I'm not trying to diss Rackspace, they seem to be doing a lot of cool things... I simply had a permanantly sour taste left in my mouth. AWS is a big juicy BBQ sandwich in comparison.

On your points about AWS itself I'm not going to argue with as they certainly are valid, however in my specific setup (everything production is c4/r3 types, 1tb gp2 volumes as all volumes, and running network benchmarks to ensure everything is properly sized) I've actually had a pretty big performance gain over dedicated servers in a DMZ behind a hardware firewall.

With regards to the bandwidth, it seems a common issue is when you have very chatty services (Apache Kafka) deployed across multiple AZ's the bill gets real big real fast. Thus far, we're not even cracking 3-digits in bandwidth monthly. Maybe we're simply not at a scale to notice these problems yet.

Re: AWS mistakes to avoid

#218

Post by zonywhoop »

tomglindmeier wrote: post #93

I wonder if AWS is a good location to run a VoIP server. VoIP is a real time application that is very prone to jitter, latency and packet loss. I'm concerned about "noisy neighbors" and decreased network performance at AWS. Does anybody have experience with running a VoIP (e. g. Asterisk) on AWS?

Check out voicehub.com, their entire stack is on ec2.

Re: AWS mistakes to avoid

#219

Post by chucky_z »

kelnos wrote: post #198
chucky_z wrote: post #137

Earlier quoted context omitted.

What about as a kind of Redis stand-in? I'm curious about the DynamoDB PHP session driver. It sounds like you have a lot of experience with Dynamo. All the use-cases I seem to keep coming up with are more for storing global environment keys outside of the environment itself and using a large number of IAM roles to access individual keys, and as a kind of throwaway 'I need to store this somewhere, but it doesn't reall…

Not the grandparent, but I have a lot of experience with DynamoDB as well. I migrated a self-managed, sharded redis store to DDB close to a year ago, and what it really depends on are your read and write patterns, and whether or not you can live with the opacity of what DDB is doing behind the scenes. An example: if you provision X capacity units, you're actually provisioning X/N capacity units per partition. AWS is…

Was the ElastiCache service around when you migrated Redis to Dynamo? This is another alternative I'm considering.

Re: AWS mistakes to avoid

#220

Post by Hadi_Asiaie »

ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

I've been a customer of linode for over a year and soon will be migrating to another provider, most probably Amazon. The main reason is a lot of connectivity issues and down time. I always hosted pet projects and simple web app on Linode, but just this week, I used it a for a major project. Now, the whole region of Fremont,CA has connectivity issues and is down for a couple of hours per day. The same happens for other regions as well. Maybe always it's been like that, but I've never paid attention. You can see their incident history in here: http://status.linode.com (I browsed it a little bit and it seems they had a lot issues in the past as well) I probably will lost about 6 times more money that Linode has saved me over AWS only for the down time in this week.

So my suggestion is for a small-non critical projects you can use Linode, but for anything serious you better use something that is reliable.

Post reply on HN