Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS mistakes to avoid

cloudonaut.io

Post reply on HN
Print view View original

141–150 of 276 posts

Re: AWS mistakes to avoid

#141

Post by chucky_z »

cachemiss wrote: post #69

So I'd modify these a bit. We run a very large AWS infrastructure as a engineering team (no dedicated ops). 1. Use CloudFormation only for infrastructure that largely doesn't change. Like VPC's, subnets/ internet gateways etc. Do not use it for your instances / databases etc, I can't recommend that enough, you'll get into a place where updating them is risky. We have a regional migration (like database migrations) th…

Have you looked at Terraform? I have everything defined (unless it doesn't work very well... which still happens as it's still under heavy development), and if it needs to be dealt with care (e.g.: core EC2 instances) I'm slowly filtering it into a separate set of units/variables, and setting the "static" infrastructure (VPC) as a downstream group, and slurping the statefile upstream as to not potentially damage anything when playing nice with deploying/redeploying EC2 instances.

Re: AWS mistakes to avoid

#142

Post by chucky_z »

cachemiss wrote: post #69

So I'd modify these a bit. We run a very large AWS infrastructure as a engineering team (no dedicated ops). 1. Use CloudFormation only for infrastructure that largely doesn't change. Like VPC's, subnets/ internet gateways etc. Do not use it for your instances / databases etc, I can't recommend that enough, you'll get into a place where updating them is risky. We have a regional migration (like database migrations) th…

Have you looked at Terraform? I have everything defined (unless it doesn't work very well... which still happens as it's still under heavy development), and if it needs to be dealt with care (e.g.: core EC2 instances) I'm slowly filtering it into a separate set of units/variables, and setting the "static" infrastructure (VPC) as a downstream group, and slurping the statefile upstream as to not potentially damage anything when playing nice with deploying/redeploying EC2 instances.

Re: AWS mistakes to avoid

#144

Post by devonkim »

late2part wrote: post #131
devonkim wrote: post #127

Earlier quoted context omitted.

Most of the reasons to go with AWS as a major strategic decision are to avoid paying for 3-4 (or perhaps... 40+) ops engineers to run and manage so many things that really don't add much business value in themselves and that you could hold to some form of an SLA (especially with enterprise support accounts) and recoup some costs if things came down to it. Furthermore, engineering a lot of AWS services would cost you…

"The cost benefits get better the more incompetent / incapable your internal IT organization is. " Yes. There will always be a less stupid solution out there that is not as bad as the bad solution. Or, you could fix IT. But that takes thought. Easier to give money to a vendor you can blame...?

Fixing completely dysfunctional F500 IT organizations is something that many, many people have tried across completely different company cultures with maybe a handful of successes I've heard of. Leading huge organizations (especially ones damaged by previously failed leadership efforts - think how attempts to refactor a legacy codebase go and multiply by an order magnitude in number of variables) is very hard and because large organizations tend to have so much capital working for them, IT management failures create a large opportunity (in theory) for optimization by others. In practice, consulting for organizations that don't want to fundamentally change how they do X (or think $ must solve everything and leadership becomes so fragmented into non-communicating middle managers) while wanting large changes to X are the equivalent of an obese person wanting to lose weight without changing their diet or exercising. Yet we know there's a huge "fitness" industry with large profits being made off of those that do not use their products and subscriptions effectively.

Re: AWS mistakes to avoid

#145

Post by kawsper »

ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Hi, first of all, thanks for your comment, and your full-disclosure. I am a happy Linode customer, and I have been using your service since 2010-09-02 (that is the date of the first invoice), but I have a few issues with your product that you might be able to look at.

- It is too hard to figure out that you offer hourly billing, I found it by accident, and a year too late.

- I tried out your Nodebalancers, as I thought they would be a nice help for my stack, but I ended up building my own Haproxy solution since the performance wasn't good. When I pay for a dedicated loadbalancers, I would expect it to be top-tuned from your side.

- Your API is a mess, and so is your documentation. Everything is GET-requests. And creating a new instance, shouldn't take 11 requests, and a dance to set every little detail correctly. And to be fair, your documentation doesn't even document that, this is a subset of what I use: https://gist.github.com/kaspergrubbe/756f0a227db8aeb92818 I even managed to find some errors in the documentation when I went through it, so it didn't feel very polished or tested, but maybe it isn't used a lot. It also seems strange that to emulate one or two clicks in the interface, needs 11 requests through the API.

- Your advanced settings are extremely hard to find, especially "Edit Configuration Profile" and the "Auto-configure Networking"

- Elastic IPs. I feel like it should be easier to swap IPs around Linodes, without me using heartbeatd, and some strange configs. This is much easier with Amazon, and one of the reason why I prefer them.

- Anycast routing. Linode has several datacenters, it would be awesome if my setup in the US could have the same IP as my setup in the UK, because right now I am doing ugly DNS hacks to send my customers to the closest datacenter. Is this planned?

- Lack of an object store, even though I use Linode, I am still using AWS S3, which means added latency, and I have to pay Amazon for outside traffic, you can't even compare with AWS on these things without starting to have these things.

Can I setup a Linode through the API with private networking enabled?

I am a happy customer, and I loved your KVM upgrade, I love your prices, and I love your service. But it would be nice to see more innovations happening.

Re: AWS mistakes to avoid

#146

Post by loki77 »

ninjay wrote: post #12

What are the current ways to make creating CloudFormation templates not so painful?

You should check out troposphere (https://github.com/cloudtools/troposphere/) and stacker (https://github.com/remind101/stacker/). I'm a maintainer of both - we try to make CF easier by catching errors earlier, and allowing you to do things like write for loops, in troposphere. stacker tries to take your troposphere templates and tie them together as totally separate stacks.

Re: AWS mistakes to avoid

#147

Post by fizixer »

ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

As neither a AWS or Linode customer (but might become one) I have a couple questions:

- Can Linode offer short-term compute or memory intensive service? e.g., If I want to consume 100 teraflops 24x7 for about a couple weeks, or maybe 10 teraflops but a terabyte of memory. (Pay-as-you-go, not monthly/yearly subscription)

- On top of that, Can I do above with my IPython notebook code, which is already setup (it uses numpy at the backend so python slowdown is not a problem), with a step-by-step process of how to make it run on Linode (without having to install python scientific stack)?

These are some of the use cases that I think are getting popular these days, and for which AWS is known for, e.g., in the areas of machine learning, 3D rendering, etc.

Re: AWS mistakes to avoid

#148

Post by manyxcxi »

zerotosixty wrote: post #7

You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…

I agree that a lot more AWS tutorials and cookbooks would be helpful- but one reason I find so many developers having issues wrapping their head around AWS concepts is that most developers don't have the basic understanding of what they're doing with AWS. Programmers that just write code don't really need to know ANYTHING about AWS because someone else should be setting it up for them. I see so many people doing the most ridiculous things or leaving things completely wide open because they have no concept of network topology, firewalls, segments, VPCs, security groups, deployment templates, etc. I would not trust the vast majority of developers I've ever met to properly setup a simple architecture in AWS because the things you NEED to know aren't covered in the things they've been focusing on for most of their lives. It was predominantly in the domain of the operations/networking/system admin side of things.

I think there's a big need for a crash course for devs that starts with all the crap they previously ignored or had someone else do for them and I say this as someone who has always written code first and done sysadmin second.

Re: AWS mistakes to avoid

#149

Post by nicobn »

ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Since you're dealing in absolutes, let me return the favor; you're being a little silly.

Yes, Linode beats AWS when it comes to price. On the other hand, Linode's offer is incredibly basic and simplistic. AWS offers service after service that Linode simply doesn't and realistically cannot.

Sure, you can emulate a subset of these services (and a subset of their features) using open-source software but at what price ? That's the major flaw in your reasoning. Getting to the point where your installations are as stable and reliable as AWS', given a large stress on the system, will cost you a lot of money and time. Directly comparing the cost of hardware access is ignoring other costs and headaches that are not very easy to estimate.

There's a market for a service like Linode and there's a market for a service like AWS. You've simply never worked on a project/system that works better on AWS than on Linode. I know I couldn't run the systems that I currently operate on Linode without multiplying the workload that is needed to maintain them.

Linode and AWS are competitors but there's space in the market for both; they simply fill different niches. Establishing one as absolutely superior to the other is silly and closed-minded. A lot of people chose AWS; go and ask them why (feel free to reach out to me at nick at nasx dot io - I'll be more than happy to talk to you).

Re: AWS mistakes to avoid

#150

Post by latch »

chucky_z wrote: post #140
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

I simply don't believe the $250 - $20,000 month comparison. Could you please provide some real-world numbers/hardware/stack information? I just did this migration (non-AWS -> AWS), and we saw a ~20% reduction in monthly prices with 1-year contract at AWS vs. physical hosting (admittedly we were getting terrible pricing with our physical hosts, but still). This will be reduced with the addition of t2.nano servers as w…

Your parent is [rightfully] including bandwidth. If he was able to get 1gbps unmetered, then the math is in the right ballpark (that's a huge if though).

Still, I agree with the parent's general sentiment re price. The gap used to be much worse, but it's still anywhere from 2x-20x depending on what you're doing.

- EC2 prices are simply high and exclude things like bandwidth.

- The CPUs are usually 1-3 generations behind (they've gotten better with this).

- Virtualization adds overhead (again, this can be very significant if you're doing a lot of system calls)

- IO options are significantly worse

- Their network connectivity is, at best, average

You said it yourself, you're comparing a "terrible" deal to AWS and only saved 20%. Sounds to me like AWS is only 20% better than terrible ;)

Post reply on HN