Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS mistakes to avoid

cloudonaut.io

Post reply on HN
Print view View original

171–180 of 276 posts

Re: AWS mistakes to avoid

#171

Post by jacques_chester »

eropple wrote: post #49
andystanton wrote: post #33

Earlier quoted context omitted.

If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider ( https://terraform.io/docs/providers/aws/index.html ) which creates resources and maintains the state in a file that you can store in version control ( https://terraform.io/docs/state/index.html ).

Terraform, as an idea, is brilliant. Mitchell and company isolated a hugely important need and tried to fill it, and I give them all the credit in the world for that. Cross-platform cloud provisioning? Gimme. But I cannot in good conscience not relate what a disastrous experience Terraform has been for me at both jobs and clients. Writing reusable code in Terraform is an exercise in frustration due to the extreme clu…

If you're writing your own, you might also look to BOSH[1] for inspiration.

It's older than CloudFormation and Terraform (born 2010). It can manage anything that someone's written a driver for. So far that includes AWS, Azure, vSphere/vCloud, OpenStack, VirtualBox, Google Compute Engine, Apache CloudStack and there might be others I missed.

It stores state in a database. It is able to recover from mismatches between the state of the world and the desired state. Cloud Foundry users have been using it for years to deploy and update CF installations. Pivotal Web Services (I work for Pivotal, in a different division) has been upgrading to the most recent CF release every few weeks, live, without much fuss, for years.

For any kind of heavily stateful infrastructure, BOSH is a strong candidate.

[1] https://bosh.io/

Re: AWS mistakes to avoid

#172

Post by neya »

nicobn wrote: post #149
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Since you're dealing in absolutes, let me return the favor; you're being a little silly. Yes, Linode beats AWS when it comes to price. On the other hand, Linode's offer is incredibly basic and simplistic. AWS offers service after service that Linode simply doesn't and realistically cannot. Sure, you can emulate a subset of these services (and a subset of their features) using open-source software but at what price ?…

Do you even understand his point? He is just nice enough to disclose his current employer. He's not recommending Linode. Really, all he's doing is just asking you to do some research and it needn't be Linode, there are plenty of alternatives out there.

If we keep attacking the people for their honest disclosures, we're only discouraging them from expressing their conflict of interest in the future. Please. Don't.

Re: AWS mistakes to avoid

#173

Post by comex »

bobbyi_settv wrote: post #126
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Linode offers the equivalent of a tiny, tiny subset of AWS. I haven't looked recently, but do they have something equivalent to SQS? Autoscaling groups? DynamoDB? SES? Cloudwatch? If you are going to end up re-building AWS's services from scratch on top of what are essentially naked linux machines (or doing your integration work between services from many vendors), you are doing a huge amount of work that isn't neces…

$20k/month is $240k/year, which is at least two developer salaries. Do you really need two full time devops/whatever to manage the open source equivalents to the AWS services?

Aren't there tools that can provide a similarly high level/seamless/polished experience running on your own computer, using your choice of cloud provider as the backend? If not, why not?

(I wouldn't know - I don't even run a website, and if I did, I would avoid the locked-in services like the plague, out of principle. But I'm curious.)

Re: AWS mistakes to avoid

#174

Post by windowsworkstoo »

EGreg wrote: post #154
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

We started with Linode. We moved to AWS. Here's the main reason: you can't decouple disk space from RAM. So we paid thru the nose just to get more disk space. With AWS it's pay for what you need. That's the killer feature for us.

Sure but you could have also just moved to SoftLayer or any number of dedicated hardware providers for the same, cheap probably cheaper, result.

Re: AWS mistakes to avoid

#175

Post by querulous »

nzoschke wrote: post #88
querulous wrote: post #72

Earlier quoted context omitted.

how does convox scale when moving to non-trivial aws scenarios? if i have three autoscale groups running seven different apps with associated kinesis streams, s3 buckets, elasticache, rds and elasticsearch instances how cleanly does convox handle this? what about monitoring and reporting?

We are working to normalize the AWS scenario for all apps. Every Convox cluster is an autoscale group managed cluster of ECS instances. Every Convox app gets its own Kinesis stream for logs, ELB for load balancing, and S3 buckets for settings, build artifacts and encrypted environment. And the app processes are run via ECS. So I'm confident we could handle the 7 apps in a single cluster, scale the cluster instance si…

i appreciate you taking the time to reply, thanks.

some additional questions though:

1. are security groups opaque within convox or are they exposed to developers?

2. when you say monitoring tools are built in and you have tools for logging does this lock me in to the convox log pipeline and monitoring? what i want to use sensu on my instances? do i have to add sensu to every container? if you run, for instance, five containers per vm do i pay the overhead of five seperate sensu instances? same question for something like logstash?

3. you mention vpc. aws has proven stingy with vpc service limit requests in the past. i have trouble getting them to grant more than low double digits per region/account. can i run multiple convox racks per vpc or is the one vpc per rack a hard requirement?

Re: AWS mistakes to avoid

#176

Post by LoSboccacc »

euphemize wrote: post #40
zerotosixty wrote: post #7

You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…

Great point. Also, the pipeline you mention removes many risks related to the 'common' mistakes mentioned in the article (scaling, monitoring, provisioning it all done for you - or almost). We're using gateway -> lambda -> dynamodb, and there are a tons of gotchas and small things that AWS need to iron out, especially with gateway -> lambda.

Agreed. Was looking into that for a serverless model inspired by (1) but there is still loads of things they can't do/won't do

1 https://github.com/serverless/serverless

Re: AWS mistakes to avoid

#177

Post by Sir_Cmpwn »

nicobn wrote: post #149
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Since you're dealing in absolutes, let me return the favor; you're being a little silly. Yes, Linode beats AWS when it comes to price. On the other hand, Linode's offer is incredibly basic and simplistic. AWS offers service after service that Linode simply doesn't and realistically cannot. Sure, you can emulate a subset of these services (and a subset of their features) using open-source software but at what price ?…

I'm really not trying to compare Linode to AWS here. I just felt that it was necessary to mention that I work for Linode because there _is_ some overlap. I'm really also not trying to sell anyone on Linode in particular. I'm pointing out that from my own personal experience, AWS is grossly overpriced and worse than the alternatives for a large number of use-cases.

Re: AWS mistakes to avoid

#178

Post by eropple »

jacques_chester wrote: post #171
eropple wrote: post #49

Earlier quoted context omitted.

Terraform, as an idea, is brilliant. Mitchell and company isolated a hugely important need and tried to fill it, and I give them all the credit in the world for that. Cross-platform cloud provisioning? Gimme. But I cannot in good conscience not relate what a disastrous experience Terraform has been for me at both jobs and clients. Writing reusable code in Terraform is an exercise in frustration due to the extreme clu…

If you're writing your own, you might also look to BOSH[1] for inspiration. It's older than CloudFormation and Terraform (born 2010). It can manage anything that someone's written a driver for. So far that includes AWS, Azure, vSphere/vCloud, OpenStack, VirtualBox, Google Compute Engine, Apache CloudStack and there might be others I missed. It stores state in a database. It is able to recover from mismatches between…

Augh, how did bosh slip my mind? I've never used it in production, but I've used it to roll out a CF environment for testing and was impressed to dig into it a little more (most of a year ago now, I think your mention of CF was actually what kicked that off). From (admittedly limited) experience I'm not crazy about its developer-facing feel, but I appreciate the significant and responsible effort in it.

Re: AWS mistakes to avoid

#179

Post by Sir_Cmpwn »

kawsper wrote: post #145
ddevault wrote: post #103

Full disclosure: I work for Linode and I am super biased. This is my own opinion etc etc Perhaps the first AWS mistake you might make is... using AWS? Even before I started at Linode, I thought it was terrible. It's extremely, unreasonably pricey. The UI is terrible. Their offerings are availble elsewhere. I started MediaCrush, a now-defunct media hosting website, on AWS. After a while, we switched to dedicated hosti…

Hi, first of all, thanks for your comment, and your full-disclosure. I am a happy Linode customer, and I have been using your service since 2010-09-02 (that is the date of the first invoice), but I have a few issues with your product that you might be able to look at. - It is too hard to figure out that you offer hourly billing, I found it by accident, and a year too late. - I tried out your Nodebalancers, as I thoug…

Hi there. I wasn't looking to represent Linode tonight (just my personal opinions here), but I'll go ahead and fill you in. I can't say more than "I've made a note" of most of these things, but I wanted to mention that we're looking at overhauling some things that will definitely fix a lot of your gripes. Linode has been around for 13 years now, and a huge part of my job and the reason I was hired is to modernize a lot of the stale parts, including lots of the things that are frustrating you.

Re: AWS mistakes to avoid

#180

Post by LoSboccacc »

BraveNewCurency wrote: post #163
late2part wrote: post #132

Earlier quoted context omitted.

False. Most AWS services are implementations of open source solutions. Once you implement it yourself on Linux, the ongoing cost drops. With AWS you pay more the larger you get.

> Most AWS services are implementations of open source solutions. This is fundamentally not understanding what AWS is. - You could say "EC2 is just Xen". But next time there is a 0-day exploit, I'll have AWS working all weekend to patch my servers. And Xen still doesn't have an API for scaling physical hardware.. - You could say "S3 is just Apache". But I will never see a "disk full" message, I will never get paged i…

I am especially curious to see that 'cheaper if you do it' done for their databases offers.

I'm using them in prod already, and trying to create a similar setup on real hardware for enterprise customers using pg, pgpool2 and haproxy is already a pain, and yet you couldn't autorecover them as conveniently when they go belly up.

Post reply on HN