Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The many lies about reducing complexity part 2: Cloud

ea.rna.nl

Post reply on HN
Print view View original

51–60 of 131 posts

Re: The many lies about reducing complexity part 2: Cloud

#51

Post by jsiepkes »

jbmsf wrote: post #25
maria_weber23 wrote: post #19

Earlier quoted context omitted.

I second that. It's not only that you make yourself completely intertwined with a Cloud by using more than fundamental services. The costs of lambda or even DDB are IMMENSE. These only pay off for services that have a high return per request. I.e. if you get a lot of value out of lambda calls, sure, use them. But for anything high-frequency that earns you little to nothing on its own, forget about it. Generally all y…

I can't agree, at least not in general. The optionality of being cloud agnostic comes with a huge cost, both because of all the pieces you have to build+operate and because of the functionality you have to exclude from your systems. I am sure there are scales where you either have such a large engineering budget that you can ignore these costs or where decreasing your cloud spend is the only way to scale your busines…

> But for the average company, I can't see how spending so much on infrastructure (and future optionality) pays off, especially when you could spend on product or marketing or anything else that has a more direct impact on your success.

If you change "average company" to "average startup" then your point make sense. But for a normal company not everything needs to make a direct impact on your success. For example guaranteeing long term business continuity is an important factor too.

Re: The many lies about reducing complexity part 2: Cloud

#52

Post by mumblemumble »

bob1029 wrote: post #6

This shared responsibility principle that underlies cloud marketing speak sounds a lot like the self-driving mess we find ourselves in today - I.e. the responsibility boundary between parties exists in a fog of war and results in more exceptions than if one or the other were totally responsible. We have been a customer of Amazon AWS for ~6 years now, and we still really only use ~3 of their products: EC2, Route53 and…

I recently inherited a product that was developed from the ground up on AWS. It's been a real eye opener.

Yes, it absolutely is locked in, and will never run on anything but AWS. That doesn't surprise me. What surprises me is all of the unnecessary complexity. It's one big Rube Goldberg contraption, stringing together different AWS products, with a great deal of "tool in search of problem" syndrome for good measure. I am pretty sure that, in at least a few spots, the glue code used to plug into Amazon XYZ amounted to a greater development and maintenance burden than a homegrown module for solving the same problem would have been.

NIH syndrome is certainly not any fun. But IH syndrome seems to be no better.

Re: The many lies about reducing complexity part 2: Cloud

#53

Post by tidepod12 »

p_l wrote: post #28
jasode wrote: post #24

Earlier quoted context omitted.

>I would also be interested to hear the other side of the coin. Who out there is using 20+ AWS/Azure/GCP products to back a single business app and is having a fantastic time of it? Netflix uses a lot of AWS higher-level services beyond the basics of EC2 + S3. Netflix definitely doesn't restrict its use of AWS to only be a "dumb data center". Across various tech presentations by Netflix engineers, I count at least 17…

Certain services IMHO have to be discounted from this list: - VPC - basic building block for any AWS-based infra that isn't ancient - CloudTrail - only way to get audit logs out of AWS, no matter what you feed them into - CloudWatch - similar with CloudTrail, many things (but not all) will log to CloudWatch, and if you use your own log infra you'll have to pull from it. Also necessary for metrics. - ELB/ELBv2/NLB/ALB…

I don't think you can discount them like that. Maybe they aren't as front of mind as services like S3, EC2, etc, but if you were to try to rebuild your setup in a personal data center, replacing the capabilities of VPC, IAM, CloudTrail, NAT gateways, ELBs, KMS etc would be a huge effort on your part. The fact that they are "basic building blocks" makes them more important, not less. In a discussion about the complexity of cloud providers versus other setups, that seems especially relevant.

Re: The many lies about reducing complexity part 2: Cloud

#54

Post by fiddlerwoaroof »

p_l wrote: post #28
jasode wrote: post #24

Earlier quoted context omitted.

>I would also be interested to hear the other side of the coin. Who out there is using 20+ AWS/Azure/GCP products to back a single business app and is having a fantastic time of it? Netflix uses a lot of AWS higher-level services beyond the basics of EC2 + S3. Netflix definitely doesn't restrict its use of AWS to only be a "dumb data center". Across various tech presentations by Netflix engineers, I count at least 17…

Certain services IMHO have to be discounted from this list: - VPC - basic building block for any AWS-based infra that isn't ancient - CloudTrail - only way to get audit logs out of AWS, no matter what you feed them into - CloudWatch - similar with CloudTrail, many things (but not all) will log to CloudWatch, and if you use your own log infra you'll have to pull from it. Also necessary for metrics. - ELB/ELBv2/NLB/ALB…

I mostly agree, although ECS with Fargate is often nicer to use than EC2

Re: The many lies about reducing complexity part 2: Cloud

#55

Post by [deleted] »

mumblemumble wrote: post #52
bob1029 wrote: post #6

This shared responsibility principle that underlies cloud marketing speak sounds a lot like the self-driving mess we find ourselves in today - I.e. the responsibility boundary between parties exists in a fog of war and results in more exceptions than if one or the other were totally responsible. We have been a customer of Amazon AWS for ~6 years now, and we still really only use ~3 of their products: EC2, Route53 and…

I recently inherited a product that was developed from the ground up on AWS. It's been a real eye opener. Yes, it absolutely is locked in, and will never run on anything but AWS. That doesn't surprise me. What surprises me is all of the unnecessary complexity. It's one big Rube Goldberg contraption, stringing together different AWS products, with a great deal of "tool in search of problem" syndrome for good measure.…

[deleted]

Re: The many lies about reducing complexity part 2: Cloud

#56

Post by dkarl »

ehnto wrote: post #36

I don't understand what people are building in order to need half of this decoupled and managed elsewhere anyway. It wasn't all that challenging to self manage it five years ago, what's changed? My guess is that the average small to medium project has drank the enterprise coolaid, and they are suffering the configuration and complexity nightmares that surround managing cloud infrastructure before they really needed t…

I guess it's Kool-Aid? I don't know; I don't remember being lied to when I started using cloud services. I think of cloud resources as being amazing and basically magical, but I know there's a limit to the magic, and the rest is work. People using (for example) AWS S3 should not be surprised that they still have to work to manage the naming, organization, access control, encryption, retention, etc. of their data, and they might encounter problems if they try to load a 100GB S3 object into a byte array in a container provisioned with 1GB of RAM. But they are. I don't know if that's human nature or if they're being lied to by consultants and marketers.

Re: The many lies about reducing complexity part 2: Cloud

#57

Post by mumblemumble »

hyperdimension wrote: post #27
bane wrote: post #17

Earlier quoted context omitted.

I've worked with a number of teams over the last few years who use AWS and I'd say from top to bottom they all build their strategy more or less the same way: 0. Whatever is the minimum needed to get a VPC stood up. 1. EC2 as 90%+ of whatever they're doing 2. S3 for storing lots of stuff and/or crossing VPC boundaries for data ingress/egress (like seriously, S3 seems to be used more as an alternative to SFTP than for…

I'm not very familiar with AWS or The Cloud, but I'm having trouble understanding what you said about Amazon leaving money on the table by not directing customers toward specific-purpose services as opposed to EC2? Wouldn't (for AWS to make a profit anyway) whatever managed service have to be cheaper than some equivalent service running on an EC2 VM? I get the concerns re: pricing and predictability, but it still see…

It's not just about a straight cost comparison. It's about how organizational decision-making works.

The people shopping for products are not spending their own money, but they are spending their own time and energy. The people approving budgets are not considering all possible alternatives, they are only considering the ones that have been presented to them by the people doing the shopping.

If the shoppers decide that an option will cost them too much in time and irritation, then it may be that the people holding the purse-strings are never even made aware that it exists. Even if it is the cheapest option.

Re: The many lies about reducing complexity part 2: Cloud

#58

Post by nucleardog »

hyperdimension wrote: post #27
bane wrote: post #17

Earlier quoted context omitted.

I've worked with a number of teams over the last few years who use AWS and I'd say from top to bottom they all build their strategy more or less the same way: 0. Whatever is the minimum needed to get a VPC stood up. 1. EC2 as 90%+ of whatever they're doing 2. S3 for storing lots of stuff and/or crossing VPC boundaries for data ingress/egress (like seriously, S3 seems to be used more as an alternative to SFTP than for…

I'm not very familiar with AWS or The Cloud, but I'm having trouble understanding what you said about Amazon leaving money on the table by not directing customers toward specific-purpose services as opposed to EC2? Wouldn't (for AWS to make a profit anyway) whatever managed service have to be cheaper than some equivalent service running on an EC2 VM? I get the concerns re: pricing and predictability, but it still see…

No, usually the managed services are a premium over the bare hardware. When you use RDS for example, you’re paying for the compute resources but also paying for the extra functionality they provide and their management and maintenance they’re doing for you. You can run your own Postgres database, or you can pay the premium for Aurora on RDS and get a multi-region setup with point in time restore and one-click scaling and automatically managed storage size and automatic patching and monitoring integrated into AWS monitoring tools and...

They’re leaving money in the table because instead of using “Amazon Managed $X” potentially at a premium or paying a similar amount but in a way where AWS can provide the service with fewer compute resources than you or I would need because of their scale and thus more profitably, people look and see they’ll be paying $0.10/1000 requests and $0.05/1gb of data processed in a query and $0.10/gb for bandwidth for any transfer that leaves the region and... people just give up and go “I have no idea what that will cost or whether I can afford it, but this EC2 instance is $150/mo, I can afford that.”

Re: The many lies about reducing complexity part 2: Cloud

#59

Post by [deleted] »

bob1029 wrote: post #6

This shared responsibility principle that underlies cloud marketing speak sounds a lot like the self-driving mess we find ourselves in today - I.e. the responsibility boundary between parties exists in a fog of war and results in more exceptions than if one or the other were totally responsible. We have been a customer of Amazon AWS for ~6 years now, and we still really only use ~3 of their products: EC2, Route53 and…

[deleted]

Re: The many lies about reducing complexity part 2: Cloud

#60

Post by zoomablemind »

Subjectively, it increasingly feels that while the complexity has been increasing, the notion of longevity of the underlying products and services has been degrading.

While updates to software were expected, general outlook would be that they would not be breaking the core features. The emphasis on backwards compatibility was in a way an assurance to businesses that building their operations on vendor's products is not risky. Even then, some mission-critical elements would be defensively abstracted to avoid the dependency risks (at least theoretically...)

Now, we all witness the "eternal-beta" paradigm across the most of the major software products. Frequent builds with automatic updates, when new features could be suddenly pushed, and old features removed.

Sure, it's still possible to spec out a "hard-rock" steady platform, postpone updates, abstract dependencies and just focus on business. But...such approach won't be approved, as it's widely acknowledged that the presence of critical bugs is rather a "feature" of all software. Postponing the updates is not prudent, it's a liability.

So the rock-solid expectations are just an illusion or perhaps a fantasy promoted widely, just to get the foot in the door.

Ironically, the most stable elements are the so much dreaded "legacy", too often in charge of the business-critical logic.

Post reply on HN