Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The many lies about reducing complexity part 2: Cloud

ea.rna.nl

Post reply on HN
Print view View original

61–70 of 131 posts

Re: The many lies about reducing complexity part 2: Cloud

#61

Post by orlovs »

ratww wrote: post #47
skohan wrote: post #42

Earlier quoted context omitted.

Yeah I certainly agree that the complexity doesn't really go away completely, and sometimes it's much more frustrating to have to configure poorly documented services rather than just having access to the host OS. I guess my overall point would be that two of the hardest things to do in terms of making a production-ready application are scaling and security, and Serverless pretty much obviates them. So it's not a mag…

Yes, I agree with that point. I think my point was more that Serverless is a good idea, but the current implementations are still not good at removing complexity. But I can see this easily changing, with open standards and the such.

Well, we just need to admit that running applications if ack all know risks are complex. If we blissfully ignore risks like lamp or lemp stack its much more easier. Main question do we need to take in account most of risks, running within small scale.

Re: The many lies about reducing complexity part 2: Cloud

#62

Post by nucleardog »

jsiepkes wrote: post #51
jbmsf wrote: post #25

Earlier quoted context omitted.

I can't agree, at least not in general. The optionality of being cloud agnostic comes with a huge cost, both because of all the pieces you have to build+operate and because of the functionality you have to exclude from your systems. I am sure there are scales where you either have such a large engineering budget that you can ignore these costs or where decreasing your cloud spend is the only way to scale your busines…

> But for the average company, I can't see how spending so much on infrastructure (and future optionality) pays off, especially when you could spend on product or marketing or anything else that has a more direct impact on your success. If you change "average company" to "average startup" then your point make sense. But for a normal company not everything needs to make a direct impact on your success. For example gua…

Unless you’re planning for the possibility of AWS dropping offline permanently with little to no notice, it really feels like you’re just paying a huge insurance premium. Like any insurance, it’s down to whether you need insurance or could cover the loss. Whether you’d rather incur a smaller ongoing cost to avoid the possibility of a large one time loss.

If AWS suddenly raised their prices 10x overnight, it would hurt but not be an existential threat for most companies. At that point they could invest six months or a year into migrating off of AWS.

Rough numbers that would end up costing us like $4m in cloud spend and staff if we retasked the entire org to accomplishing that for a year.

There’s certainly an opportunity cost as well, but I’d argue it’s not dissimilar to the opportunity cost we’d have been paying all along to maintain compatibility with multiple clouds.

Obviously it’s just conjecture, but my gut says the increased velocity of working on a single cloud and using existing Amazon services and tools where appropriate has made us significantly more than the costs of something that may never happen.

Re: The many lies about reducing complexity part 2: Cloud

#63

Post by yls »

singron wrote: post #34
maria_weber23 wrote: post #19

Earlier quoted context omitted.

I second that. It's not only that you make yourself completely intertwined with a Cloud by using more than fundamental services. The costs of lambda or even DDB are IMMENSE. These only pay off for services that have a high return per request. I.e. if you get a lot of value out of lambda calls, sure, use them. But for anything high-frequency that earns you little to nothing on its own, forget about it. Generally all y…

Do you have examples of Google Cloud arbitrarily changing prices and terminating products? Sure they terminate consumer products, and there was a Maps price hike, but I'm not aware of anything that's part of Cloud.

IIRC they introduced a cluster management fee in GKE.

Re: The many lies about reducing complexity part 2: Cloud

#64

Post by jbmsf »

jsiepkes wrote: post #51
jbmsf wrote: post #25

Earlier quoted context omitted.

I can't agree, at least not in general. The optionality of being cloud agnostic comes with a huge cost, both because of all the pieces you have to build+operate and because of the functionality you have to exclude from your systems. I am sure there are scales where you either have such a large engineering budget that you can ignore these costs or where decreasing your cloud spend is the only way to scale your busines…

> But for the average company, I can't see how spending so much on infrastructure (and future optionality) pays off, especially when you could spend on product or marketing or anything else that has a more direct impact on your success. If you change "average company" to "average startup" then your point make sense. But for a normal company not everything needs to make a direct impact on your success. For example gua…

I take your point, but I still don't quite agree.

There are obviously plenty of companies that are willing to couple themselves to a single cloud vendor (e.g. Netflix with AWS) and plenty of business continuity risks that companies don't find cost effective to pursue. Has anyone been as vocal about decoupling from CRM or ERB systems as they are with cloud?

My own view is that these kinds of infrastructure projects create as many risks as the solve and happen at least as much because engineers like to solve these kinds of problems than for any other reason.

Re: The many lies about reducing complexity part 2: Cloud

#65

Post by jbmsf »

nucleardog wrote: post #62
jsiepkes wrote: post #51

Earlier quoted context omitted.

> But for the average company, I can't see how spending so much on infrastructure (and future optionality) pays off, especially when you could spend on product or marketing or anything else that has a more direct impact on your success. If you change "average company" to "average startup" then your point make sense. But for a normal company not everything needs to make a direct impact on your success. For example gua…

Unless you’re planning for the possibility of AWS dropping offline permanently with little to no notice, it really feels like you’re just paying a huge insurance premium. Like any insurance, it’s down to whether you need insurance or could cover the loss. Whether you’d rather incur a smaller ongoing cost to avoid the possibility of a large one time loss. If AWS suddenly raised their prices 10x overnight, it would hur…

Strong agree.

Plus I've seen more than a few efforts at multi-cloud that resulted in a strong dependency on all clouds vs the ability to switch between them. So not only do you not get to use cloud-specific services, you don't really get any benefit in terms of decoupling.

Re: The many lies about reducing complexity part 2: Cloud

#66

Post by ma2rten »

singron wrote: post #34
maria_weber23 wrote: post #19

Earlier quoted context omitted.

I second that. It's not only that you make yourself completely intertwined with a Cloud by using more than fundamental services. The costs of lambda or even DDB are IMMENSE. These only pay off for services that have a high return per request. I.e. if you get a lot of value out of lambda calls, sure, use them. But for anything high-frequency that earns you little to nothing on its own, forget about it. Generally all y…

Do you have examples of Google Cloud arbitrarily changing prices and terminating products? Sure they terminate consumer products, and there was a Maps price hike, but I'm not aware of anything that's part of Cloud.

A very long time ago App engine went out of beta and there was a price hike leaving many scrambling. App engine was in beta so long that many people didn’t think that label meant anything.

Re: The many lies about reducing complexity part 2: Cloud

#67

Post by bane »

hyperdimension wrote: post #27
bane wrote: post #17

Earlier quoted context omitted.

I've worked with a number of teams over the last few years who use AWS and I'd say from top to bottom they all build their strategy more or less the same way: 0. Whatever is the minimum needed to get a VPC stood up. 1. EC2 as 90%+ of whatever they're doing 2. S3 for storing lots of stuff and/or crossing VPC boundaries for data ingress/egress (like seriously, S3 seems to be used more as an alternative to SFTP than for…

I'm not very familiar with AWS or The Cloud, but I'm having trouble understanding what you said about Amazon leaving money on the table by not directing customers toward specific-purpose services as opposed to EC2? Wouldn't (for AWS to make a profit anyway) whatever managed service have to be cheaper than some equivalent service running on an EC2 VM? I get the concerns re: pricing and predictability, but it still see…

Yeah good question. Sibling comments to this one explain it well, but basically AWS managed services come at a premium price over some equivalent running in just EC2. (Some services in fact do charge you for EC2 time + the service + storage etc.)

"Managed" usually means "pay us more in exchange for less work on your part". This is usually pitched as a way to reduce admin/infrastructure/devops type staff and the overhead that goes along with having those people on the payroll.

Re: The many lies about reducing complexity part 2: Cloud

#68

Post by NicoJuicy »

the-smug-one wrote: post #9
sanp wrote: post #4

Earlier quoted context omitted.

There is an element of _how_ as well. You could create simple monoliths or overengineered microservices. Or, complex monoliths with heavy coupling vs cleanly designed microservices with clear separations of concern.

Are microservices meant to separate data too? As in, each service has its own database. Wouldn't that lead to non-normalisation of the data or a lot of expensive network lookups to get what I want/need? What is the point of micro services anyway :-)?

Microservices is a solution for an organisational problem ( multiple employees in one project), not a technical one.

If you're flying solo, just use DDD for example. It will give you the same patterns without the devops complexity

Re: The many lies about reducing complexity part 2: Cloud

#69

Post by glogla »

hyperdimension wrote: post #27
bane wrote: post #17

Earlier quoted context omitted.

I've worked with a number of teams over the last few years who use AWS and I'd say from top to bottom they all build their strategy more or less the same way: 0. Whatever is the minimum needed to get a VPC stood up. 1. EC2 as 90%+ of whatever they're doing 2. S3 for storing lots of stuff and/or crossing VPC boundaries for data ingress/egress (like seriously, S3 seems to be used more as an alternative to SFTP than for…

I'm not very familiar with AWS or The Cloud, but I'm having trouble understanding what you said about Amazon leaving money on the table by not directing customers toward specific-purpose services as opposed to EC2? Wouldn't (for AWS to make a profit anyway) whatever managed service have to be cheaper than some equivalent service running on an EC2 VM? I get the concerns re: pricing and predictability, but it still see…

For example:

Managed Airflow Scheduler on AWS with "large" size costs $0.99/hour, or $8,672/year per instance. That's ~ $17,500 considering Airflow for at least non-prod and prod instances.

Building it on your own on same size EC2 instance would cost $3,363/year for the EC2. Times two for two environments, let's say $6,700. $4,000 if you prepay the instance.

That looks way cheaper, but then you have to do the engineering and the operational support yourself.

If you consider just the engineering and assume engineer costs $50/hour and estimate this to initial three weeks of work and then 2.5 days / month for support (upgrades, tuning, ...) that's extra $4,000 upfront and $1,000/month.

So on AWS you're at $17,500/year and on-prem you're at best $20,000 first year and $16,000 next years.

So the AWS only comes a bit more expensive - but the math is tricky on several parts:

- maybe you need 4 environments deployed instead of 2, which is more for AWS but not much more for engineering?

- maybe there's less sustaining cost because you're ok with upgrading Airflow only once a quarter?

- you probably already pay the engineers, so it's not an extra money cost, it's extra cost of them not working on other stuff - different boxes and budgets

- maybe you're in part of a world where good devops engineer doesn't cost $50/hour but $15 hour

- I'm ignoring cost of operational support, which can be a lot for on-prem if you need 24/7

- maybe you need 12+ Airflow instances thanks to your fragmented / federated IT and can share the engineering cost

- etc, etc.

So I think what OP was saying is that if AWS priced Managed Airflow at $0.5 per hour, it would be no brainer to use instead of build your own. The way it is, some customers will surely for their own Airflow instead, because the math favors it.

Does that make sense?

Re: The many lies about reducing complexity part 2: Cloud

#70

Post by miscaccount »

yls wrote: post #63
singron wrote: post #34

Earlier quoted context omitted.

Do you have examples of Google Cloud arbitrarily changing prices and terminating products? Sure they terminate consumer products, and there was a Maps price hike, but I'm not aware of anything that's part of Cloud.

IIRC they introduced a cluster management fee in GKE.

not much 10 cents per hour https://www.reddit.com/r/kubernetes/comments/fdgblk/google_g...
Post reply on HN