Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is a billion dollars worth of server lying on the ground?

cerebralab.com

Post reply on HN
Print view View original

251–260 of 338 posts

Re: Is a billion dollars worth of server lying on the ground?

#251

Post by giantrobot »

thu2111 wrote: post #242
giantrobot wrote: post #228

Earlier quoted context omitted.

While infrastructure astronauts are typically money wasters and padding their resumes, I don't think SO is the best counter example. SO's read to write ratio is enormous. While they talk about a handful of different bare metal servers a vast majority of their hits are handled by their caching layer which is not one of their bare metal machines. Their write loads are not explicitly time sensitive either, if there's a…

Well, it's an interactive database backed web site. That describes like maybe 90% of the things running on AWS. StackOverflow is useful because it reminds people that machines are fast, and you probably don't need that many of them to scale to large sizes. StackOverflow is used by the entire global population of developers more or less and it runs off of one large MS SQL Server + some web servers. No auto scaling (no…

You're not wrong but it's worth noting SO as a case study for site design has some important caveats. Beefy hardware is great but aggressive caching and setting same expectations around write latency can be a massive scaling win.

Re: Is a billion dollars worth of server lying on the ground?

#252

Post by rcxdude »

mwcampbell wrote: post #250
thomaslord wrote: post #215

Earlier quoted context omitted.

> If your team is filled with devops engineers, then, sure, go with the non-AWS route. This seems backwards to me - running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Setting the same thing up on AWS requires slogging through the d…

> running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Until that server goes down.

I'd say the risks of a server going down are about the same as an AWS setup going haywire at that level.

Re: Is a billion dollars worth of server lying on the ground?

#253

Post by throwaway894345 »

thu2111 wrote: post #248
throwaway894345 wrote: post #197

Earlier quoted context omitted.

Cool, this is the stuff that interests me. Comparing the nominal cost of hardware with the nominal AWS costs is boring.

Why? If the primary benefit you're getting from the cloud is VMs, it's valid to compare to hardware. The overhead of bare metal on top of cloud VMs is basically knowing how to handle RAID and set up load balancing.

VMs are certainly not the primary benefit of the cloud; cloud providers offer many services in addition to VM orchestration, but that hardly matters because your own example is illustrative of your error:

> The overhead of bare metal on top of cloud VMs is basically knowing how to handle RAID and set up load balancing.

This implies human beings to handle raid and set up load balancing, which suggests that you need to compare the cost of cloud providers with the cost of hardware and the cost of those engineering resources.

In addition to RAID and load balancing, most organizations/applications also need networking (good luck balancing load without a network), databases (including backup management), TLS, DNS, access management, etc, etc. All of this takes humans to build and operate. AWS services do a lot of this for you. In the on-prem world, you have to build (or buy/integrate) this yourself, but that's not free so you have to account for that cost in your comparison.

You can still make the argument that on-prem is a better value proposition when accounting for the total cost of ownership, but that's a different argument than those which ignore engineering costs altogether.

Re: Is a billion dollars worth of server lying on the ground?

#254

Post by eli »

mwcampbell wrote: post #250
thomaslord wrote: post #215

Earlier quoted context omitted.

> If your team is filled with devops engineers, then, sure, go with the non-AWS route. This seems backwards to me - running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Setting the same thing up on AWS requires slogging through the d…

> running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Until that server goes down.

Sure but sometimes that’s ok. Everything goes down sometimes. You need to figure out how much effort you’re willing to expend for each extra “9”.

I wonder what percentage of all sites hosted at AWS are actually ready for a zone to fail.

Re: Is a billion dollars worth of server lying on the ground?

#255

Post by foxtr0t »

viraptor wrote: post #10

He kind of touches on this in point IV, but never mentioned specifically: the cost comparison is not AWS vs ovh. It's fragment-of-EC2 vs ovh. If you need a predefined small number of VMs and no other functionality, it would be silly to go with AWS. But on the other hand, if you want a set of servers of a given class spawning on demand, with traffic coming in via load balancers, with integrated certificate and DNS man…

This article may be poorly written, but if you actually read through the whole thing, it makes some brilliant points:

1. IaaS providers are incentivized to create services that lure you with managed solutions that seem like a great deal on paper, while they are actually more expensive to operate than their self-rolled alternative.

2. "DevOps" and "Infra" people charged with making these decisions often follow the industry zeitgeist, which for the moment is whatever AWS, GCP, and Azure are offering. This is in spite of the best interests of the company they work for. The right decision in so many cases is what choice one can defend when something goes wrong, or what you can parlay into the next gig. If costs, performance, and features aren't closely managed, scrutinized, compared, going with a solution like Aurora/Redshift/Dynamo won't be challenged.

3. Nothing is easily comparable because these services aren't just a CPU as you mention, instead they're hardware and software rolled into one. This is intentional, and has little to do with making things easier for you. At best, IaaS providers defend this as "differentiation" but it's closer to obfuscation of true cost. Go ahead and ask your AWS rep if use case x, y, and z will perform well on lambdas and you'll likely get a near canned response that they all fit perfectly, even if y is a job with high load spread uniformly through time. The only way you can make this comparison is by creating the two services, standing them up in production for a week and checking the bill. In other cases such as FPGAs and databases, you'll have a much harder time as not only is the software/hardware offering unique, but the user code will be completely different and require entire rewrites to get what _might_ be a fair comparison.

Re: Is a billion dollars worth of server lying on the ground?

#256

Post by annoyingnoob »

pontifier wrote: post #231

Personal observation: I purchased a company that was locked into cloud infrastructure. They had received over $10M in venture capital several years ago, and had been running their own servers at some point because I found some of them in their warehouse. At some point they had switched to AWS, and then at a later point switched to GC. When I encountered them they had shut down due to lack of funding. Their servers we…

> had been running their own servers at some point

Wonder why they stopped?

Re: Is a billion dollars worth of server lying on the ground?

#257

Post by romanoderoma »

an_opabinia wrote: post #141
viraptor wrote: post #10

He kind of touches on this in point IV, but never mentioned specifically: the cost comparison is not AWS vs ovh. It's fragment-of-EC2 vs ovh. If you need a predefined small number of VMs and no other functionality, it would be silly to go with AWS. But on the other hand, if you want a set of servers of a given class spawning on demand, with traffic coming in via load balancers, with integrated certificate and DNS man…

Sounds like you're listing the enterprise features people pay for. If Amazon sells it so cheaply why don't enterprises go direct to Amazon instead of paying a SaaS provider like you? It's about economics not accounting. Once you get into cost comparisons you've already lost. AWS is purpose built for an accounting narrative. To illustrate what a dead end accounting is, accounting can't explain things like, "I spend mo…

Don't know why you are being downvoted, but I agree with your reasoning.

I've worked most of my career in places where technology was king and AWS is certainly technologically ahead of its competitors.

Fast forward a couple decades and I ended up in financial institutions where economics is king and they don't use AWS, not as much as they should, if you listen to the average internet guru.

Not because AWS doesn't do what they say it does, but because economically it's not advantageous, at least until it is.

Which is not at the beginning.

To put it in simple words: they don't ask "are we ready to scale" but instead "are we making money out of this".

Re: Is a billion dollars worth of server lying on the ground?

#258

Post by smolder »

throwaway894345 wrote: post #162
smolder wrote: post #132

Earlier quoted context omitted.

> At no cost That's so ridiculous I'm not sure how to reply or even interpret your comment.

I can't tell if your comment is a particularly rude way of picking the nit that "nothing is free, the cost (e.g., of IAM) is built into other services" or if you really find it absurd that the nominal price of many AWS services is $0 or something else entirely.

I apologize for my rudeness. I wasn't adding anything with that comment. I did find the parent comment jarring, as if it was willfully ignoring the downsides in an effort to promote AWS. That's in a comment thread that otherwise seemed like a candid discussion of the pros, the cons, and how it's a good deal for some but a worse deal for others.

Of course, the cost is built into other services. As pointed out, AWS can get very expensive for many use cases, and that's exactly what you're paying for: access to managed services.

Beyond that by developing on AWS you are taking steps to lock yourself into using their system - your configuration isn't portable to other services. So the time/manpower you spend configuring AWS-specific things is another cost associated exclusively with its use.

Re: Is a billion dollars worth of server lying on the ground?

#259

Post by Aeolun »

foxtr0t wrote: post #255
viraptor wrote: post #10

He kind of touches on this in point IV, but never mentioned specifically: the cost comparison is not AWS vs ovh. It's fragment-of-EC2 vs ovh. If you need a predefined small number of VMs and no other functionality, it would be silly to go with AWS. But on the other hand, if you want a set of servers of a given class spawning on demand, with traffic coming in via load balancers, with integrated certificate and DNS man…

This article may be poorly written, but if you actually read through the whole thing, it makes some brilliant points: 1. IaaS providers are incentivized to create services that lure you with managed solutions that seem like a great deal on paper, while they are actually more expensive to operate than their self-rolled alternative. 2. "DevOps" and "Infra" people charged with making these decisions often follow the ind…

Anecdotally, my company spends several millions a year on AWS, and they do it mostly so they don’t have to think too much about the hardware every team uses to implement their solutions. I’d say the strategy is an unmitigated success, even if it costs them 3 times more than a similar on-premise (or dedicated) solution would cost.

My last company went the dedicated route, and they were perpetually in need of more servers because nobody was willing to pre-plan capacity. To the point that the database team just couldn’t provision any new databases for a year.

Re: Is a billion dollars worth of server lying on the ground?

#260

Post by hw »

drchopchop wrote: post #129
hw wrote: post #115

Earlier quoted context omitted.

> AWS gives you all the things you'd need to scale, without heavy up-front costs A startup doesn't need AWS right off the bat. Planning for scale right from the beginning is a way to quickly bleed $. Of course, if you have VC money, why not spend that cash right? Where I work we've started non-AWS and have continued non-AWS. We don't have a team of devops engineers, but rather a team where there are a engineers who _…

You don't necessarily need to plan from scale from the start. Often you just need a few servers, a load balancer, a database, some file storage, internal networking, user permissions, and some level of security/firewall. That is very easy to set up on AWS in a day or two, and you don't need all your engineers to simultaneously be devops experts. The scale can happen once you've validated your startup, and when that h…

There are some merits to AWS, I can agree with that, and there comes a point where a startup outgrows baremetal / cookie cutter VPS-es, but I disagree with "AWS is trivial" and that it takes a day or two to get things set up. For basic set up like getting a few servers done behind a load balancer and a database - sure - and I'd argue that services like DigitalOcean and Linode are actually easier to set up than AWS for basic services.

To actually do more advanced stuff (the thing that AWS is good for) and utilize tools such as Terraform, you'd essentially need to hire engineers that are experts in AWS in addition to engineers who can do devops, as there's only so much "magic" AWS can provide.

Post reply on HN