Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is a billion dollars worth of server lying on the ground?

cerebralab.com

Post reply on HN
Print view View original

191–200 of 338 posts

Re: Is a billion dollars worth of server lying on the ground?

#191

Post by throwaway894345 »

dzonga wrote: post #184
throwaway894345 wrote: post #156

Earlier quoted context omitted.

I would guess SO had a relatively small number of employees and their servers were running a pretty straight-forward CRUD app and a database. Comparing that to the heterogeneous workloads that large organizations deal with is a bit silly. No doubt there's still a lot of fat to trim in those large organizations, but trimming that fat is rarely their largest opportunity (much to the chagrin of those of us who like simp…

my guy, you literally made me laugh. how's this https://stackexchange.com/performance different from any so called workloads large organizations are running. you've your app servers, db servers, load balancers and failover servers. pretty standard setup. yet SO is running on bare-metal. resume driven development and everyone thinking they're google | fb has killed and made money in our industry

> how's this https://stackexchange.com/performance different from any so called workloads

StackExchange is largely a CRUD app. High volume of tiny requests that hit the app layer, then the database, and back. Other organizations have lower volumes of compute-intensive requests, async tasks, etc.

With respect to the size of an organization, the cost of coordinating deployments and maintenance over a handful of servers grows with the size of the organization. It frequently behooves larger organizations to allow dev teams to operate their own services.

None of this is to say that there isn't waste or poor decisions throughout our industry; only that it's not the sole factor and SO's isn't the ideal architecture for all applications.

> my guy

I'm not your guy.

Re: Is a billion dollars worth of server lying on the ground?

#192

Post by shuntress »

throwaway1777 wrote: post #190
hinkley wrote: post #175

I wonder how many billions of dollars worth of unsold cars are just 'lying around'. How many billions of dollars worth of fire trucks are parked right now doing nothing? Given redundancy and inventory logistics, once your industry gets to some number of billions of dollars of equipment, you're goddamn right there's going to be a billion dollars worth of equipment lying around. It's not the magnitude, it's the ratio t…

Definitely need to consider spikes in demand too. A perfect example of this was covid 19 supply shocks. Supply chains got a lot better at just in time delivery and didn’t keep much back stock, but with covid this broke down when everyone went out shopping to stock up for lockdown and work from home. Edit: s/panic shopping/shopping/

It's not "panic shopping" if overnight I transition from eating half my meals at work to eating all my meals at home.

Re: Is a billion dollars worth of server lying on the ground?

#193

Post by kuon »

manquer wrote: post #84
sgt wrote: post #63

Earlier quoted context omitted.

I like the idea, but remember that the SLA with your fiber provider doesn't mean your fiber magically will come back within a few hours when someone in the street cuts through the cable by accident. Your clients will still experience downtime unless you have two fiber lines going in different directions - completely redundant. If you don't have that, I would recommend some kind of a licensed microwave link on your ro…

You always have a DR offsite, even if you are hosting on Big Cloud that would be sensible to setup. Your RTO and RPO needs will dictate your DR setup in both scenarios. Most apps can take few hours hit if the alternative is spending 2-3x .

Yes, and we also have off site replication using ZFS send/recv. Which would let us restore everything in a few hours in case of major disaster.

Re: Is a billion dollars worth of server lying on the ground?

#194

Post by throwaway894345 »

freeone3000 wrote: post #186
throwaway894345 wrote: post #162

Earlier quoted context omitted.

I can't tell if your comment is a particularly rude way of picking the nit that "nothing is free, the cost (e.g., of IAM) is built into other services" or if you really find it absurd that the nominal price of many AWS services is $0 or something else entirely.

The nominal price of $0 absolutely is absurd, since every individual service has a separate pricing chart down to the ELB.

I can't find a pricing chart for IAM, which is the cited example.

Re: Is a billion dollars worth of server lying on the ground?

#195

Post by kuon »

wrkronmiller wrote: post #25
kuon wrote: post #13

With the price of pro fiber (redundant with SLA) I recently moved some apps back to our own servers in house. This did cut the price down dramatically. I would not recommend it for super critical apps (except if you have your own state of the art data center, but I am not speaking about that), but having 5-10 servers in a secure cabinet will give you infinite flexibility for little cost. We pay around 5000$/y to keep…

How do you deal with data egress costs w/r/t AWS though?

We don't use AWS.

Re: Is a billion dollars worth of server lying on the ground?

#196

Post by throwaway1777 »

shuntress wrote: post #192
throwaway1777 wrote: post #190

Earlier quoted context omitted.

Definitely need to consider spikes in demand too. A perfect example of this was covid 19 supply shocks. Supply chains got a lot better at just in time delivery and didn’t keep much back stock, but with covid this broke down when everyone went out shopping to stock up for lockdown and work from home. Edit: s/panic shopping/shopping/

It's not "panic shopping" if overnight I transition from eating half my meals at work to eating all my meals at home.

Ok, edited.

Re: Is a billion dollars worth of server lying on the ground?

#197

Post by throwaway894345 »

vidarh wrote: post #185
throwaway894345 wrote: post #158

Earlier quoted context omitted.

It's such a meme on this site to compare costs of AWS to hardware costs as though the hardware operates itself. I'm sure there are many cases in which onprem actually is a better value proposition than AWS, and I would find it much more interesting if we talked about when to go with AWS vs onprem, but instead we try to make these silly comparisons about the upfront cost of a server blade vs an EC2 instance.

I've done operations stuff for ~25 years now. I've used AWS since it launched. I've done devops consulting for years. I've yet to see an AWS deployment be competitive on cost with onprem or managed hosting. In fact, I used to make good money helping people cut costs by moving off AWS. That includes paying more for devops , because AWS is incredibly complicated, and most people struggle to get it set up well. There ar…

Cool, this is the stuff that interests me. Comparing the nominal cost of hardware with the nominal AWS costs is boring.

Re: Is a billion dollars worth of server lying on the ground?

#198

Post by zootboy »

ohazi wrote: post #180

If I get a three-year reserved instance of a t3.small, it works out to $6/month. A t3.micro works out to $3/mo. Storage and bandwidth, for my use-cases, amount to less than a dollar a month. Are there any other providers that compete on the low end like this? The common suggestions (DO, Linode, etc.) all cost at least twice as much. I buy the author's argument for "real" use-cases where you need one or more expensive…

There's plenty of small VPS providers out there who will give you slivers of a server for very cheap:

https://lowendbox.com/

Re: Is a billion dollars worth of server lying on the ground?

#199

Post by vidarh »

jrochkind1 wrote: post #169
user5994461 wrote: post #95

Earlier quoted context omitted.

Heroku is both incredibly cheap and incredibly expensive. It's only $7/month to deploy a web application. More if you want some of the paid features and a database instance. It all works out of the box with instant deployment out of the box, it's fantastic. Then it suddenly goes $50 more per gigabyte of RAM, which makes it massively expensive for any serious workload. It's crazy hot much they can try to charge, makes…

If it saves you a FTE or two from managing your own infrastructure, there is a lot of headroom before it's a losing proposition. Which is what I think the OP misses discussing in as much detail as they could with AWS -- are there ways AWS is saving some customers developer/ops staff time over the cheaper alternatives? Cause that is often more expensive than the resources themselves. Could just be "because that's what…

My experience - from doing devops consulting and moving clients off AWS every chance I get - is that it's bad business for devops consultants to move clients off AWS in terms of short term billable hours, because clients spent more money on me when they're on AWS. If I was after maximising billable hours in the short term, then I'd recommend AWS all the time...

As such a lot of devops consultants certainly have all the wrong incentives to recommend it, and these days most of them also lack experience of how to price out alternatives.

E.g. a typical beginners mistake is to think you'd price out a one-to-one match of servers between AWS and an alternative, but one of the benefits of picking other options is that you're able to look at your app and design a setup that fits your needs better. Network latency matters. Ability to add enough RAM to fit your database working set in RAM if at all possible matters. And so on. With AWS this is often possible but often at the cost of scaling out other things too that you don't need.

And most developers won't do it if you don't give them budget responsibility and make them justify the cost and then cut their budget.

Development teams used to AWS tends to spin up instance after instance instead of actually measuring and figuring out why they're running into limits to keep costs down. Giving dev teams control over infra without having someone with extensive operations experience is an absolute disaster if you want to manage costs.

I work for a VC now. When I evaluate the tech teams of people who apply to us, it's perfectly fine if they use AWS, but it's a massive red flag to me if they don't understand the costs and the costs of alternatives. Usually the ones who do know they're paying for speed and convenience, and have thoughts on how to cut costs by moving parts or all of their services off AWS as they scale, or they have a rationale for why their hosting is never going to be a big part of their overall cost base.

The only case where AWS is cost effective is if you get big enough to negotiate really hefty discounts. It's possible - I've heard examples.

But if you're paying AWS list prices, chances are sooner or later you'll come across a competitor that isn't.

Re: Is a billion dollars worth of server lying on the ground?

#200

Post by bobthepanda »

shuntress wrote: post #192
throwaway1777 wrote: post #190

Earlier quoted context omitted.

Definitely need to consider spikes in demand too. A perfect example of this was covid 19 supply shocks. Supply chains got a lot better at just in time delivery and didn’t keep much back stock, but with covid this broke down when everyone went out shopping to stock up for lockdown and work from home. Edit: s/panic shopping/shopping/

It's not "panic shopping" if overnight I transition from eating half my meals at work to eating all my meals at home.

While this also happened, there was definitely some level of panic buying that happened in the initial stages, at least in the US. The toilet paper/paper towel shortage was pretty much driven by pure panic buying, and you had reports of crazy lines outside of big bulk supercenters like Costco.
Post reply on HN