Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

blog.tomilkieway.com

Post reply on HN
Print view View original

71–80 of 397 posts

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#71

Post by donmcronald »

walrus01 wrote: post #13

writing as somebody who runs a big collection of bare-metal hypervisors for ISP infrastructure purposes... this post quite honestly just makes me smirk. I have truly lost track of the numerous instances, and number of people who would be better served by buying a $1200 test/development 1U dual socket server with a few fast SSDs in it, and putting it in colocation somewhere for a few hundred dollars a month. The costs…

The crazy part to me is using the cloud for testing. It’s crazy. I have a 5 year old dual CPU Xeon with 128GB of RAM and a couple NVME disks that I’ve spent about $1000 CAD total to build ($700 USD). Something in that range on Azure is about $1 / hour if you reserve a year. ~$9000 per year.

All the people running workloads that don’t require the redundancy given, like CI, blow my mind. The costs are astronomical vs buying a cheap or used server. Sure, use the cloud for you production builds, but why not augment it with something that doesn’t cost as much?

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#72

Post by walrus01 »

donmcronald wrote: post #71
walrus01 wrote: post #13

writing as somebody who runs a big collection of bare-metal hypervisors for ISP infrastructure purposes... this post quite honestly just makes me smirk. I have truly lost track of the numerous instances, and number of people who would be better served by buying a $1200 test/development 1U dual socket server with a few fast SSDs in it, and putting it in colocation somewhere for a few hundred dollars a month. The costs…

The crazy part to me is using the cloud for testing . It’s crazy. I have a 5 year old dual CPU Xeon with 128GB of RAM and a couple NVME disks that I’ve spent about $1000 CAD total to build ($700 USD). Something in that range on Azure is about $1 / hour if you reserve a year. ~$9000 per year. All the people running workloads that don’t require the redundancy given, like CI, blow my mind. The costs are astronomical vs…

as a totally randomly chosen example, that I spent not more than 20 seconds searching for, here's a system with 128GB of RAM for way under $500.

https://www.ebay.com/itm/DELL-R910-16SFF-model-4x-Intel-X755...

need to add your own storage (good quality SSDs, of course).

and it assumes you have somewhere to put noisy things...

I would estimate it's about a 500W electrical load, so figure $40-50 additional electrical bill, if you're trying to precisely account for all costs.

you can totally set up a desktop workstation dual xeon for a similar price as well.

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#73

Post by donmcronald »

bjarneh wrote: post #54
spacemanmatt wrote: post #26

Earlier quoted context omitted.

Author is an ex-Googler so he knew exactly how to speak to the system

I'm sure that was an important factor, but wouldn't this in any case just be a bill to a startup with no money on hand? It's hard to make companies (without money) economically responsible for anything I guess, it even seems hard to make companies with money responsible sometimes.

I can’t set a limit like that on my CC, so the first charge for $5k would have cleared meaning it would have run way longer and racked up way more usage. I’d bet you my computer I would have been out at least the $5k that cleared.

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#74

Post by donmcronald »

LeonM wrote: post #11
throwaway7281 wrote: post #8

To put it into perspective: You give me $72K and I'll set you up a 1PB replicated storage infra with a total of 100+ available CPU cores and half a TB RAM. I saw people burning through cash in the cloud, which makes you wonder weather money is any concern at all.

You forget your own cost here. A full-time system administrator costs more than 72k a year.

I wonder what a full time cloud engineer costs. IMO it’s trading a simple system for a complex system, so now the maintainers cost even more than sysadmins used to.

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#75

Post by klohto »

nojito wrote: post #67
scrollaway wrote: post #60

Earlier quoted context omitted.

AWS pricing is not obscure, it's just not for you. So in that sense, you are correct to not see a reason to move to the cloud, but your advice does not apply to everyone. And I don't believe they make "more money" that way at all. AWS margins are either very low or very high, and the higher margins and prices tend to be the "simpler" ones: packaged, managed products such as Redshift that are billed on fewer tiers and…

AWS's margin is currently 30+% which is massive. >AWS pricing is not obscure There is a massive secondary consulting market because of AWS's price obscurities.

> There is a massive secondary consulting market because of AWS's price obscurities.

While that's true, there is consulting market for most things that are complicated. Doesn't mean they are shady. It's simply not for you. You are welcome either to dive in or get a consultant. I promise you though, that AWS pricing isn't difficult once you understand few concepts and know your way around the Cost Explorer. With proper tagging, it's easy to drill down which resource is consuming how much. I don't believe there is a way to have a simple billing for a complicated product(s).

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#76

Post by klohto »

jcelerier wrote: post #29
LeonM wrote: post #11

Earlier quoted context omitted.

You forget your own cost here. A full-time system administrator costs more than 72k a year.

Aha what ? Where I live (Bordeaux, France) a quick glance through the job offers for full-time sysadmin are between 25 and 35k€ / year

sysadmin isn't infrastructure engineer nor SRE. The salaries for latter are way above 74k/year

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#77

Post by zigzag312 »

LeonM wrote: post #11
throwaway7281 wrote: post #8

To put it into perspective: You give me $72K and I'll set you up a 1PB replicated storage infra with a total of 100+ available CPU cores and half a TB RAM. I saw people burning through cash in the cloud, which makes you wonder weather money is any concern at all.

You forget your own cost here. A full-time system administrator costs more than 72k a year.

Why full-time? It's possible to outsource IT administration to a local IT company and pay only for set-up and maintenance that is needed. Way less than 72k a year for many use cases.

Also, companies that employ bunch of developers can find a developer that has IT administration expertise and allocate some of his time to this. Still cheaper than 72k a year or employing someone full-time, if IT requirements don't call for full time job.

It's not just cloud vs full-time.

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#78

Post by akh »

scrollaway wrote: post #60
LordHeini wrote: post #42

That sort of crap is the reason we host all our stuff on root servers. Even trying to read the amazon pricing for their instances, hours and what not, drives me insane. Seems this is done on purpose. no wonder they make so much money with it. So i have never seen a reason to move any stuff to the cloud. Just grab a dedicated server for a few bucks and put a bunch of docker containers on those. Its way cheaper, usuall…

AWS pricing is not obscure, it's just not for you. So in that sense, you are correct to not see a reason to move to the cloud, but your advice does not apply to everyone. And I don't believe they make "more money" that way at all. AWS margins are either very low or very high, and the higher margins and prices tend to be the "simpler" ones: packaged, managed products such as Redshift that are billed on fewer tiers and…

How do you take pricing into your design considerations? Does it come with experience from using an AWS service in production and understanding how it's priced, combined with the usage numbers the new system might get? I'm trying to learn more about how engineers currently do this.

Re: Burnt $72k testing Firebase and Cloud Run and almost went bankrupt

#79

Post by MattyMc »

walrus01 wrote: post #13

writing as somebody who runs a big collection of bare-metal hypervisors for ISP infrastructure purposes... this post quite honestly just makes me smirk. I have truly lost track of the numerous instances, and number of people who would be better served by buying a $1200 test/development 1U dual socket server with a few fast SSDs in it, and putting it in colocation somewhere for a few hundred dollars a month. The costs…

> writing as somebody who runs a big collection of bare-metal hypervisors for ISP infrastructure purposes.

I run a cloud SaaS company (3 employees). If I had the skillset that it sounds like you do, I might be inclined to hose on bare metal. But I don't. I don't know what a 1U dual socket server is.

It would take me some time to build these skills, and to match the agility that the cloud offers. I don't think it's worth my time, and probably not the author's time, either.

Post reply on HN