Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every infrastructure decision I endorse or regret

cep.dev

Post reply on HN
Print view View original

411–420 of 644 posts

Re: Almost every infrastructure decision I endorse or regret

#411

Post by marcinzm »

CSMastermind wrote: post #10

So by and large I agree with the things in this article. It's interesting that the points I disagree with the author on are all SaaS products: > Moving off JIRA onto linear I don't get the hype. Linear is fine and all but I constantly find things I either can't or don't know how to do. How do I make different ticket types with different sets of fields? No clue. > Not using Terraform Cloud No Regrets I generally recom…

> Cost is the most common complaint and it's almost always from people who don't have it configured correctly (which to be fair Datadog makes it far too easy to misconfigure things and blow up costs).

Datadog's cheapest pricing is $15/host/month. I believe that is based on the largest sustained peak usage you have.

We run spot instances on AWS for machine learning workflows. A lot of them if we're training and none otherwise. Usually we're using zero. Using DataDog at it's lowest price would basically double the cost of those instances.

Re: Almost every infrastructure decision I endorse or regret

#412

Post by danpalmer »

rswail wrote: post #253
morsecodist wrote: post #131

> Picking AWS over Google Cloud I know this is an unpopular opinion but I think google cloud is amazing compared to AWS. I use google cloud run and it works like a dream. I have never found an easier way to get a docker container running in the cloud. The services all have sensible names, there are fewer more important services compared to the mess of AWS services, and the UI is more intuitive. The only downside I ha…

We are a reasonably large AWS customer and our account manager sends out regular emails with NDA information on what's coming up, we have regular meetings with them about things as wide ranging as database tuning and code development/deployment governance. They often provide that consulting for free, and we know their biases. There's nothing hidden about the fact that they will push us to use AWS services. On the oth…

In a previous role I got all of these things from GCP – they ran training for us, gave us early access to some alpha/beta stage products (under NDA), we got direct onboarding from engineers on those, they gave us consulting level support on some things and offered much more of it than we took up.

Re: Almost every infrastructure decision I endorse or regret

#413

Post by emseetech »

electroly wrote: post #6

> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…

From what I’ve read, a common model for mmorpg companies is to use on-prem or colocated as their primary and then provision a cloud service for backup or overage.

Seems like a solid cost effective approach for when a company reaches a certain scale.

Re: Almost every infrastructure decision I endorse or regret

#414

Post by ildjarn »

marcosdumay wrote: post #109
inquist wrote: post #88

Earlier quoted context omitted.

What’s wrong with uuidv4 as PK?

Serial integers always work better than any uuid as PKs, but the thing with uuid4 is that it disrupts any kind of index or physical ordering you decide to put on your data. Uuids are really for external communication, not in-system organization.

Serial index forces a synchronisation point on every entity that can create records. If this is only ever a single database that’s fine but plenty of apps can’t scale this way.

Re: Almost every infrastructure decision I endorse or regret

#415

Post by hknmtt »

electroly wrote: post #6

> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…

[dead]

Re: Almost every infrastructure decision I endorse or regret

#416

Post by jpb0104 »

colechristensen wrote: post #53
CSMastermind wrote: post #10

So by and large I agree with the things in this article. It's interesting that the points I disagree with the author on are all SaaS products: > Moving off JIRA onto linear I don't get the hype. Linear is fine and all but I constantly find things I either can't or don't know how to do. How do I make different ticket types with different sets of fields? No clue. > Not using Terraform Cloud No Regrets I generally recom…

PagerDuty’s cheapest plan is $21 per user month OpsGenie’s cheapest is $9 per user month but arbitrarily crippled, the plan anybody would want to use is $19 per user month So instead of a factor of ten it’s ten percent cheaper. And i just kind of expect Atlassian to suck. Datadog is ridiculously expensive and on several occasions I’ve run into problems where an obvious cause for an incident was hidden by bad behavior…

I just started building out on-call rotation scheduling to fit teams that already have an alerting solution and need simple automated scheduling. I’d love to get some feedback: https://majorpager.com

Re: Almost every infrastructure decision I endorse or regret

#417

Post by Sankozi »

sgarland wrote: post #37
vrosas wrote: post #34

Earlier quoted context omitted.

Bad schema decisions are made regardless of whether you’re one database or 50. At least with many databases the problems are localized.

But then the DB Team – if you have one – is responsible for 50 databases, each full of their own unique problems. This will undoubtedly go over poorly, but honestly I think every data decision should be gated through the DB Team (again, if you have them). Your proposed schema isn’t normalized? Straight to jail. You don’t want to learn SQL? Also straight to jail. You want to use a UUIDv4 as a primary key? Believe it o…

No single team should not be responsible for all databases. If such team exists they will either become bottleneck for every other team (by auditing carefully each schema change) or become bloated and not utilized 90% of time, or (most common) they will become nearly useless or even harmful - they will not be really responsible and they will act as dumb proxy - they will introduce latency to the schema updates, but they will not bother to check them very well (why would they? they are not responsible for the whole product, just for the database), some DB refactoring/migrations will be totally abandoned because DB team will make them too painful.

DB team could act as an auditor and expert support, but they should never be fully responsible for DB layer.

Re: Almost every infrastructure decision I endorse or regret

#418

Post by thelastparadise »

steveBK123 wrote: post #11
electroly wrote: post #6

> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…

RDS pricing is deranged at the scales I've seen too. $60k/year for something I could run on just a slice of one of my on-prem $20k servers. This is something we would have run 10s of. $600k/year operational against sub-$100k capital cost pays DBAs, backups, etc with money to spare. Sure, maybe if you are some sort of SaaS with a need for a small single DB, that also needs to be resilient, backed up, rock solid bullet…

> $600k/year operational against sub-$100k capital cost pays DBAs, backups, etc with money to spare.

One of these is not like the others (DBAs are not capex.)

Have you ever considered that if a company can get the same result for the same price ($100K opex for RDS vs same for human DBA), it actually makes much more sense to go the route that takes the human out of the loop?

The human shows up hungover, goes crazy, gropes Stacy from HR, etc.

RDS just hums along without all the liabilities.

Re: Almost every infrastructure decision I endorse or regret

#419

Post by klabb3 »

ildjarn wrote: post #394

Reading this I couldn’t help but think: yeah all of these points make sense in isolation, but if you look at the big picture, this is an absurd level of complexity. Why do we need entire teams making 1000s of micro decisions to deploy our app? I’m hungry for a simpler way, and I doubt I’m alone in this.

You’re not alone. There is a constant undercurrent of pushback against this craziness. You see it all the time here on hacker news and with people I talk to irl.

Does not mean each of these things don’t solve problems. The issue as always about complexity-utility tradeoff. Some of these things have too much complexity for too little utility. I’m not qualified to judge here, but if the suspects have Turing-complete-yaml-templates on their hands, it probably ties them to the crime scene.

Re: Almost every infrastructure decision I endorse or regret

#420

Post by hardolaf »

nextos wrote: post #128
nyc_data_geek wrote: post #86

Earlier quoted context omitted.

Some orgs are looking at moving back to on prem because they're figuring this out. For a while it was vogue to go from capex to opex costs, and C suite people were incentivized to do that via comp structures, hence "digital transformation" ie: migration to public cloud infrastructure. Now, those same orgs are realizing that renting computers actually costs more than owning them, when you're utilizing them to a signif…

Same experience here. As a small organization, the quotes we got from cloud providers have always been prohibitively expensive compared to running things locally, even when we accounted for geographical redundancy, generous labor costs, etc. Plus, we get to keep know how and avoid lock-in, which are extremely important things in the long term. Besides, running things locally can be refreshingly simple if you are just…

I was part of a relatively small org that wanted us to move to cloud dev machines. As soon as they saw the size of our existing development docker images that were 99.9% vendor tools in terms of disk space, they ran the numbers and told us that we were staying on-prem. I'm fairly sure just loading the dev images daily or weekly would be more expensive than just buying a server per employee.
Post reply on HN