Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every infrastructure decision I endorse or regret

cep.dev

Post reply on HN
Print view View original

201–210 of 644 posts

Re: Almost every infrastructure decision I endorse or regret

#201

Post by nyc_data_geek »

layoric wrote: post #139
nextos wrote: post #128

Earlier quoted context omitted.

Same experience here. As a small organization, the quotes we got from cloud providers have always been prohibitively expensive compared to running things locally, even when we accounted for geographical redundancy, generous labor costs, etc. Plus, we get to keep know how and avoid lock-in, which are extremely important things in the long term. Besides, running things locally can be refreshingly simple if you are just…

> Plus, we get to keep know how and avoid lock-in, which are extremely important things in the long term. So much this. The "keep know how" has been so greatly avoided over the past 10 years, I hope people with these skills start getting paid more as more companies realize the cost difference.

Even without owning the infrastructure, running in the cloud without know-how is very dangerous.

I hear tell of a shop that was running on ephemeral instance based compute fleets (EC2 spot instances, iirc), with all their prod data in-memory. Guess what happened to their data when spot instance availability cratered due to an unusual demand spike? No more data, no more shop.

Don't even get me started on the number of privacy breaches because people don't know not to put customer information in public cloud storage buckets.

Re: Almost every infrastructure decision I endorse or regret

#202

Post by bilalq »

I love this write-up and the way it's presented. I disagree with some of the decisions and recommendations, but it's great to read through the reasoning even in those cases.

It'd be amazing if more people published similar articles and there was a way to cross-compare them. At the very least, I'm inspired to write a similar article.

Re: Almost every infrastructure decision I endorse or regret

#203

Post by darknavi »

morsecodist wrote: post #131

> Picking AWS over Google Cloud I know this is an unpopular opinion but I think google cloud is amazing compared to AWS. I use google cloud run and it works like a dream. I have never found an easier way to get a docker container running in the cloud. The services all have sensible names, there are fewer more important services compared to the mess of AWS services, and the UI is more intuitive. The only downside I ha…

> I have never found an easier way to get a docker container running in the cloud

I don't have a ton of Azure or cloud experience but I run an Unraid server locally which has a decent Docker gui.

Getting a docker container running in Azure is so complicated. I gave up after an hour of poking around.

Re: Almost every infrastructure decision I endorse or regret

#205

Post by roughly »

The Bazel one made me chuckle - I worked at a company with an scm & build setup clearly inspired by Google’s setup. As a non-ex-Googler, I found it obviously insane, but there was just no way to get traction on that argument. I love that the rest of this list is pretty cut and dry, but Bazel is the one thing that the author can’t bring themself to say “don’t regret” even though they clearly don’t regret not using it.

Re: Almost every infrastructure decision I endorse or regret

#206

Post by macNchz »

jacurtis wrote: post #153
macNchz wrote: post #54

Earlier quoted context omitted.

> Cost is the most common complaint and it's almost always from people who don't have it configured correctly (which to be fair Datadog makes it far too easy to misconfigure things and blow up costs). I loved Datadog 10 years ago when I joined a company that already used it where I never once had to think about pricing. It was at the top of my list when evaluating monitoring tools for my company last year, until I go…

> Datadog makes it far too easy to misconfigure things and blow up costs I'll give you a fun example. It's fresh in my mind because i just got reamed out about it this week. In our last contract with DataDog, they convinced us to try out the CloudSIEM product, we put in a small $600/mo committment to it to try it out. Well, we never really set it up and it sat on autopilot for many months. We fell under our contract…

That’s just the sort of hypothetical scenario that kept running through my head as I tried to find a way for us to use Datadog. I even particularly wanted to use the CloudSIEM product. Bummer.

Re: Almost every infrastructure decision I endorse or regret

#207

Post by crabmusket »

FridgeSeal wrote: post #57
cqqxo4zV46cp wrote: post #21

Earlier quoted context omitted.

Newer (aka next gen aka Team-managed) Jira projects are pretty solid.

Do jira pages still take 30 seconds to load, and have all the interaction speed of cold molasses? Does it have nice keyboard shortcuts yet? Do I still need to perform an arcane ritual of setup to get the ticket statuses to be what I want? Linear has been such a breath of fresh air, with such a solid desktop app (on Mac OS) that I don’t ever want to go back. Stuff happens instantly , the layout and semantics are an ex…

Linear is making (fairly) good on the promises of local-first software. As opposed to "every click is a round trip to the server" software.

Re: Almost every infrastructure decision I endorse or regret

#208

Post by Spivak »

xyzzy_plugh wrote: post #95
cedws wrote: post #91

Earlier quoted context omitted.

K8S is a disastrous complexity bomb. You need millions upon millions of lines of code just to build a usable platform. Securing Kubernetes is a nightmare. And lock-in never really went away because it's all coupled with cloud specific stuff anyway. Many of the core concepts of Kubernetes should be taken to build a new alternative without all the footguns. Security should be baked in, not an afterthought when you need…

This isn't my experience at all. Maybe three or four years ago? Who exactly needs millions of lines of code?

I think they're more getting a k8s requiring a whole mess of 3rd party code to actually be useful when bringing it to prod. For EKS you end up having coredns, fluentbit, secrets store, external dns, aws ebs csi controller, aws k8s cni, etc.

And in the end it's hard to say if you've actually gained anything except now this different code manages your AWS resources like you were doing with CF or terraform.

Re: Almost every infrastructure decision I endorse or regret

#209

Post by osigurdson »

callalex wrote: post #129
steveBK123 wrote: post #66

Earlier quoted context omitted.

Once you commit to more deeply Amazon flavored parts of AWS like Aurora, aren't you now fairly committed to hoping your scale never exceeds the cost-benefit tradeoff?

If you’re paying list price at scale you are doing it very wrong.

Interesting how cloud services are sold like used cars.

Re: Almost every infrastructure decision I endorse or regret

#210

Post by singron »

charred_patina wrote: post #125
LispSporks22 wrote: post #74

I feel like this is overkill for a startup. Why not dump your application server and dependencies into rented data center (or EC2 if you must) and setup a coarse DR? Maybe start with a monolith in PHP or Rails. None of that word salad sounds like startup to me, but then again everyone loves to refer to themselves as a startup (must be a recruiting tool?), so perhaps muh dude is spot on.

I don't want to be negative, but this post reads like a list of things that I want to avoid in my career. I did a brief stint in cloud stuff at a FAANG and I don't care to go back to it. Right now I'm engineer No. 1 at a current startup just doing DDD with a Django monolith. I'm still pretty Jr. and I'm wondering if there's a way to scale without needing to get into all of the things the author of this article mentio…

You don't need this many tools, especially really early. It also depends on the particulars of your business. E.g. if you are B2B SaaS, then you need a ton of stuff automatically to get SOC2 and generally appease the security requirements of your customers.

That said, anything that's set-and-forget is great to start with. Anything that requires it's own care and feeding can wait unless it's really critical. I think we have a project each quarter to optimize our datadog costs and renegotiate our contract.

Also if you make microservices, you are going to need a ton of tools.

Post reply on HN