Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

cep.dev

Post reply on HN
Print view View original

101–110 of 247 posts

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#101

Post by Imustaskforhelp »

0xbadcafebee wrote: post #76
neo_doom wrote: post #42

> Regret: Not adopting an identity platform early on. I stuck with Google Workspace at the start... I've worked with hundreds of customers to integrate IdP's with our application and Google Workspace was by far the worst of the big players (Entra ID, Okta, Ping). Its extremely inflexible for even the most basic SAML configuration. Stay far, far away.

And it's a horrible moat. I've gotten locked out of a Google Workspace permanently because the person who set it up left, used a personal email/phone to do it, and despite us owning/controlling the domain, Google wouldn't unlock admin access to the Workspace for us, they would only delete it . Unacceptable business risk.

Holy moly. this is nightmare fuel.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#103

Post by piokoch »

I've just look out of curiosity on Appsmith, as the author endorsed this tool as some admin panel builder. I had to double check the name, as right now this is, surprise, surprise, AI powered application builder...

I used to use Replit for educational purposes, to be able to create simple programs in any language and share them with others (teachers, students). That was really useful.

Now Replit is a frontend to some AI chat that is supposed to write software for me.

Is this jumping into AI bandwagon everywhere a new trend? Is this really needed? Is this really profitable?

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#104

Post by jamiemallers »

jrjeksjd8d wrote: post #4

I see you regret Datadog but there's no alternative - did you end up homebrewing metrics, or are you just living with their insane pricing model? In my experience they suck but not enough to leave.

"No alternative" isn't quite right anymore, though I understand the feeling. The real problem with Datadog isn't the pricing - it's that their per-host model incentivizes you to care about infrastructure topology rather than user-facing behavior. You end up with 10,000 dashboards and still can't answer "is checkout broken right now?"

The open source stack has gotten genuinely viable: Prometheus/VictoriaMetrics for metrics, Grafana for viz, and OpenTelemetry as the collection layer means you're not locked into anyone's agent. The gap used to be in correlation - connecting a metric spike to a trace to a log line - but that's narrowed significantly.

The actual hard part of leaving DD isn't technical, it's organizational. DD becomes load-bearing for on-call runbooks, alert routing, and team muscle memory. Migration is less "swap the backend" and more "retrain your incident response."

If you're evaluating: the question I'd ask isn't "which vendor has the best dashboards" but "can I get from alert to root cause in under 5 minutes with this tool?" That's the metric that actually correlates with MTTR, and it's where most monitoring setups (including expensive ones) fail.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#105

Post by mettamage »

As a non infra guy I'll say this. I'm curious about Linear. At my own company I vibecoded my own project management app against the JIRA API because I can't stand our version of JIRA. It's too many clicks, too many things to remember and it's unintuitive.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#106

Post by bob1029 »

> Not using Function as a Service(FaaS) more

FaaS is almost certainly a mistake. I get the appeal from an accountant's perspective, but from a debugging and development perspective it's really fucking awful compared to using a traditional VM. Getting at logs in something like azure functions is a great example of this.

I pushed really hard for FaaS until I had to support it. It's the worst kind of trap. I still get sweaty thinking about some of the issues we had with it.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#108

Post by stroebs »

The Bottlerocket issues really surprise me - not an experience I've shared even with heavy use. I use EKS with Bottlerocket + managed addons + Karpenter, and our security team is super happy that _nobody_ has access to the underlying nodes. Immutable OS is a key selling point, and Brupop "just works" to keep everything up to date without any input. Patching nodes is something I haven't had to think about in almost a year.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#109

Post by CodesInChaos »

bob1029 wrote: post #106

> Not using Function as a Service(FaaS) more FaaS is almost certainly a mistake. I get the appeal from an accountant's perspective, but from a debugging and development perspective it's really fucking awful compared to using a traditional VM. Getting at logs in something like azure functions is a great example of this. I pushed really hard for FaaS until I had to support it. It's the worst kind of trap. I still get s…

What's the issue with logging? I would have expected stdout/stderr to get automatically transferred to the providers managed logging solution (e.g. cloudwatch).

Though I never really understood the appeal of FaaS over something like Google-Cloud-Run.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#110

Post by coverj »

notyourwork wrote: post #96
calmbonsai wrote: post #5

This is the best post to HN in quite some time. Kudos to the detailed and structured break-down. If the author had a Ko-Fi they would've just earned $50 USD from me. I've been thinking of making the leap away from JIRA and I concur on RDS, Terraform for IAC, and FaaS whenever possible. Google support is non-existent and I only recommend GC for pure compute. I hear good things about Big Table, but I've never used in i…

Curious from you or others when FaaS isn’t possible? What criteria do you look for to decide or migrate off?

not possible: - workloads over 15m for lambda last time I checked, unsure on other providers - if you are looking to do anything stateful

possible but not ideal/inconveniences: - cold starts can hamper latency sensitive apps (language dependant + there are things you can do) - if you have consistent traffic its not very good value for money - if you value local debugging

Post reply on HN