Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

cep.dev

Post reply on HN
Print view View original

41–50 of 247 posts

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#41

Post by x3n0ph3n3 »

stackskipton wrote: post #11
zem wrote: post #6

I would love to read more about the pros and cons of using a single database, if anyone has pointers to articles

SRE here who has dealt with this before. Everything in article is excellent point but other big point is schema changes become extremely difficult because you have unknown applications possibly relying on that schema. It's also at certain point, the database becomes absolutely massive and you will need teams of DBAs care and feeding it.

Not only will you need a team of DBAs caring for it, but you'll never be able to hire them.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#42

Post by neo_doom »

> Regret: Not adopting an identity platform early on. I stuck with Google Workspace at the start...

I've worked with hundreds of customers to integrate IdP's with our application and Google Workspace was by far the worst of the big players (Entra ID, Okta, Ping). Its extremely inflexible for even the most basic SAML configuration. Stay far, far away.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#43

Post by rawgabbit »

zem wrote: post #6

I would love to read more about the pros and cons of using a single database, if anyone has pointers to articles

The things that impact the most are locking/blocking, data duplication (ghosting due to race conditions), and poor performance. The best advice is RTFM the documentation for your database; yes, it is a lot to digest that is why DBAs exist. Most of these foot guns are due to poor architecture. You have to imagine multiple users/processes are literally trying to write to the same record at the same time; when you realize this, a single table with simple key-values is completely inadequate.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#44

Post by t-writescode »

stackskipton wrote: post #12
jrjeksjd8d wrote: post #4

I see you regret Datadog but there's no alternative - did you end up homebrewing metrics, or are you just living with their insane pricing model? In my experience they suck but not enough to leave.

Not author but Prometheus is perfectly acceptable alternative if you don't want to go whole Otel route.

Prometheus + … what? Datadog is a visualization platform, prometheus is a data gathering infrastructure.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#45

Post by jbmsf »

Thanks. I've been meaning to write one of these for a long time, but you went into detail in a very effective, organized way.

I also reached a lot of similar decisions and challenges, even where we differ (ECS vs EKS) I completely understand your conclusions.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#46

Post by kaycey2022 »

nine_k wrote: post #20
kaycey2022 wrote: post #17

Feels like a minor glimpse into what's involved in running tech companies these days. Sure this list could be much simpler, but then so would the scope of the company's offerings. So AI would offer enough accountability to replace all of this? Agents juggling million token contexts? It's kind of hard to wrap my head around.

Agents run tools, too. You can make an LLM count by the means of language processing, but it's much more efficient to let it run a Python script. By the same token , it's more efficient to let an LLM operate all these tools (and more) than to force an LLM to keep all of that on its "mind", that is, context.

Agents are just not deterministic. You will see wierd things like in one thread an agent simply says it cannot access a CLI tool for whatever reason. You inspect the call, it worked just fine in another thread. You eventually shrug your shoulders and close the thread, pick up from another instead of having the agent flail around some obvious BS for hours and hours.

Just because they can run tools, doesn't mean they run them reliably. Running tools is not a be all and end all of the problem.

Amdahl's law is still in play when it comes to agents orchestrating entire business processes on their own.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#47

Post by cindyllm »

kaycey2022 wrote: post #46
nine_k wrote: post #20

Earlier quoted context omitted.

Agents run tools, too. You can make an LLM count by the means of language processing, but it's much more efficient to let it run a Python script. By the same token , it's more efficient to let an LLM operate all these tools (and more) than to force an LLM to keep all of that on its "mind", that is, context.

Agents are just not deterministic. You will see wierd things like in one thread an agent simply says it cannot access a CLI tool for whatever reason. You inspect the call, it worked just fine in another thread. You eventually shrug your shoulders and close the thread, pick up from another instead of having the agent flail around some obvious BS for hours and hours. Just because they can run tools, doesn't mean they r…

[dead]

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#49

Post by stackskipton »

t-writescode wrote: post #44
stackskipton wrote: post #12

Earlier quoted context omitted.

Not author but Prometheus is perfectly acceptable alternative if you don't want to go whole Otel route.

Prometheus + … what? Datadog is a visualization platform, prometheus is a data gathering infrastructure.

Grafana is most common one.

Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)

#50

Post by brandmeyer »

zem wrote: post #6

I would love to read more about the pros and cons of using a single database, if anyone has pointers to articles

They are very similar to the pros and cons of having a monorepo. It encourages information sharing and cross-linkage between related teams. This is simultaneously its biggest pro and its biggest con.
Post reply on HN