Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What the hell have you built

wthhyb.sacha.house

Post reply on HN
Print view View original

201–210 of 239 posts

Re: What the hell have you built

#201

Post by sevensor »

vbezhenar wrote: post #176
strken wrote: post #175

Earlier quoted context omitted.

I don't see why it's ridiculous to have an architectural barrier for org reasons. Requiring every component to be behind a network call seems like overkill in nearly all cases, but encapsulating complexity into a library where domain experts can maintain it is how most software gets built. You've got to lock those demons away where they can't affect the rest of the users.

The problem is, that library usually does not provide good enough boundaries. C library can just shit over your process memory. Java library can cause all the hell over your objects with reflection, can just call System.exit(LOL). Minimal boundary to keep demons at bay is process boundary and you need some way for processes to talk to each other. If you're separating components into processes, it's very natural to pu…

> it's very natural to put them to different machines, so you need your IPC to be network calls

But why is this natural? I’m not saying we shouldn’t have network RPC, but it’s not obvious to me that we should have only network RPC when there are cheap local IPC mechanisms.

Re: What the hell have you built

#202

Post by 1313ed01 »

fouc wrote: post #86

Funny, from the title I was expecting a productivity-adjacent "What have you even built?" article. Except it's really a "What over-engineered monstrosity have you built?" in the theme of "choose boring technology" p.s. MariaDB (MySQL fork) is technically older and more boring than PostgreSQL so they're both equally valid choices. Best choice is ultimately whatever you're most familiar with.

MariaDB from 2009, based on MySQL from 1995.

PostgreSQL from 1996, based on Postgres95 from 1995, based on POSTGRES from 1989, based on INGRES from 1974(?).

I wonder if any lines of 1970's or at least 1980's code still survive in some corner of the PostgeSQL code base or if everything has been rewritten at least once by now? Must have started out in K&R C, if it was even C?

Re: What the hell have you built

#203

Post by 1718627440 »

hshdhdhehd wrote: post #188
1718627440 wrote: post #27

Earlier quoted context omitted.

When you use sqlite, you can distribute your program by distributing a single executable file. That's what I call simple.

Yes that is simple, especially for a desktop app. I would argue it is not resilient enough for a web app. No one wrote the rules in stone, but I assume server side you want the host to manage data recovery and availability. Client side it is the laptop owners problem. On a laptop, availability is almost entirely correlated with "has power source" and "works" and data recovery "made a backup somehow". So I think we ar…

I was especially thinking about a program running on a server. I wouldn't statically link everything for a desktop program, because it just causes incompatibilities left-and-right. But for a server, you don't have external vendors, so this is a good option, and if you use SQLite as the database, then this means that deploying is not more complicated than uploading a single executable and this is also something, that can be done atomically.

I don't see how this has worse effects on recovery and availability. The data is still in a separate file, that you can backup and the modification still happens through a database layer which handles atomic transactions and file system interaction. The availability is also not worse, unless you would have hot code reloading without SQLite, which seems like an orthogonal issue.

Re: What the hell have you built

#204

Post by immibis »

torben-friis wrote: post #164
benterix wrote: post #154

Earlier quoted context omitted.

With time, I discovered something interesting: for us, techies, using container orchestration is about reliability, zero-downtime deployments, limiting blast radius etc. But for management, it's completely different. It's all about managing complexity on an organizational level. It's so much easier to think in terms "Team 1 is in charge of microservice A". And I know from experience that it works decently enough, at…

It’s not a management thing. I’m an engineer and I think it’s THE main advantage micro services actually provide: they split your code hard and allow a team to actually get ownership of the domain. No crossing domain boundaries, no in between shared code, etc. I know: it’s ridiculous to have an architectural barrier for an organizational reason, and the cost of a bad slice multiplies. I still think in some situations…

And then you have some other group of people that sees all the redundancy and decides to implement a single unified platform on which all the microservices shall be deployed.

Re: What the hell have you built

#205

Post by noirscape »

wewewedxfgdf wrote: post #23

"Maybe Redis for caching". Really that's going way too far - you do NOT need Redis for caching. Just put it in Postgres. Why go to this much trouble to put people in their place for over engineering then concede "maybe Redis for caching" when this is absolutely something you can do in Postgres. The author clearly cannot stop their own inner desire for overengineering.

A cache can help even for small stuff if there's something time-consuming to do on a small server.

Redis/valkey is definitely overkill though. A slightly modified memcached config (only so it accepts larger keys; server responses larger than 1MB aren't always avoidable) is a far simpler solution that provides 99% of what you need in practice. Unlike redis/valkey, it's also explicitly a volatile cache that can't do persistence, meaning you are disincentivized from bad software design patterns where the cache becomes state your application assumes any level of consistency of (including it's existence). If you aren't serving millions of users, stateful cache is a pattern best avoided.

DB caches aren't very good mostly because of speed; they have to read from the filesystem (and have network overhead), while a cache reads from memory and can often just live on the same server as the rest of the service.

Re: What the hell have you built

#206

Post by 9dev »

pjc50 wrote: post #196
9dev wrote: post #143

Earlier quoted context omitted.

Because it adds friction, and whoever introduces that CI pipeline will be the one getting messages from annoyed developers, saying "your pipeline isn't working again". It's definitely a source of complexity on its own, so something you want to consider first.

I'm aware of how much overhead CI pipelines can be, especially for multiple platforms and architectures, but at the same time developing for N>1 developers without some sort of CI feels like developing without version control: it's like coming to work without your trousers on.

Yeah, that was my entire point really—there's some complexity that's just warranted. It's similar to a proper risk assessment analysis: The goal isn't to avoid all possible risks, but accepting some risk factors as long as you can justify them properly.

As long as you're pragmatic and honest with what you need from your CI setup, it's okay that it makes your system more complex—you're getting something in return after all.

Re: What the hell have you built

#207

Post by 9dev »

pjmlp wrote: post #187
9dev wrote: post #145

Earlier quoted context omitted.

You don't. When your server crashes, your availability is zero. It might crash because of a myriad of reasons; at some times, you might need to update the kernel to patch a security issue for example, and are forced to take your app down yourself. If your business can afford irregular downtime, by all means, go for it. Otherwise, you'll need to take precautions, and that will invariably make the system more complex t…

Well, load balancers are an option.

They are: But now you've expanded the definition of "a single monolith with postgres" to multiple replicas that need to be updated in sync, you've suddenly got shared state across multiple, fully isolated processes (in the best case) or running on multiple nodes (in the worst case), and a myriad of other subtle gotchas you need to account for, which raises the overall complexity considerably.

Re: What the hell have you built

#208

Post by YetAnotherNick »

macspoofing wrote: post #127
9dev wrote: post #81

It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server. If the process crashes or your harddisk dies, you want redundancy so even those twelve customers can still access the appli…

>It's sure a corny stance to hold if you're navigating an infrastructure nightmare daily, but in my opinion, much of the complexity addresses not technical, but organisational issues: You want straightforward, self-contained deployments for one, instead of uploading files onto your single server ... You can get all that with a monolith server and a Postgres backend.

Most times it isn't complexity that bites, it is the brittleness. It's much easier to work with bad but well documented solution(e.g github actions) where all the issues have been faced by users and workaround is documented by community, rather than rolling out your own(e.g. simple script based CI/CD).

Re: What the hell have you built

#209

Post by ruperthair »

fouc wrote: post #86

Funny, from the title I was expecting a productivity-adjacent "What have you even built?" article. Except it's really a "What over-engineered monstrosity have you built?" in the theme of "choose boring technology" p.s. MariaDB (MySQL fork) is technically older and more boring than PostgreSQL so they're both equally valid choices. Best choice is ultimately whatever you're most familiar with.

Not that oldness is much of a metric for quality, but Postgres does go quite a lot further back: https://en.wikipedia.org/wiki/PostgreSQL#Ingres_and_Universi...

Re: What the hell have you built

#210

Post by 9dev »

sfn42 wrote: post #157
9dev wrote: post #145

Earlier quoted context omitted.

You don't. When your server crashes, your availability is zero. It might crash because of a myriad of reasons; at some times, you might need to update the kernel to patch a security issue for example, and are forced to take your app down yourself. If your business can afford irregular downtime, by all means, go for it. Otherwise, you'll need to take precautions, and that will invariably make the system more complex t…

I don't see how you solve this with microservices. You'll have to take down your services in these situations too, a monolith vs microservices soup has the exact same problem. Also in 5 years of working on both microservicy systems and monoliths, not once has these things you describe been a problem for me. Everything I've hosted in Azure has been perfectly available pretty much all the time unless a developer messed…

> I don't see how you solve this with microservices.

I don't think I implied that microservices are the solution, really. You can have a replicated monolith, but that absolutely adds complexity of its own.

> But sure let's make our app 100 times more complicated because maybe some time in the next 10 years the complexity might save us an hour of downtime.

Adding replicas and load balancing doesn't have to be a hundred times more complex.

> I'd say it's more likely the added complexity will cause more downtime than it saves.

As I said before, this is an assessment you will need to make for your use case, and balance uptime requirements against your complexity budget; either answer is valid, as long as you feel confident with it. Only a Sith believes in absolutes.

Post reply on HN