Ask HN: What are some architectural decisions that improved your codebase?

I recently started a large project, so did some reading on architectures/patterns like DDD and Clean-Arch. One of the most important points I took from both was to clearly define your domain. But based upon past experiences, I have developed a dislike for "heavy" objects like those used by DDD and ORM's in general. I like to keep things simple, sort of "functional" in nature - what your link refers to as anemic objects. So I have stuck to the SOLID principles, and in particular the D = dependency injection. I've also taken a fancy to RPC style code, so that influences my code. BTW, clean arch isn't too different from the image of Layered-Arch in your link, more of an evolution really.

So here is how I apply my concepts to your problems...

Users want to know the Events they are registered for, and Events want to know the registered Users. You have a circular dependency! But really, the problem to me is that you haven't expanded your domain enough. I think you should have a third entity, something like UserEventRegistrations. Now User's and Event's don't depend on each other, and UserEventRegistrations will depend upon them. No circle!

As per my like for anemic objects, I would have a User model object to hold properties like name, and a UserRepository for doing CRUD style operations with methods like GetByID() that returns a User instance. The same would apply for Event, and something similar for UserEventRegistrations, except it's repository would have a dependency on the User and Event repository so that it can do methods like GetEventsByUserID().

Then to apply this in Clean-Arch style, I leverage whatever statically typed language I am using (Go, TypeScript, etc) to implement interfaces. So I define the domain layer as the model objects, and interfaces for the repositories. For the persistence layer, I would create a concrete implementation of the repository interfaces, and they would return instances of the domain model objects. Then for presentation, I would create a layer that expects to be dependency-injected with a concrete implementation of the repository interface. So my layers are separate, based upon the "contract" that is my domain layer.

Now your example for User.getProfileAsJson() is vague in meaning, but if you wanted to return the data in a different format than the domain model, you could have another layer on the presentation side of the equation that handles this. It would utilise the repositories to build what you need. So your "Profile" might be a single JSON payload containing a User with their Events. Your function would do UserRepo.GetByID(), check you have a User, then do UserEventRegistrationsRepo.GetEventsByUserID(User.ID). Then it would stick it in your payload, and viola.

I've not completed my project yet, but I've implemented some functionality in all layers (Go server pulling data from RDBMS and sending to TypeScript UI), and it seems to be working well. I've also noticed after the fact that my domain layer ends up looking exactly like a protocol buffers definition, so maybe just use those.

An example would be if you have a module that calls a REST API to get/put something (say time sheets for your invoicing app), then have that be its own module that is testable.

Create internal TimeSheet data structures that you pass to/from that module. The core functionality of your app should be implemented using the TimeSheet data structures and you can have tests that use those and then separate tests around calling an API.

New customer comes along and says they want to send you CSV files via SFTP (yuck, but they got money). You just have to write a new interface that works with exchanging those files and gets them into your TimeSheet data structures, the core of your app should remain unchanged.

The principle is decouple, decouple, decouple. Memory isn't just memory, it's paged and mapped, and the mappings are in a small cache called the TLB, one for each core. Each "hugetlb" page, 2MB or 1GB on x86, takes just one such cache entry, so anything big, like buffers, should live in hugepages.

A ring buffer is a kind of queue with just a head, and one writer. Each new item goes at the next place in the buffer, round-robin. A head pointer -- if it's in shared memory, an index -- gets updated "atomically" to point to the newest item. Downstream readers poll for updates to the head. New stuff overwrites the oldest stuff, so downstream readers can look until it gets overwritten, and can often avoid copying. They don't need to lock anything, but need to check that the head hasn't swept in and and overwritten what they were looking at; that is called being lapped. It is their responsibility to keep up, and prevent this.

Because there is never any question where the next entry goes, hardware devices understand ring buffers, and can be set to write to them whenever there is data. Typically a proprietary library talks to a proprietary driver to set this up, and then the hardware device runs free with no more interaction. (io_uring, AF_XDP, libexanic, ef_vi, DPDK, PF_RING, netmap, etc.)

Usually the hardware ring buffer is pretty small, a few MB, so for high-rate flows there might be cores dedicated to copying from it to one or more much, much bigger ring buffers in shared, mapped memory. Typically, multiple downstream readers watch for interesting traffic to show up on such a ring, splitting the work out to multiple cores.

Threads famously interfere with one another, mainly when competing for locks; but also, whenever they fool with the memory map, other threads may experience TLB cache stalls. Separate programs are better isolated, and can be further isolated by running on a dedicated core ("isolcpu", "NOHZ", and "taskset") that is protected against the OS sticking other threads on it, or vectoring interrupts to it. In extreme cases a core may offload its own RCU retirements, or even not run any kernel code.

A unikernel may run on such a core, running a single program, so what it thinks are system calls just call a static library. There is a lot of work going on on variations on this theme -- exokernels, parakernels, etc.

Instead of getting the file system and buffer cache all mixed up in your program, you can append to files with O_DIRECT writes, or store to mapped memory and let the kernel expose it to other processes, and spool to disk, asynchronously. A monitoring process can look at event counters in such memory as they are updated in real time. It is generally better if the program updating the counters also stores a generic description of them -- type, name, a hierarchical structure that can be read out to a JSON record, periodically, by a separate program. That might be written to a log and/or feed a status dashboard. Thus, the code doing the work just updates memory words pointed to from its working configuration, but doesn't ever need to format or write out updates. If there is any actual text logging, it goes through another ring buffer to a background logging process that, ideally, is also responsible for formatting.

Memory management -- new and delete -- is a source of unpredictable delays. Such allocations are always OK during startup, but often not after. A function that needs memory, then, should use memory provided by its caller. The top level can handle memory deterministically, pre-allocated or on the stack, with a global view of program behavior.

Using separate processes enables starting and stopping downstream processing independently, and isolates crashes. Ring buffers being read are always mapped read-only, so a crashed reader cannot corrupt any shared state.