Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Defense of Simple Architectures (2022)

danluu.com

Post reply on HN
Print view View original

91–100 of 447 posts

Re: In Defense of Simple Architectures (2022)

#91

Post by phendrenad2 »

zer00eyz wrote: post #10

Early in my career one of the IT guys told me that one of the people on staff was "a technical magpie". I looked at him with a raised eyebrow and he said "He has to grab every shiny piece of tech that shows up and add it to the pile". This is where we are. I can't tell you how many times I have seen projects get done just to pad a PM or developers resume. Just because it was the lastest and greatest hot shit thing to…

It's the job of engineering management to stop this. We're supposed to say "why do you need this? Justify the need for this". I.E. "Why do you need kafka here? Will we have enough traffic volume to warrant it? Make a proposal." And they need to follow up and ask "Was that needed? Show how we're using it".

But engineering management is so busy filling out TPS reports they don't have time to actually do any oversight.

Re: In Defense of Simple Architectures (2022)

#92

Post by whizzter »

As a young gun coming from working in games before touching "internet"/"enterprise" software back around 2006 I had an eye on performance matters and all the early Google papers caught my eye (esp as I was on a project with an overpositive sales CEO that had in his mind that we'd run to Google valuations within a year).

A sobering account was our second CTO who told us that their previous 65000 user application ran on a single database/server, so for the rest of that project we just kept most our application on a single DB and focused on tuning that where needed.

Re: In Defense of Simple Architectures (2022)

#93

Post by 2d8a875f-39a2-4 »

lelanthran wrote: post #62
2d8a875f-39a2-4 wrote: post #39

I agree with the general sentiment that simple architectures are better and monoliths are mostly fine. But. I've dealt with way too many teams whose shit is falling over due to synchronous IO even at laughably low volumes. Don't do that if you can avoid it. "Subtle data-integrity bugs" are not something we should be discussing in a system of financial record. Avoiding them should have been designed in from the start.

> shit is falling over due to synchronous IO even at laughably low volumes. Like? I ask because even synchronous IO let's you serve millions of requests per month on a cheap VPS. That's enough in the b2b space to keep a company of 1000 employees in business.

Until one of your IO destinations develops some latency. Or your workflow adds a few more sync IOs into each request. Or you suddenly run outta threads.

Then even if you're only at millions per month you've probably got problems.

Re: In Defense of Simple Architectures (2022)

#94

Post by bunderbunder »

dns_snek wrote: post #84
nijave wrote: post #72

Earlier quoted context omitted.

Microservices can help with performance by splitting off performance critical pieces and allowing you to rewrite in a different stack or language (Rust or go instead of Ruby or Python) But yeah, they also tend to explode complexity

An important point that people seem to forget is that you don't need microservices to invoke performant native code, just a dynamic library and FFI support. The entire Python ecosystem is built around this idea.

This is why Python stole enterprise big data and machine learning from Java. It actually has a higher performance ceiling for certain specific situations because, almost uniquely among garbage collected high-level languages, it can call native code without marshaling or memory pinning.

Re: In Defense of Simple Architectures (2022)

#95

Post by throwaway2037 »

glenjamin wrote: post #48

In my opinion the best antidote to overly complex architectures is to have engineering teams and the engineers in them be rewarded based on actual business outcomes. I suspect the era of VC-money and ZIRP led to a large number of engineers who were disconnected from successful long-term business outcomes, so there was no incentive to simplify.

What is this obsession on HN about "everything can explained with the end of ZIRP" (zero interest rate policy)? Really: Even with overnight rates at 5%, the returns are still awful compared to even moderatly successful VC firms. And, I do not write this as a fanboi/gurl for VCs. (Many have shitty, immoral business practices to enrich themselves at the expense of hard-working non-executive staff.) Also, "end of VC-money": No such thing. They are practically bursting at the seams with uninvested money gathered during ZIRP.

Re: In Defense of Simple Architectures (2022)

#96

Post by chefandy »

TeMPOraL wrote: post #40
zer00eyz wrote: post #29

Earlier quoted context omitted.

> I'm guessing this guy was a serial job hopper who had no expectation of being able to progress up the ladder at the company you were at. The magpie was practically furniture (Over a decade there). We speculated that he had buried a literal body for the CEO based on what he got away with. Shiny objects was an astute call on the part of IT guy (he was setting up another new MacBook for him)

On the other hand, at least someone was exploring new tech. In the exploration/exploitation problem, going 100% exploitation and only ever using the same boring old tech for everything is not the optimal choice either.

Exploring tech is great! … for smaller projects for proofs of concept, prototypes, side projects, projects specifically for researching new technologies… heck yeah.

Just not for your prod architecture. Many late night beepers have beeped and phones lit up because the piece that holds together the two pieces that let that thing talk to the database queue monitor thing stopped working so the whole thing went down.

Re: In Defense of Simple Architectures (2022)

#97

Post by TeMPOraL »

zer00eyz wrote: post #10

Early in my career one of the IT guys told me that one of the people on staff was "a technical magpie". I looked at him with a raised eyebrow and he said "He has to grab every shiny piece of tech that shows up and add it to the pile". This is where we are. I can't tell you how many times I have seen projects get done just to pad a PM or developers resume. Just because it was the lastest and greatest hot shit thing to…

> Workman like functionality isnt sexy, it wont be the hot bullet point on your resume, it wont get you your next job, but it is dam effective.

So, not fun, not rewarding, no intellectual challenge, no career benefit. Why exactly should I want to do it? This isn't the goddamn United Federation of Planets, nor is the company a church - why exactly should I go above and beyond what I agreed to in exchange for my salary? It's not like the bosses go above and beyond either, nor do they believe in company "mission".

To be clear: I understand the importance of actually doing your job right, and benefits of using boring tech, but you are not selling that well here. Employees need food and shelter and creature comforts, and so do their families. They are going to think beyond the current job, because if they won't, nobody else will.

Re: In Defense of Simple Architectures (2022)

#98

Post by klabb3 »

rugger wrote: post #64
rugger wrote: post #8

Monolith is fine if you have a fairly simple process for manipulating your data. Like posting an article, then a comment, with some moderation thrown in. But when you start adding business rules, and start transforming your data and moving it around, then your monolith will become too complex and often too expensive to run. Lots of moving parts tightly coupled together, long-running transaction wrapping multiple join…

A few comments point out that replacing a monolith with micro services doesn't reduce complexity. I agree 100%. That's why I mentioned Event Sourcing pattern, not "microservices". Think of a single event log as a source of truth where all the data goes, and many consumer processes working in parallel alongside, picking only those events (and the embedded data) that concern them, reacting to them, then passing it on n…

ES has the potential but is too immature of a pattern to be simple. It’s a shame, but let’s not pretend.

For instance, an immutable event log is illegal in many cases (PII). So you have to either do compaction on the log or use an outside mutable store.

Another issue is code evolution: if you change your event processing logic at runtime, you get a different state if you replay it. Maybe some users or orders will not be created at all. How you deal with that? Prevent it with tooling/testing or generate new events for internal actions?

Also, all the derived state is eventually consistent (so far so good) but for non-toy apps you absolutely need to use derived state to process events, which naively breaks determinism (now your event processing depends on the cursor of the derived state).

Check out Rama[1]. They’re solving this problem, and it’s super interesting but again let’s not fool ourselves – we’re far from mature and boring now.

Something like it could hopefully become boring in the future. Many of these features could probably be simplified or skipped entirely in later iterations of these patterns.

[1]: https://redplanetlabs.com/learn-rama

Re: In Defense of Simple Architectures (2022)

#99

Post by palata »

I believe it is as easy to over-engineer as it is to under-engineer. Architecture is the art of doing "just enough"; it must be as simple as possible, but as complex as necessary.

But that is hard to do, and it takes experience. And one thing that does not describe the IT industry well is "experience": don't most software developers have less than 5 years experience? You can read all the books you want and pass all the certifications you can like a modern Agile manager, but at the end of the day you will have the same problem they have: experience takes time.

Inexperienced engineers throw sexy tech at everything, inexperienced manager throw bullshit Agile metrics at everything. Same fight.

Re: In Defense of Simple Architectures (2022)

#100

Post by jimnotgym »

I got told the other day that an ecommerce to ERP integration I built 8 years ago at a company I don't work for anymore, is still running without so much as a reboot. Apparently it is on it's third different website with no changes.

Linux, Bash, Python FTW. Everything was designed to be as simple as humanly possible. I mostly built it in my spare time too.

I once showed it to a real developer and he was horrified that some long messages that had to be sent were hardcoded into a file and concatenated to the one bit that changes...it was my prototype that worked so well I never changed it.

It only ran every 15 minutes, since all the systems that depended on the order ran less frequently than that. Everything was handled sequentially, and quite deliberately so.

It actually had a lot of functionality it gained over a couple of months, it had to print some stuff, detect fraud, fix addresses etc.

My favourite bit is that several companies had offered to build it for £mega and decided it couldn't be done! I wish I could describe it in full...

Post reply on HN