Earlier quoted context omitted.
Microservices almost always increase the amount of partial failures, but if used properly can reduce the amount of critical failures. You can certainly misapply the architecture, but you can also apply it well. It's unsurprising that most people make bad choices in a difficult domain.
Fault tolerance doesn't necessarily require microservices (as in separate code bases) though, see Erlang. Or even something like Unison. But for some reason it seems that few people are working on making our programming languages and frameworks fault tolerant.
In Defense of Simple Architectures (2022)
371–380 of 447 posts
Re: In Defense of Simple Architectures (2022)
#372This is what I tell engineers. Microservices aren't a performance strategy. They are a POTENTIAL cost saving strategy against performance. And an engineering coordination strategy. Theoretically If you have a monolith that can be scaled horizontally there isn't any difference between having 10 replicas of your monolith and having 5 replicas of two microservices with the same codebase. UNLESS you are trying to undersc…
Surely one could split a shared monolith into many internal libraries and modules, facilitating ownership?
Re: In Defense of Simple Architectures (2022)
#373Loved this line:
"I’ll discuss a few boring things we do that help us keep things boring."
Re: In Defense of Simple Architectures (2022)
#374Earlier quoted context omitted.
Honest question: if you’ve never known the tangible value of your work, how did you decide what to do? It’s an uncomfortable question to ask, but I genuinely don’t understand how that would be possible.
Your manager tells you? Or, higher up the career ladder, whatever is most urgent for the best-paying customer? Like, I know what's on the list to finish for the next multi-million-dollar payout from a major customer, but how my work contributes to it, compared to work done by 20+ other people involved in dev, operations, qa, deployment, customer negotiations, etc.? Who the fuck knows? Best I can estimate is how much…
"Your manager tells you what to work on" isn't how big tech runs for the most part, in fact it's a bit sad to think that some people work like that
Re: In Defense of Simple Architectures (2022)
#375Earlier quoted context omitted.
Right? I was going to ask OP "have you ever asked anyone?" Because, IME, managers, etc. love it when you show an interest in how the business works and where your part fits in. It also makes their job easier if they can relate the crappy stuff they have to assign you to how much benefit the business gets from it.
I must be doing something wrong because most of the time, getting interested in the wider impact of my work is held against me. I just ask why they want the stuff, suggest alternatives, point out issues, and the next day I'm an uncontrollable Maverick that means to rewrite everything and waste a ton of time… This also happens after explicit requests for feedback. Maybe they didn't actually meant it and I'm supposed t…
I'm not privy to any discussions you've had, of course, but I will comment on this because I see it in many tech people: don't question people's decisions. No matter how you phrase it, it will appear critical. And people (neurotypical or otherwise!) don't like criticism, no matter how much they claim to welcome it.
Those "explicit" requests for feedback? They want positive feedback. If you really must say something negative, suggest the opposite in a positive sense instead. i.e., instead of saying "we should not have done X. It was a bad idea," try saying "I think if we had done Y, things would have worked well" while avoiding any potentially embarrassing references to X.
Also
> neurotypical bullshit
Be careful that you're not "othering" the person you're talking to and dismissing their concerns because that's what I think when I read this.
Re: In Defense of Simple Architectures (2022)
#376Earlier quoted context omitted.
Yeah, a sense. A guess. A gut feeling . Based on what exactly? I sure do get a sense of what will require less effort in the long run, or what will makes the user's life easier, or even what is likely to decrease the defect rate… but I dare 95% of programmers, even the subset active here on HN, to reliably assess the monetary impacts of those decisions within one order of magnitude, especially compared to the alterna…
Napkin math and ROI, no one is asking for the cents. For example, build system improvement on 10% of build time across 200 developers who on average get paid 300k a year - that's a very easy math, no ? Same for time to deploy, improvements on time to fix a bug, etc. etc. You can extrapolate and compare initiatives and projects on t-shirt sizes and ROIs. Knowing where yours sit as well. What places I've worked at ? Mo…
Ground software for an observation satellite. My internship was about implementing a dead simple neural "network" (2 hidden layers, no feedback), everything was specified from up top, we didn't even get to touch the learning algorithms. Impact? I guess a big flat zero, since all the differentiators was in the learning parameters.
Peer-to-peer social network before Facebook. Never made a cent.
Geographic Information System for the military. I was for obvious reasons not allowed to know enough to estimate the impact of my work. And even then all decisions was made by the customer, and once the user (a different entity) saw the Rube Goldberg contraption we dully made for them they predictably balked, and we did what we could from there. Which was, not that much. I did some useful stuff for sure, but mostly I participated in a system that was arguably worse than the one that preceded it.
A visualiser for civil radar data. Data in, little planes in the screen out. And other nice stuff. I designed a simple C++ API that allowed the client to write business code faster than we would have ourselves (if only because of communication overhead), saving weeks of work. That contribution was utterly ignored for personal reasons, and I was eventually out. I have no idea what my actual impact was, because I don't know how far the project even went, and how widely it was eventually deployed.
The maintenance of ground software for small civil observation drones. I did some cool stuff, but then was asked to transfer ownership of this software to a recently bought team (that did stuff similar to the company I worked for). I could have known how many drones were actually deployed, but to be honest my thing just saved a few minutes of flight, while most of the cost is to get the drone and its operator on site. That company was never really profitable, I hope the good people I met there are doing well.
Scripting language for a programmable logic controller test environment. For the military, so I don't think I was allowed to even know the size of the team we'd deliver the software to. I got good feedback from them (they were happy about what I did), and I'm pretty sure my static typing made things easier for them than if I had just picked Lua or something, but how easier, and how much money it will save in the long run I have no freaking clue.
Stuff in a missile company I cannot disclose. I believe my impact was almost nil, I couldn't stand their abysmal tech environment.
Maintenance of a ground troops training system (a glorified laser tag, with debrief helpers and so on). Good luck assessing the impact of a better system, and I was just asked to do small tasks I could barely chose anyway.
Prototype ADAS system. It was never deployed. Actual impact was therefore basically nil. Cool stuff to work on though, the CAN bus is a think of beauty. One of the rare instances where I could actually learn from example, instead of seeing yet again one of the gazillion obvious ways how not to do stuff.
Ground software for some IoT device. Impact fundamentally uncertain, we had yet to sell it to anyone.
Incident reporting software, based upon a more generic distributed base. I made the encryption layer (between users & company server), with a security based on PAKE (thus avoiding a PKI, which simplified the work of the sysadmin, at a slight loss of security). Impact fundamentally uncertain, we had yet to sell it to anyone.
Charging stations for electric vehicles. I did the TPM provisioning, and mentioned a low-key security issue along the way. I participated in a questionable micro-service that was meant to help user interfaces (yeah, their IoT stuff had a micro-service architecture). Impact: whatever I did didn't save them: one year after I left, they're now going under.
Preliminary study on the possible use of AMD-SEV to prevent users from peeking at our secret sauce (DRM). I don't think I was allowed to know the list of clients, and it's not even the only alternative. I don't think I could ever have assessed the long term impact of my work there.
Flight recorder for trains (not a flight recorder then, but you get the idea). I just did little tasks here and there, didn't get the chance to have a good bird's eye view of the thing or its environment. Deployment base was knowable, but the business impact of my work was likely minimal, beyond "finish this step so we can show the client we're on track for the next short term milestone". The whole thing is a heap of technical debt, common components are impossible to update (user projects aren't locked to a given revision, they all pull from trunk), the build system is a home made monstrosity that doesn't help more than the standard monstrosities (I hate build systems)… and I was just axed from a round of layoffs.
Cryptographic library I did on my free time: https://monocypher.org/ Nice little thing with a significant user base in the embedded ecosystem (not even my primary target). I controlled everything from start to finish, and I have no idea how many users I have, let alone how much time and money I saved them. In part because it is so simple, with such an outstanding documentation (which I mostly didn't write), that most users don't even have to bug me.
---
To sum this up, my resume looks fairly horrible with respect to what I know of my actual business impact. Most of it, I think, was entirely outside my control. And I don't think I'm exceptional in this.
Re: In Defense of Simple Architectures (2022)
#377Earlier quoted context omitted.
> In a shared monolith however 0 people own it because everyone owns it and the repo becomes a disaster faster than you can say "we need enterprise caching". * I give two teams one repo each and tell each one: "This is your repo, keep it clean." -or- * I give two teams one folder each in a repo and tell each one: "This is your folder, keep it clean." If you've got a repo or folder (or anything else) that no one is re…
i think doing releases, deployments, and rollbacks is trickier in a monorepo. with multiple services/repos, each team can handle their own release cycles, on-call rotation, and only deal with the bits, tests, and failures they wrote. you lose some CPU/RAM/IO efficiency, but you gain autonomy and resilience, and faster time to resolution. https://blog.nelhage.com/post/efficiency-vs-resiliency/ e.g. at Grafana we're wo…
But I will note that almost all the benefits listed aren't specific to micro services.
(Also, to me, it's flawed that a dependency can really push to production without coordination. At the least, dependent components should decide when to "take" a new version. There are lot of ways to manage that -- versioned endpoints, e.g., so that dependents can take the new when they are ready. But it's extra complexity that's tricky in practice. e.g., you really ought to semver the endpoints, and keep versions as long as there could be any dependents... but how many is that, and how do you know? From what I've seen, people don't do that and instead dependencies push new versions, ready or not, and just deal with the problems as breakages in production, which is pretty terrible.)
Re: In Defense of Simple Architectures (2022)
#378Earlier quoted context omitted.
Your manager tells you? Or, higher up the career ladder, whatever is most urgent for the best-paying customer? Like, I know what's on the list to finish for the next multi-million-dollar payout from a major customer, but how my work contributes to it, compared to work done by 20+ other people involved in dev, operations, qa, deployment, customer negotiations, etc.? Who the fuck knows? Best I can estimate is how much…
At a lot of companies I've worked at, the engineers are empowered to decide on what to work at, it's not like they're 100% booked by PMs. Even if PMs, they can argue. "Your manager tells you what to work on" isn't how big tech runs for the most part, in fact it's a bit sad to think that some people work like that
The empowerment you speak of is but a faint dream in my town.
Re: In Defense of Simple Architectures (2022)
#379Earlier quoted context omitted.
I must be doing something wrong because most of the time, getting interested in the wider impact of my work is held against me. I just ask why they want the stuff, suggest alternatives, point out issues, and the next day I'm an uncontrollable Maverick that means to rewrite everything and waste a ton of time… This also happens after explicit requests for feedback. Maybe they didn't actually meant it and I'm supposed t…
>start questioning their decisions I'm not privy to any discussions you've had, of course, but I will comment on this because I see it in many tech people: don't question people's decisions. No matter how you phrase it, it will appear critical. And people (neurotypical or otherwise!) don't like criticism, no matter how much they claim to welcome it. Those "explicit" requests for feedback? They want positive feedback.…
When someone asks them feedback so they can improve, I tend to concentrate on what could have actual impact, and this heavily biases towards negative feedback: the build system is bulky and not well documented, newcomers waste time on it, we should replace those global variables by messages if we actually meant to do an actor model, this piece of code is 10 times bigger than it needs to be and will drag us behind if we port it as is…
To be honest, phrasing the above in a nice way that avoids hurting people's feeling is exhausting. They want to know how to make their system better? Well, here are the parts that suck the most, addressing those should have the biggest positive impact. Short and to the point. Oh you didn't want me to tell you that? Why did you fucking asked then?
Re: In Defense of Simple Architectures (2022)
#380Early in my career one of the IT guys told me that one of the people on staff was "a technical magpie". I looked at him with a raised eyebrow and he said "He has to grab every shiny piece of tech that shows up and add it to the pile". This is where we are. I can't tell you how many times I have seen projects get done just to pad a PM or developers resume. Just because it was the lastest and greatest hot shit thing to…
As a product manager, I am frequently confronted by UX people who declare something as „standard“, a feature that is supposed to be an absolute „must-have“, or else our organisation would loose even the last of our users. Unfortunately, developers tend to accept these things as interesting challenges and (knowingly or not) underestimate the effort and complexity needed to implement it. My very lonesome role in these…
The Wall Street Journal Guide to Information Graphics is an essential reading for anyone who regularly presents data.