171–180 of 181 posts
I've been writing services... as libraries first. Then just wrap the library in a very simple `main()`: ``` #include "servicelib.hpp" int main(int argc, char argv) { return servicelib{argc, argv}.run(); } ``` The library can be re-used in other apps or services. Then the whole damn library is unit-testable with any arguments you throw at it. Got an OS where argv may be null? You can unit test that. Got a user who dec…
Earlier quoted context omitted.
> It's all fun and games until you hit version n+1 or n+2, and often realize how slow many customers are to upgrade the library. How is that different with services? I don't develop services, but I can imagine that before breaking the services, you have to poll with your biggest customers and you don't break until they are ready to move to the new version. The alternative I can imagine is to keep providing the old se…
Simple answer is YOU control the deploy of a server in miroservices, so YOU decide when it goes live. You aren't playing spreadsheet telephone at scale to get everyone even im an internal org to update. If you're smart you have your callers send version information and you usually just add fields, only removals or renames cause version issues. If you're having everyone use a library they control when they update the…
You write a library, then wrap a thin service interface around it. Distribute the lib as needed. Publish the service as needed. Maintain the library, modifying the service on as it is affected. So this effectively comes down to “write a library” as tfa suggests. But there’s no reason the library can’t then be the core of a service.
You write a library, then wrap a thin service interface around it. Distribute the lib as needed. Publish the service as needed. Maintain the library, modifying the service on as it is affected. So this effectively comes down to “write a library” as tfa suggests. But there’s no reason the library can’t then be the core of a service.
You can also start calling your library - a "distributable, embedded, natively consumable micro-service" and then other folks start using it. (Believe it or not - this is what some teams in my org started calling a traditional library)
I've been writing services... as libraries first. Then just wrap the library in a very simple `main()`: ``` #include "servicelib.hpp" int main(int argc, char argv) { return servicelib{argc, argv}.run(); } ``` The library can be re-used in other apps or services. Then the whole damn library is unit-testable with any arguments you throw at it. Got an OS where argv may be null? You can unit test that. Got a user who dec…
Can you give details on how you do “automatic tagging”?
Earlier quoted context omitted.
And with libraries you can't make breaking changes? The fact that I don't need to recompile everything whenever libcurl or libssl has a security fix proves otherwise.
That's not a great example given OpenSSL versions are famously backwards-incompatible. Older versions get ABI-compatible security patches because people put in the time to backport all of them to every version still supported, in many cases by distro package maintainers. It's exactly the situation libraries should generally avoid, because there are maintainance costs for older versions of the library as well as migra…
Earlier quoted context omitted.
And yet.. I wholeheartedly agree with your sentiment. Unfortunately, the obvious economic incentive to effectively paywalling code through the Internet, is so strong that on this very website (that was supposed to have hacker mentality) most of the comments are in favour of it, because “you have to make a living”… Yes, I think we are lost.
If you have any feasible ideas that could plausibly work where people don't "have to make a living" so that we can find ourselves and not be lost, we're all ears. Until then we're stuck in a place where people need money to pay for frivolous things like "food" or "rent" or "transportation". Still, in the face of that, there's a site called GitHub where people do freely share code, despite the economic misincentives,…
Earlier quoted context omitted.
What I’ve heard from friends working at places where the monorepo model has worked well, is that it also involved a culture shift such that the burden of upgrading the consuming services is put on the shoulders of the _providers_ of the library, rather than the consumers. This implicitly brings along some benefits, like making the providers not cause excessive/unnecessary version churn and ensuring easy upgrade paths…
Wow, that is certainly a _huge_ culture shift. It certainly explains why it's not sitting right with me - while I do think that "providers not caus[ing] excessive/unnecessary version churn and ensuring easy upgrade paths" is a good thing in isolation, it seems like it would be massively outweighed by the downsides of forcing a library provider to maintain familiarity with all their consumers' codebases and business a…
I’m curious about this aspect that you mentioned:
> forcing a library provider to maintain familiarity with all their consumers' codebases and business areas
In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to accomplish transparent upgrades of any of their library’s API changes in a consuming codebase. Or if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues—before it’s “too late” to rethink because the library update was formally released :) Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn, or if it’s a non-starter and a pun upstream solution that works for both parties needs to be discussed/implemented.
Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue. And when you multiply that by N library producers in a company of a given scale, it’s easy for app/service dev teams to have a lot of their time non-productively taken up just trying to keep up with the constant treadmill of poorly handled upstream changes. So removing the externality effect by having the producers have some “skin in the game” can really help improve the experience for the consuming teams.
hth
Earlier quoted context omitted.
Wow, that is certainly a _huge_ culture shift. It certainly explains why it's not sitting right with me - while I do think that "providers not caus[ing] excessive/unnecessary version churn and ensuring easy upgrade paths" is a good thing in isolation, it seems like it would be massively outweighed by the downsides of forcing a library provider to maintain familiarity with all their consumers' codebases and business a…
Cheers! I’m curious about this aspect that you mentioned: > forcing a library provider to maintain familiarity with all their consumers' codebases and business areas In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to accomplish transparent upgrades of any of their library’s API changes in a consuming codebase. Or if there are breaking API changes that make i…
> I’m curious about this aspect that you mentioned:
>> forcing a library provider to maintain familiarity with all their consumers' codebases and business areas
> In my experience, this hasn’t been an issue—the library authors are inherently intimately familiar with how to [make changes] in a consuming codebase
(See my comments below where I'm not sure what a "transparent upgrade" means)
Wow - that's genuinely surprising (and impressive!) to me. This implies that, for all N consumers of the library, the library team are already familiar (without having to look it up - otherwise, that's added ramp-up burden) with the consumers' preferred code style, testing expectations, commit and branching structure, personal development environment setup, running integration tests against personal environments, how to announce PRs (if at all), and so on - all the "other stuff" that goes "around" making the actual code change. In an ideal world, a lot of those things _should_ be trivially clear/provided (code-style provided by auto-linters, testing carried out automatically during PR, etc.) - but practically speaking they very rarely are. This seems like a _lot_ of extra knowledge which is (in my opinion - though clearly not in a monorepo-mindset opinion) outside of their area of ownership and expertise that they have to carry around on a day-to-day basis, on the off-chance that they make a breaking change to their library (which, as I think we both agree, should hopefully be rare).
Even in the best-reasonable case, where all of those things are well-documented (and they _should_ be, because an outside contributor is indistinguishable from a New Hire), that's still a fair bit of reading, ramp-up, and workspace-setup that the library team needs to do - vs. simply telling the consuming team "here's the general shape of change that you need to make, apply this as appropriate to your own situation - ask us if anything's unclear"
OK, on to the direct reply:
---
So, to be clear about terminology, I think we're only really concerned with breaking changes, here - i.e. those for which, if the consumer moves from consuming v- to v-, they will also need to change something about how their code calls the library. The most obvious ways I can think of for this to be necessary are:
* The type signature changes in a non-backwards compatible way - i.e. adding required parameters (including making previously-optional parameters required), changing the type of a parameter, or changing the type of the return value to something incompatible with the previous type ("incompatible" is hand-waving because different type systems think about this differently, but I think my meaning is clear in general) * The name of the method itself is changed, or the entire call pattern is changed (e.g. instead of `foo.execute(command)`, you now have to call `foo.prepareCommand(command); foo.execute()`)
For any library change which _isn't_ a breaking change (i.e. one for which the consumers could keep their call-site code exactly the same, and expect the same behaviour) I believe this whole discussion is moot - both perspectives would agree that nothing would need to be done in the consuming code. My perspective would say that the consuming team has nothing to do, and the monorepo perspective would say that the library team has nothing to do - but, either way, `0 === 0`, even JavaScript agrees on that :)
So, then, I might need a bit of elaboration on what a "transparent upgrade" is, since my initial naïve interpretation (a response to a non-breaking change - one in which the call-site code can remain the same) cannot be the thing at issue, since - well, yeah, of course a library team knows how to make a no-op change to their consumers ;) might "transparent upgrade" instead mean "a change where all the data required to make the library call is available at the call-site, but it needs to be reshaped, retyped, or otherwise reorganized before being passed to the library function in v-"? If so, then...yeah, I can see it being _nice_ if the library owners were to publish some deterministic code-diff tool which would transform all their consumers' call-sites to use the newly-reshuffled data, but honestly I'd expect them instead to do that "reshuffling" _internally_ to their function and not even publish the new "breaking but with reorganized data" version in the first place. Maybe there's a good case for this "data-reshuffling" that I'm missing, though.
Maybe a pseudocode example of a "transparent upgrade" would help me understand what it is?
---
> if there are breaking API changes that make it impossible/infeasible to maintain the current consumer behavior, that’s a very useful signal that it’s time for the library producer to go back to the drawing board because the proposed library update has issues[...]Or at least, helps trigger a conversation between the producer and consumers to see if the breakage can be accommodated without too much heartburn
Yeah, I think we're generally on the same page, here; just assigning differing weights, priorities, and likelihoods to breaking changes. Wherever possible, for sure, aim for non-breaking changes; and where breaking changes are necessary, make sure you have had a full discussion with consumers to see whether the churn you're going to create is justified (and maybe consider providing a long period of support for the previous version so that consumers have a long period to upgrade and to amortize that upgrade-churn) - but, if a breaking change _is_ truly justified (is worth more than the upgrade-burden it will cost), don't shy away from it!
> Overall, it really reduces the “library producer in their ivory tower releases changes without fully considering the downstream impacts, causing a crapload of burden/schedule slip for XX teams” issue.
Maybe I've just been very lucky to have barely run into that problem! :) In ~10 years in my previous role, I'm struggling to think of a handful of times where a major version upgrade was any more than a trivial process - and the few times that it _was_ a meatier change, I was perfectly content with being given a guide or walkthrough on how to migrate. But, yeah - if cavalier library teams making profligate and unjustified breaking changes _is_ a problem being faced, then I can see that that culture of "you break it, you fix it" would rein them in.
---
FWIW, this has _already_ really helped me to understand where a coworker was coming from in an (unrelated) proposal which, to me and all other colleagues, initially looked nonsensical. After chewing on it a bit, I realized he's coming at this from the same perspective - "people who make changes to things that other people depend on, should bear the pain of updating those other people's things to work with the new dependency-thing". Gotta say I am still extremely-unconvinced (I'm still on the side of "ceteris paribus, an owner making a change is more efficient than mandating that someone else make the change; so I'd prefer to find other ways of forcing library owners to confront the externalities of disruptive breaking changes that don't introduce that inefficiency"), but it does very much help to understand where he's coming from - thanks!