21–30 of 32 posts
This is the wrong direction. I can say this having written a monkeypatching management library in Ruby a long time ago.
This feels too much like breaking the guarantee sticker of a vendor code, and if your vendor pushes updates weekly, or daily, you are stuck pushing updates to your shimmed code, which officially becomes "unnamed fork". Even for tests, let's say that they changed an input type, I don't see an improvement in my workflow: I still need to update my "unnamed fork". At least with a fork I get to see the whole git history,…
Your patch "with retries" might never be accepted, and maintaining any kind of fork(s) or "out-of-tree patches" is not as integrated into the programming environment. Being able to say "assert WrappedLoginLibrary().login(), '...with retries...'" keeps you testable and "in" the language proper.
For context: one of the several projects I'm working on right now is an automated extraction system for literate-code-style documentation in python. This isn't the place nor time to talk about the why of it (especially compared to other existing similar solutions). The important thing is the how: it uses a temporary import hook to stub out all module imports, allowing the docs generator to process each module indepen…
This solution is interesting, as it provides the patched code as if it were a new package, indendant of the existing one you have installed, like vendoring, but without the burden of it.
In case you want to be the only one seing your patch, this is great. It also makes the whole maintenance easier, as you don't have to wonder if you patch it at the right time or in the right way. MK can fail in many subtle edge cases.
Inheritance, particularly, is a great Mk pitfall I expect this method to transparently work with.
For context: one of the several projects I'm working on right now is an automated extraction system for literate-code-style documentation in python. This isn't the place nor time to talk about the why of it (especially compared to other existing similar solutions). The important thing is the how: it uses a temporary import hook to stub out all module imports, allowing the docs generator to process each module indepen…
Monkey patching an object attribute, such as a method or a function of a module, may affect 3rd party libraries code that use said object. This solution is interesting, as it provides the patched code as if it were a new package, indendant of the existing one you have installed, like vendoring, but without the burden of it. In case you want to be the only one seing your patch, this is great. It also makes the whole m…
I mean if you really need super strong isolation, you can always create a copy of the library object; metaprogramming, dynamic classes, etc, all make it really easy to even, say, create a duplicate class object with references to the original method implementations. Or decorated ones. Or countless other approaches.
My point isn't that I don't see problems that could be solved by this; my point is that I can't think of any problems that this solves, that wouldn't be better solved by things that don't do any innards-fiddling in what is arguably the most sharply-edged part of python: packaging and imports.
And speaking from experience... if you think patching can fail in subtle edge cases, then I've got some bad news for you re: import hooks.
At the end of the day, people who might use this library are looking for a solution to a particular problem. When documenting things, it's really important to be explicit about the pros and cons of your solution, from the perspective of someone with a particular problem, and not from the perspective of someone who's built a particular solution. If I need to drive a nail, and you're selling wrenches, I don't want to hear about all of the features of your wrenches; I want to know if your wrench can drive my nail, and why I would ever want to choose it instead of a hammer.
I can think of a lot of differently-shaped metaphorical nails that fall under the broad umbrella of "I need to change some upstream code but don't want to maintain a fork". And I can think of a whole lot of python-specific specialty hammers that can accomplish that task. But I still can't think of a signle situation where using import hooks to solve the problem is doing anything other than throwing a wrench into a very delicate gearbox. That is the explanation I would need, if I were in the market for such a solution, to evaluate modshim as a potential approach.
It's much cleaner than monkey patching, and it will more likely detect if an update conflicts with your patching.
I've used it by packaging everything through nix, but that can be cumbersome.
This is the wrong direction. I can say this having written a monkeypatching management library in Ruby a long time ago.
Can you elaborate? I’m just curious. I’m still not sold on monkey patching at all (it largely seems like a way to get around writing modular code).
Note: Have not touched in > 13 years, so there's that lol
At the time I was working on a million+-line Ruby codebase at Desk.com. We were in a situation where people were monkeypatching out bugs in dependent libraries that weren't patched upstream yet, and then forgetting about them, and they would eventually end up causing problems that were difficult to run down. So I wrote this tool to basically organize and "vet" the monkeypatches BEFORE they were applied, using a runtime test (at app startup/stack load) to see if it was still necessary to apply, and if not, write a warning to stderr. Otherwise, it would re-apply it (but also notify to stderr). I wanted these patches to be a bit noisy so that they wouldn't be forgotten about and so they would be removed once no longer necessary.
Of course, what I'd NOW do instead is 1) fork the library into my own repo, 2) apply the patch, 3) tell my app to use my fork, 4) have some rigorous process to re-depend back on upstream somehow once things had settled again. That would keep things more easily traceable.
I more or less left Ruby and have been doing Elixir for years now, because I realized that functional/declarative is the way to go for long-term code maintenance (and general ease of testing/debugging, and lower production of bugs per LOC written, etc.).
Regarding the naming, I thought that throwing a bunch of monkeypatches at a codebase was kind of like dropping metal balls in a pachinko machine, in that the outcome would be non-deterministic (we avoid non-determinism at all costs!). For example, there was no way to guarantee the order that they would apply in, in case there were conflicts (which also couldn't be detected at the time of application, only via specific unit testing)... If I was smart (I don't remember if I did this or not), pachinko would intentionally apply the patches in a randomized order, so that latent dependency issues would be floated to the top...
For context: one of the several projects I'm working on right now is an automated extraction system for literate-code-style documentation in python. This isn't the place nor time to talk about the why of it (especially compared to other existing similar solutions). The important thing is the how: it uses a temporary import hook to stub out all module imports, allowing the docs generator to process each module indepen…
I've written a Jupyter client for the terminal (euporie), for which I've had to employ monkey-patching of various third-party packages to achieve my goals and avoid forking those packages. For example, I've added terminal graphics support & HTML/CSS rendering to prompt-toolkit (a Python TUI library), and I've changed aiohttp to not raise errors on non-200 http responses. These are things the upstream package maintainers do not want to maintain or will not implement, and likewise I do not want to maintain forks of these packages.
So far I've got away with monkey-patching, but recently I implemented a kernel for euporie which runs on the local interpreter (the same interpreter as the application itself). This means that my patches are exposed to the end user in a REPL, resulting in potentially unexpected behaviour for users when using certain 3rd party packages in Python through euporie. Modshim will allow me to keep my patched versions isolated from the end user.
Additionally, I would like to publish some of my patches to prompt_toolkit as a new package extending prompt_toolkit, as I think they would be useful to others building TUI applications. However, the changes required need to be deeply integrated to work, which would mean forking prompt_toolkit (something I'd like to avoid). modshim will make it possible for me to publish just my modifications.
Perhaps it's a somewhat niche use-case, and modshim is not something most Python users would ever need to use. I just thought it was something novel enough to be of interest to other HN users.
> messing around with import internals in python is extremely tricky to get right
This is true! modshim has been the most complicated thing I've written by some way!
Earlier quoted context omitted.
> Because our enhanced Session class now enables retries by default, we don't even need to instantiate it directly. modshim's AST rewriting ensures that internal references within the requests module are updated. This means convenience functions like requests.get() will automatically use our enhanced Session class This seems to explicitly handle the case you are interested in - automatically updating library-internal…
That's talking about internal imports (and static - as much as python supports - ones at that), not external ones. If A is my application, B is buggy, and C is some other library, consider: # A.py monkeypatch_B() import C # C.py B = __import__('B') # B.py bugs()
What Python versions have you tested on, OP? Good license choice, by the way.