Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jurigged – Hot code reloading for Python

github.com

Post reply on HN
Print view View original

21–30 of 33 posts

Re: Jurigged – Hot code reloading for Python

#21

Post by emj »

julvo wrote: post #18
emj wrote: post #12

Looks more usable than the reloading library which does a similar thing, going to be interesting to use them together. I found Reloading when it was discussed in 2019: https://news.ycombinator.com/item?id=21268324

Hey author of reloading here - cool to see this mentioned. Would love to know how reloading could be more usable for you. Since the post on HN in 2019, reloading can also be used as a decorator to load functions from source before execution. For some use cases that may work better than the reloading loop.

I still use the version from 2019, and I usually do not have much code in tight loops, but rather need to reload different parts of a much bigger application so I tend to decorate the wrong parts of the code. I have no complaints about Reloading, just admiration using the AST module like that was genious.

Re: Jurigged – Hot code reloading for Python

#22

Post by dnautics »

ohples wrote: post #5
musingsole wrote: post #3

Earlier quoted context omitted.

In a proper managed environment (whatever that may mean...), this could be useful as a means for continuous integration. Maybe not worth all the hassle, but maybe there's a usecase where this would be a beloved silver bullet.

Hot code reloading (ala Erlang) is often used in embedded devices for control code where restarting is very expensive. Mostly telco and networking devices.

we use it in prod to instrument code to check up on some internal data, say, from the database, when some forensic analysis is needed

Re: Jurigged – Hot code reloading for Python

#23

Post by adinisom »

Neat!

Attempted to run on Windows and it almost works on my machine.

- Watchdog's win32 implementation watches directories rather than files. If Jurigged is patched to watch the directory instead, this part works.

- When the toplevel code is patched, the thread dispatching Watchdog events stops and won't pick up further filesystem changes. This can happen when changing code at toplevel or sometimes a file update gets picked up as Delete/Add instead which causes Jurigged to patch toplevel.

Still trying to track down why that happens.

Re: Jurigged – Hot code reloading for Python

#25

Post by adinisom »

adinisom wrote: post #23

Neat! Attempted to run on Windows and it almost works on my machine. - Watchdog's win32 implementation watches directories rather than files. If Jurigged is patched to watch the directory instead, this part works. - When the toplevel code is patched, the thread dispatching Watchdog events stops and won't pick up further filesystem changes. This can happen when changing code at toplevel or sometimes a file update gets…

And the reason Jurigged breaks on changes to toplevel is it's re-running the top level in its own thread... including the infinite loop that is your application.

This makes sense: it has to re-run the module's code. As long as you don't touch the module that kicks off your application it'll work fine. How can we make it work in the same module that runs the app?

One approach that seems to work is to use the: if __name__ == "__main__" guard on your app code, then modify Jurigged to change __name__ to something else upon reloading. I'm not familiar enough with Python internals to tell if changing __name__ is kosher.

Another possibility is to examine the stack and stop executing code when reaching the root of the current stack... although there's no guarantee the code has been running long enough to reach the program's infinite loop such as when the file was modified a split second after the program started.

Re: Jurigged – Hot code reloading for Python

#27

Post by Too »

jfrunyon wrote: post #24

It seems like this is just watching a file for changes and then calling reload on it? What am I missing?

See the How it works and the following Comparision sections of the readme. It's a bit more advanced than simply reloading.

Reimporting a module would likely work most of the times and be a simpler solution but i think it does not affect already created instances of classes, which this solution seem to do.

Re: Jurigged – Hot code reloading for Python

#28

Post by elcomet »

aidos wrote: post #15

If you use iPython you can enable auto reloading for all your sessions. It works really well - I've been using it for years. You need to know the limitations, but on the whole it give huge performance gains when I'm coding. To enable it add the following to your ipython_config.py: c = get_config() c.InteractiveShellApp.exec_lines = ['%autoreload 2'] c.InteractiveShellApp.extensions = ['autoreload']

I do this too and it's amazing.

I also import common packages and classes that I use regularly, such as numpy, os, math, collections...

Re: Jurigged – Hot code reloading for Python

#29

Post by breuleux »

adinisom wrote: post #25
adinisom wrote: post #23

Neat! Attempted to run on Windows and it almost works on my machine. - Watchdog's win32 implementation watches directories rather than files. If Jurigged is patched to watch the directory instead, this part works. - When the toplevel code is patched, the thread dispatching Watchdog events stops and won't pick up further filesystem changes. This can happen when changing code at toplevel or sometimes a file update gets…

And the reason Jurigged breaks on changes to toplevel is it's re-running the top level in its own thread... including the infinite loop that is your application. This makes sense: it has to re-run the module's code. As long as you don't touch the module that kicks off your application it'll work fine. How can we make it work in the same module that runs the app? One approach that seems to work is to use the: if __nam…

I mean, if you watch the example gif in the README, it shows the toplevel module being changed (you can see the if __name__ guard on the screen). It should work.

Basically, Jurigged only re-runs the diff: if you don't change the `if __name__ ...`, it shouldn't run it again. If you change a function in the toplevel module (outside of the guard) it should figure that out by comparing the old AST to the new one and only re-execute that specific function.

Can I see the code you're using for your tests? There might be some other issue.

Re: Jurigged – Hot code reloading for Python

#30

Post by breuleux »

jrmann100 wrote: post #26

I've been using watchgod [0], which seems to work similarly—not sure if jurigged also monitors all included modules, or is cross-platform-friendly. Anyone know the similarities/differences? [0] https://github.com/samuelcolvin/watchgod

I believe watchgod basically restarts the process whenever changes happen. Jurigged does not restart anything: it computes a diff between the old and new code, executes that diff, fudges the code object pointers inside the existing functions, and prays that doesn't break it.

The advantage is that it keeps the program state, so you can edit a program that has a slow startup or salvage a process that has been running for hours. It's especially nice with a repl, if you run jurigged with no arguments or jurigged -m IPython, your changes will be automatically included in your interactive session.

The disadvantage is that it keeps the program state, so if you make changes to init functions for example, it'll most likely just crash. So it depends on your use case. If your process is lightweight and/or you don't care about keeping the current program state, watchgod is probably better for you (I believe there's a similar alternative called hupper that you might want to check out).

Post reply on HN