Hot-code reloading on macOS/arm64 with Zig
jakubkonka.com
Hot-code reloading on macOS/arm64 with Zig
1–10 of 57 posts
Re: Hot-code reloading on macOS/arm64 with Zig
#2Why not simply use MAP_JIT?
There is no need to do these weird tricks, plus you’re putting your process and the rest of the system at risk by elevating privilege and using task-for-pid.
Re: Hot-code reloading on macOS/arm64 with Zig
#3This seems like it would just crash or make your program get into an inconsistent state. What happens if you are executing a transaction but mid transaction your binary gets updated and your transaction now has done half of what's needed by the old version and half of what's needed by the new version. I don't think it would be that hard to get into an inconsistent state.
Re: Hot-code reloading on macOS/arm64 with Zig
#4> Instead of having your program managed by a running side-by-side loader program, what if the compiler would “simply” update the memory of the running process? You will most inevitably think I have gone completely crazy
This isn't crazy - this is how just-in-time compilers work.
Re: Hot-code reloading on macOS/arm64 with Zig
#5This seems like it would just crash or make your program get into an inconsistent state. What happens if you are executing a transaction but mid transaction your binary gets updated and your transaction now has done half of what's needed by the old version and half of what's needed by the new version. I don't think it would be that hard to get into an inconsistent state.
If that matters, you probably shouldn't be using hot code reloading regardless of the language or implementation.
Re: Hot-code reloading on macOS/arm64 with Zig
#6When I see hot code reloading, I think there are probably some server frameworks that are waiting to get this feature to reduce feedback loop to a few tenths of second of build… Nice overview!
Re: Hot-code reloading on macOS/arm64 with Zig
#7> Instead of having your program managed by a running side-by-side loader program, what if the compiler would “simply” update the memory of the running process? You will most inevitably think I have gone completely crazy This isn't crazy - this is how just-in-time compilers work.
It is a bit crazy if you do it for hot code reloading, though, as the logic has now changed.
In the case of JIT, it may swap the bytecode but the flow of execution remains the same.
Hot code reloading usually requires a different approach.
Re: Hot-code reloading on macOS/arm64 with Zig
#8Why not simply use MAP_JIT? There is no need to do these weird tricks, plus you’re putting your process and the rest of the system at risk by elevating privilege and using task-for-pid.
Hey, author of the article here. Thanks for the suggestion! I actually didn't know about the MAP_JIT flag to mmap before and will defo consider it. As to elevating your privs - this is just a temp solution until I work out how to add this entitlement https://developer.apple.com/documentation/bundleresources/en... to the Zig compiler. I wrote the default Zig's MachO linker from scratch and it can embed the adhoc code signatures no probs, but haven't worked out baking the entitlements in yet.
Re: Hot-code reloading on macOS/arm64 with Zig
#9Why not simply use MAP_JIT? There is no need to do these weird tricks, plus you’re putting your process and the rest of the system at risk by elevating privilege and using task-for-pid.
Hey, author of the article here. Thanks for the suggestion! I actually didn't know about the MAP_JIT flag to mmap before and will defo consider it. As to elevating your privs - this is just a temp solution until I work out how to add this entitlement https://developer.apple.com/documentation/bundleresources/en... to the Zig compiler. I wrote the default Zig's MachO linker from scratch and it can embed the adhoc code…
The only entitlement that is relevant here is get-task-allow. But that would allow anyone to get a control task port for your application and do with it as they may. This functionality was not designed to be used in production — except for debugging.
The debugger entitlement is even more powerful, but once again since you’re modifying your own memory you don’t need it.
Re: Hot-code reloading on macOS/arm64 with Zig
#10This seems like it would just crash or make your program get into an inconsistent state. What happens if you are executing a transaction but mid transaction your binary gets updated and your transaction now has done half of what's needed by the old version and half of what's needed by the new version. I don't think it would be that hard to get into an inconsistent state.
Yep, you are absolutely right, and hence why this is a proof-of-concept after all and a start really. With the proper tweaks to the linker though I am sure it would be possible orchestrate updates in such a way as to not clobber any existing global state or cause weird UB (by overwriting the code a thread is currently executing, etc.). Either way, thanks so much for the feedback!