The speedup is very noticeable. Like others I'm running the "native-comp" branch of Emacs, since months, without any issue. It's now been merged into trunk and it's going to be the default: https://news.ycombinator.com/item?id=26935401 The only drawback I saw is that compiling Emacs itself takes 3x to 4x longer when compiling the native-comp branch.
Is the speedup during editing or does it also improve startup time?
During editing/usage for sure: it is noticeable. Not that it was slow before but nearly everything now feels really snappy.
As for startup I don't know as I rarely relaunch it but I just tried (only for the native-comp branch):
time emacs -Q -eval '(kill-emacs)'
gives 160 ms. Or launching Emacs with -Q and then calling emacs-init-time gives basically the same (-Q bypasses the config files).Starting with my entire config which is quite beefy takes 1.2s. I could probably speed it up but haven't really looked into optimizing Emacs startup in a while.