Side note, I like the casual technical writing style used here, with the main points summarized along the way. Easily digestible and I can go back and get the details in the main text at any point if I want.
Thank you, really appreciate this!
Tracking developer build times to decide if the M3 MacBook is worth upgrading
61–70 of 432 posts
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#62Does anyone have any anecdoctal evidence around the snappiness of VsCode with Apple Silicon? I very begrudgingly switched over from SublimeText this year (after using it as my daily driver for ~10yrs). I have a beefy 2018 MBP but VScode just drags. This is the only thing pushing me to upgrade my machine right now but I'd be bummed if there's still not a significant improvement with an m3 pro.
On my 2019 MBP, I found VSCode performance poor enough to be annoying on a regular basis, enough so that I would frequently defer restarting it or my machine to avoid the lengthy interruption. Doing basically anything significant would have the fans running full blast pretty much constantly.
On my M2 Max, all of that is ~fully resolved. There is still some slight lag, and I have to figure it’s just the Electron tax, but never enough to really bother me, certainly not enough to defer restarting anything. And I can count the times I’ve even heard the fans on one hand… and even so, never for more than a few seconds (though each time has been a little alarming, just because it’s now so rare).
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#63My personal research for iOS development, taking the cost into consideration, concluded: - M2 Pro is nice, but the improvement over 10 core (8 perf cores) M1 Pro is not that large (136 vs 120 s in Xcode benchmark: https://github.com/devMEremenko/XcodeBenchmark ) - M3 Pro is nerfed (only 6 perf cores) to better distinguish and sell M3 Max, basically on par with M2 Pro So, in the end, I got a slightly used 10 core M1 P…
The M3 Pro being nerfed has been parroted on the Internet since the announcement. Practically it’s a great choice. It’s much more efficient than the M2 Pro at slightly better performance. That’s what I am looking for in a laptop. I don’t really have a usecase for the memory bandwidth…
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#64This is a great write-up and I love all the different ways they collected and analyzed data.
That said, it would have been much easier and more accurate to simply put each laptop side by side and run some timed compilations on the exact same scenarios: A full build, incremental build of a recent change set, incremental build impacting a module that must be rebuilt, and a couple more scenarios.
Or write a script that steps through the last 100 git commits, applies them incrementally, and does a timed incremental build to get a representation of incremental build times for actual code. It could be done in a day.
Collecting company-wide stats leaves the door open to significant biases. The first that comes to mind is that newer employees will have M3 laptops while the oldest employees will be on M1 laptops. While not a strict ordering, newer employees (with their new M3 laptops) are more likely to be working on smaller changes while the more tenured employees might be deeper in the code or working in more complicated areas, doing things that require longer build times.
This is just one example of how the sampling isn’t truly as random and representative as it may seem.
So cool analysis and fun to see the way they’ve used various tools to analyze the data, but due to inherent biases in the sample set (older employees have older laptops, notably) I think anyone looking to answer these questions should start with the simpler method of benchmarking recent commits on each laptop before they spend a lot of time architecting company-wide data collection
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#65[flagged]
WFH is effectively work from anywhere. I can take a month long working vacation and save a vacation day by working on the airplane. I'd be very annoyed if my work chained me to my home desk.
Most companies are cracking down on this kind of behaviour nowadays, in case you didn't get the memo.
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#66[flagged]
If you genuinely don’t understand any of the perks of having a portable workstation, that’s one thing, but this comes across as a snide deliberate dismissal without even acknowledging them.
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#67My personal research for iOS development, taking the cost into consideration, concluded: - M2 Pro is nice, but the improvement over 10 core (8 perf cores) M1 Pro is not that large (136 vs 120 s in Xcode benchmark: https://github.com/devMEremenko/XcodeBenchmark ) - M3 Pro is nerfed (only 6 perf cores) to better distinguish and sell M3 Max, basically on par with M2 Pro So, in the end, I got a slightly used 10 core M1 P…
That's interesting you saw less of an improvement in the M2 than we saw in this article.
I guess not that surprising given the different compilation toolchains though, especially as even with the Go toolchain you can see how specific specs lend themselves to different parts of the build process (such as the additional memory helping linker performance).
You're not the only one to comment that the M3 is weirdly capped for performance. Hopefully not something they'll continue into the M4+ models.
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#68My personal research for iOS development, taking the cost into consideration, concluded: - M2 Pro is nice, but the improvement over 10 core (8 perf cores) M1 Pro is not that large (136 vs 120 s in Xcode benchmark: https://github.com/devMEremenko/XcodeBenchmark ) - M3 Pro is nerfed (only 6 perf cores) to better distinguish and sell M3 Max, basically on par with M2 Pro So, in the end, I got a slightly used 10 core M1 P…
My experience was similar: In real world compile times, the M1 Pro still hangs quite closely to the current laptop M2 and M3 models. Nothing as significant as the differences in this article.
I could depend on the language or project, but in head-to-head benchmarks of identical compile commands I didn’t see any differences this big.
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#69My personal research for iOS development, taking the cost into consideration, concluded: - M2 Pro is nice, but the improvement over 10 core (8 perf cores) M1 Pro is not that large (136 vs 120 s in Xcode benchmark: https://github.com/devMEremenko/XcodeBenchmark ) - M3 Pro is nerfed (only 6 perf cores) to better distinguish and sell M3 Max, basically on par with M2 Pro So, in the end, I got a slightly used 10 core M1 P…
That's interesting you saw less of an improvement in the M2 than we saw in this article. I guess not that surprising given the different compilation toolchains though, especially as even with the Go toolchain you can see how specific specs lend themselves to different parts of the build process (such as the additional memory helping linker performance). You're not the only one to comment that the M3 is weirdly capped…
That's what Xcode benchmarks seem to say.
Yep, there appears to be no reason for getting M3 Pro instead of M2 Pro, but my guess is that after this (unfortunate) adjustment, they got the separation they wanted (a clear hierarchy of Max > Pro > base chip for both CPU and GPU power), and can then improve all three chips by a similar amount in the future generations.
Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading
#70[flagged]
True. Desktops are vastly superior in terms of specs and cooling.