Live data from Hacker News

Tracking developer build times to decide if the M3 MacBook is worth upgrading

incident.io

81–90 of 432 posts

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#81

This is a great write-up and I love all the different ways they collected and analyzed data. That said, it would have been much easier and more accurate to simply put each laptop side by side and run some timed compilations on the exact same scenarios: A full build, incremental build of a recent change set, incremental build impacting a module that must be rebuilt, and a couple more scenarios. Or write a script that…

I totally agree with your suggestion, and we (I am the author of this post) did spot-check the performance for a few common tasks first.

We ended up collecting all this data partly to compare machine-to-machine, but also because we want historical data on developer build times and a continual measure of how the builds are performing so we can catch regressions. We quite frequently tweak the architecture of our codebase to make builds more performant when we see the build times go up.

Glad you enjoyed the post, though!

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#82

[flagged]

The medium is the message here, the macbook is just bait.

The pure LLM is not effective on tabular data (so many transcripts of ChatGPT apologizing it got a calculation wrong.). To be working as well as it seems to work they must be loading results into something like a pandas data frame and having the agent write and run programs on that data frame, tap into stats and charting libraries, etc.

I’d trust it more if they showed more of the steps.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#83

I wish I needed a fast computer. It’s the CI/CD that’s killing me. All this cloud stuff we use - can’t test anything locally anymore. Can’t use the debugger. I’m back to glorified fmt.Printf statements that hopefully have enough context that the 40 min build/deploy time was worth it. At least it’s differential ¯\_(ツ)_/¯ All I can say is “I compiles… I think?” The unit tests are mostly worthless and the setup for send…

Yeah everything you just said is exactly why we care so much about a great local environment. I've not seen remote tools approach the speed/ease/flexibility you can get from a fast local machine yet, and it makes a huge difference when developing.

In the back of my mind I’m worried that our competitors have a faster software development cycle.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#84
1. If, and only if, you are doing ML or multimedia, get a 128GB system and because of the cost of that RAM, it would be foolish not to go M3 Max SoC (notwithstanding the 192GB M2 Ultra SoC). Full Stop. (Note: This is also a good option for people with more money than brains.)

2. If you are doing traditional heavyweight software development, or are concerned with perception in an interview, promotional context or just impressing others at a coffee shop, get a 32GB 16” MBP system with as large a built-in SSD as you can afford (it gets cheaper per GB as you buy more) and go for an M2 Pro SoC, which is faster in many respects than an M3 Pro due to core count and memory bandwidth. Full Stop. (You could instead go 64GB on an M1 Max if you keep several VMs open, which isn’t really a thing anymore (use VPS), or if you are keeping a 7-15B parameter LLM open (locally) for some reason, but again, if you are doing much with local LLMs, as opposed to being always connectable to the 1.3T+ parameter hosted ChatGPT, then you should have stopped at #1.)

3. If you are nursing mature apps along, maybe even adding ML, adjusting UX, creating forks to test new features, etc.. then your concern is with INCREMENTAL COMPILATION and the much bigger systems like M3 Max will be slower (bc they need time to ramp up multiple cores and that’s not happening with bursty incremental builds), so might as well go for a 16GB M1 MBA (add stickers or whatever if you’re ashamed of looking like a school kid) and maybe invest the savings in a nice monitor like the 28” LG DualUp (bearing in mind you can only use a single native-speed external monitor on non-Pro/Max SoCs at a time). (In many cases, can even use an 8GB M1 MBA for incremental builds because, after loading the project, the MacOS memory compressor is really good and the SSD is really fast and you can use a real device instead of a Simulator. But do you want any M2 MBA? No, it has inferior thermals, is heavier, larger, fingerprints easy, lack’s respect and the price performance doesn’t make sense given the other options. Same goes for 13” M1/M2 Pro and all M3 Pro.)

Also, make sure you keep hourly (or better) backups on all Apple laptops WHILE CODING. There is a common failure scenario where the buck converter that drops voltage for the SSD fails, sending 13VDC into the SSD for long enough to permanently destroy the data on it. https://youtu.be/F6d58HIe01A

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#85

MacBooks are a waste of money. You can be just as productive with a machine just as fast for 1/2 the price that doesn't include the Apple Tax. Moreover, if your whole stack (plus your test suite) doesn't fit in memory, what's the point of buying an extremely expensive laptop? Not to mention constantly replacing them just because a newer, shinier model is released? If you're just going to test one small service, that…

It's OK to just not like Apple. You don't have to justify your own feelings with pejoratives towards other peoples' choice of laptop.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#86
post #52

Earlier quoted context omitted.

[flagged]

Not everyone needs multiple 4090s to do their work. If you do, your options are limited.

And even if you do, you still might be better off running the GPU-bound tasks somewhere else and using a laptop to interact with them remotely.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#87
post #10

Does anyone have any anecdoctal evidence around the snappiness of VsCode with Apple Silicon? I very begrudgingly switched over from SublimeText this year (after using it as my daily driver for ~10yrs). I have a beefy 2018 MBP but VScode just drags. This is the only thing pushing me to upgrade my machine right now but I'd be bummed if there's still not a significant improvement with an m3 pro.

If you find your compiles are slow, I found a bug in vscode where builds would compile significantly faster when the status bar and panel are hidden. Compiles that took 20s would take 4s with those panels hidden.

https://github.com/microsoft/vscode/issues/160118

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#88

Earlier quoted context omitted.

> - M2 improves performance by about 60% This is the most shocking part of the article for me since the difference between M1 and M2 build times has been more marginal in my experience. Are you sure the people with M1 and M2 machines were really doing similar work (and builds)? Is there a possibility that the non-random assignment of laptops (employees received M1, M2, or M3 based on when they were hired) is showing…

The build events track the files that were changed that triggered the build, along with a load of other stats such as free memory, whether docker was running, etc. I took a selection of builds that were triggered by the same code module (one that frequently changes to provide enough data) and compared models on just that, finding the same results. This feels as close as you could get for an apples-to-apples compariso…

> apples-to-apples comparison

No pun intended. :)

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#89
I feel like there is a correlation between fast-twitch programming muscles and technical debt. Some coding styles that are rewarded by fast compile times can be more akin to "throw it at the wall, see if it sticks" style development. Have you ever been summoned to help a junior colleague who is having a problem, and you immediately see some grievous errors, errors that give you pause. You point the first couple out, and the young buck is ready to send you away and confidently forge ahead, with no sense of "those errors hint that this thing is really broken".

but we were all young once, I remember thinking the only thing holding me back was 4.77MHz

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#90
Solid analysis.

A word of warning from personal experience:

I am part of a medium-sized software company (2k employees). A few years ago, we wanted to improve dev productivity. Instead of going with new laptops, we decided to explore offloading the dev stack over to AWS boxes.

This turned out to be a multi-year project with a whole team of devs (~4) working on it full-time.

In hindsight, the tradeoff wasn't worth it. It's still way too difficult to remap a fully-local dev experience with one that's running in the cloud.

So yeah, upgrade your laptops instead.

Post reply on HN