Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tracking developer build times to decide if the M3 MacBook is worth upgrading

incident.io

Post reply on HN
Print view View original

371–380 of 432 posts

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#371

Post by euos »

karolist wrote: post #307
euos wrote: post #223

I am ex-core contributor Chromium and Node.js and current core contributor to gRPC Core/C++. I am never bothered with build times. There is "interactive build" (incremental builds I use to rerun related unit tests as I work on code) and non-interactive build (one I launch and go get coffee/read email). I have never seen hardware refresh toggle non-interactive into interactive. My personal hardware (that I use now and…

I too come from Blaze and tried to use Bazel for my personal project which involves backend + frontend dockerized, the build rules got weird and niche real quick and I was spending lots of time working with the BUILD files making me question the value against plain old Makefiles, this was 3 years ago, maybe the public ecosystem is better now.

I use Basel for C++. I would write normal dockerfile if I need it. Bazel docker support is an atrocity. For JS builds I also use regular TSC.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#372

Post by joshspankit »

cced wrote: post #122
joshspankit wrote: post #99

Since RAM was a major metric, there should have been more focus on IO Wait to catch cases where OSX was being hindered by swapping to disk. (Yes, the drives are fast but you don’t know until you measure)

This. I’ve routinely got a 10-15GB page file on an M2 pro and need to justify bumping the memory up a notch or two. I’m consistently in the yellow memory and in the red while building. How can I tell how much I would benefit from a memory bump?

Mainly you’d want to watch for 1. disk usage of the page file and 2. IO wait of the overall system.

Here are some commands to get you started: https://stackoverflow.com/questions/15786618/per-process-dis...

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#373

Post by db48x »

danhau wrote: post #347
db48x wrote: post #346

This is bad science. You compared the thing you had to the thing you wanted, and found a reason to pick the thing you wanted. Honesty should have compelled you to at least compare against a desktop–class machine, or even a workstation with a Threadripper CPU. Since you know that at least part of your workload is concurrent, and 14 CPUs are better than 10, why not check to see if 16, 32, or 64 is better still? And the…

I don‘t think this is bad science at all. From the article: > All incident.io developers are given a MacBook which they use for their development work. Non-MacBook machines are apparently not an option, for whatever reason. Comparing against other machines would be interesting, but irrelevant.

So it’s ok for science to be limited by politics?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#374

Post by db48x »

minimaul wrote: post #364
smoldesu wrote: post #358

Earlier quoted context omitted.

My biggest issue with Mac-only shops is that almost nobody actually deploys to Mac. The majority of Mac-only firms I've worked at deploy to x86 Linux and develop in a VM on their Macbook (even pre-M1). Unless your business is writing Mac-native apps, MacOS is probably going to be a second-class deployment platform for you. Even in an ideal scenario where your app already works on ARM, you will be dealing with OS-spec…

These are fair points, and definitely a rough spot. Eg at work we use M1/M2 macs and dev on those using docker - so that’s a Linux VM essentially with some nice tooling wrapped around it. We certainly see differences - mostly around permissions (as docker for Mac doesn’t really enforce any access checks on files on the host), but we also mostly deploy to ARM Linux on AWS. We went Mac only from a mix of Linux, Windows…

> Linux we had issues finding nice laptops

This is the same issue as before. Laptops are shiny so people don’t even bother considering a regular desktop machine. And yet desktops can be so much more powerful simply because they don’t have the thermal and power delivery restrictions that desktops have.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#375

Post by ribit »

epolanski wrote: post #352
ribit wrote: post #319

Interesting idea, but the quality of data analysis is rather poor IMO and I'm not sure that they are actually learning what they think they are learning. Most importantly, I don't understand why they would see such a dramatic increase of sub 20s build times going from M1 Pro to M2 Pro. The real-world performance delta between the two on code compilation workloads is around 20-25%. It also makes little sense to me tha…

> also makes little sense to me that M3 machines have fewer sub-20s builds than M2 machines. M3s have a smaller memory bandwidth, they are effectively a downgrade for some use cases.

You are not going to saturate a 150GB/s memory interface building some code on a six-core CPU... these CPUs are fast, but not that fast.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#376

Post by coldtea »

How does the build time average time being around 400 (seconds?) for M1 and M2 and around 200 for M3 make the M2 a substantial upgrade over M1 but M3 "an incremental improvement on M2"?

Also would it hurt the author if they had kept the same max y-value on all plots?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#377

Post by ParetoOptimal »

rsanek wrote: post #137
LASR wrote: post #90

Solid analysis. A word of warning from personal experience: I am part of a medium-sized software company (2k employees). A few years ago, we wanted to improve dev productivity. Instead of going with new laptops, we decided to explore offloading the dev stack over to AWS boxes. This turned out to be a multi-year project with a whole team of devs (~4) working on it full-time. In hindsight, the tradeoff wasn't worth it.…

This might have to do with scale. At my employer (~7k employees) we started down this path a few years ago as well, and while it has taken longer for remote to be better than local, it now definitively is and has unlocked all kinds of other stuff that wasn't possible with the local-only version. One example is working across multiple branches by switching machines instead of files on local has meant way lower latency…

Can't you locally switch between branches with git worktrees if you make your build cache key on worktree name?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#378

Post by p_j_w »

db48x wrote: post #373
danhau wrote: post #347

Earlier quoted context omitted.

I don‘t think this is bad science at all. From the article: > All incident.io developers are given a MacBook which they use for their development work. Non-MacBook machines are apparently not an option, for whatever reason. Comparing against other machines would be interesting, but irrelevant.

So it’s ok for science to be limited by politics?

They’re looking at their particular use case. That may limit the applicability of this to other people or companies, but that doesn’t make it politics.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#379

Post by daxfohl »

mcny wrote: post #144

> All developers work with a fully fledged incident.io environment locally on their laptops: it allows for a This to me is the biggest accomplishment. I've never worked at a company (besides brief time helping out with some startups) where I have been able to run a dev/local instance of the whole company on a single machine. There's always this thing, or that, or the other that is not accessible. There's always a got…

We used to have that, but it's hard to support as you scale. The level of effort is somewhat quadratic to company size: linear in the number of services you support and in the number of engineers you have to support. Also divergent use cases come up that don't quite fit, and suddenly the infra team is the bottleneck to feature delivery, and people just start doing their own thing. Once that Pandora's Box is opened, it's essentially impossible to claw your way back.

I've heard of largeish companies that still manage to do this well, but I'd love to learn how.

That said, yeah I agree this is the biggest accomplishment. Getting dev cycles down from hours or days to minutes is more important than getting them down from minutes to 25% fewer minutes.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#380

Post by sgjohnson »

j4yav wrote: post #295
Aurornis wrote: post #64

This is a great write-up and I love all the different ways they collected and analyzed data. That said, it would have been much easier and more accurate to simply put each laptop side by side and run some timed compilations on the exact same scenarios: A full build, incremental build of a recent change set, incremental build impacting a module that must be rebuilt, and a couple more scenarios. Or write a script that…

I agree, it seems like they were trying to come up with the most expensive way to answer the question possible for some reason. And why was the finding in the end to upgrade M1 users to more expensive M3s when M2s were deemed sufficient?

Because M2s are no longer produced.
Post reply on HN