Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tracking developer build times to decide if the M3 MacBook is worth upgrading

incident.io

Post reply on HN
Print view View original

311–320 of 432 posts

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#311

Post by Kwpolska »

nox101 wrote: post #265
Aurornis wrote: post #64

This is a great write-up and I love all the different ways they collected and analyzed data. That said, it would have been much easier and more accurate to simply put each laptop side by side and run some timed compilations on the exact same scenarios: A full build, incremental build of a recent change set, incremental build impacting a module that must be rebuilt, and a couple more scenarios. Or write a script that…

I didn't see any analysis of network building as an alternative to M3s. For my project, ~40 million lines, past a certain minimum threshold, it doesn't matter how fast my machine is, it can't compete with the network build our infra-team makes. So sure, an M3 might make my build 30% faster than my M1 build, but the network build is 15x faster. Is it possible instead of giving the developers M3s they should have inves…

Network full builds might be faster, but would incremental builds be? Would developers still be able to use their favourite IDE and OS? Would developers be able to work without waiting in a queue? Would developers be able to work offline?

If you have a massive, monolithic, single-executable-producing codebase that can't be built on a developer machine, then you need network builds. But if you aren't Google, building on laptops gives developers better experience, even if it's slower.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#312

Post by karolist »

brailsafe wrote: post #150

Fun read, I like how overkill it is. When I was still employed, I was building our django/postgres thing locally in Docker, with 32gb of ram, and it was a wild improvement in terms of feedback loop latency over my shitty 13" intel mbp, and I think it's seriously underappreciated how important it is to keep that pretty low , or as low as is cost effective. Now that I'm not employed and don't hope to be for a while, I…

I don't know where in the world you are, but B&H in US still sells new 16" M1 Max machines with 64GB memory, 2TB SSD for 2499-2599 depending on the current deal. This is around the price of base M3 Pro with 18/512 configuration, I figure you'll still get 5+ years of use with such machine and never worry about storage or memory.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#313

Post by karolist »

hmottestad wrote: post #157
hsbauauvhabzb wrote: post #121

Earlier quoted context omitted.

There can be plenty of friction depending on your use case.

What software are you worried about? Hackernews is a great place to share and hear if anyone has similar experiences or maybe managed to find a fix!

https://doesitarm.com/

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#314

Post by kwk1 »

rag-hav wrote: post #297
nox101 wrote: post #265

Earlier quoted context omitted.

I didn't see any analysis of network building as an alternative to M3s. For my project, ~40 million lines, past a certain minimum threshold, it doesn't matter how fast my machine is, it can't compete with the network build our infra-team makes. So sure, an M3 might make my build 30% faster than my M1 build, but the network build is 15x faster. Is it possible instead of giving the developers M3s they should have inves…

What do you mean by network build?

They probably mean tools like distcc or sccache:

https://github.com/distcc/distcc

https://github.com/mozilla/sccache

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#315

Post by Kwpolska »

WaxProlix wrote: post #109
LASR wrote: post #90

Solid analysis. A word of warning from personal experience: I am part of a medium-sized software company (2k employees). A few years ago, we wanted to improve dev productivity. Instead of going with new laptops, we decided to explore offloading the dev stack over to AWS boxes. This turned out to be a multi-year project with a whole team of devs (~4) working on it full-time. In hindsight, the tradeoff wasn't worth it.…

I suspect things like GitHub's Codespaces offering will be more and more popular as time goes on for this kind of thing. Did you guys try out some of the AWS Cloud9 or other 'canned' dev env offerings?

AWS Cloud9 is a web IDE that can run on any EC2 box. The web IDE is a custom Amazon thing and is quite mediocre.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#316

Post by DeathArrow »

rag-hav wrote: post #297
nox101 wrote: post #265

Earlier quoted context omitted.

I didn't see any analysis of network building as an alternative to M3s. For my project, ~40 million lines, past a certain minimum threshold, it doesn't matter how fast my machine is, it can't compete with the network build our infra-team makes. So sure, an M3 might make my build 30% faster than my M1 build, but the network build is 15x faster. Is it possible instead of giving the developers M3s they should have inves…

What do you mean by network build?

Dedicated build machines.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#317

Post by Kwpolska »

euos wrote: post #223

I am ex-core contributor Chromium and Node.js and current core contributor to gRPC Core/C++. I am never bothered with build times. There is "interactive build" (incremental builds I use to rerun related unit tests as I work on code) and non-interactive build (one I launch and go get coffee/read email). I have never seen hardware refresh toggle non-interactive into interactive. My personal hardware (that I use now and…

Chromium is a massive project. In more normally-sized projects, you can build everything on your laptop in reasonable time.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#318

Post by DeathArrow »

novok wrote: post #305
dash2 wrote: post #78

As a scientist, I'm interested how computer programmers work with data. * They drew beautiful graphs! * They used chatgpt to automate their analysis super-fast! * ChatGPT punched out a reasonably sensible t test! But: * They had variation across memory and chip type, but they never thought of using a linear regression. * They drew histograms, which are hard to compare. They could have supplemented them with simple me…

It's because that's usually the data scientist's job, and most eng infra teams don't have a data scientist and don't really need one most of the time. Most of the time they deal with data the way their tools generally present data, which correlate closely to most analytics, perf analysis and observability software suites. Expecting the average software eng to know what a CDF is the same as expecting them to know 3d g…

>Expecting the average software eng to know what a CDF is the same as expecting them to know 3d graphics basics like quaternions and writing shaders.

I did write shaders and used quaternions back in the day. I also worked on microcontrollers, did some system programming, developed mobile and desktop apps. Now I am working on a rather large microservice based app.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#319

Post by ribit »

Interesting idea, but the quality of data analysis is rather poor IMO and I'm not sure that they are actually learning what they think they are learning. Most importantly, I don't understand why they would see such a dramatic increase of sub 20s build times going from M1 Pro to M2 Pro. The real-world performance delta between the two on code compilation workloads is around 20-25%. It also makes little sense to me that M3 machines have fewer sub-20s builds than M2 machines. Or that M3 Pro with half the cores has more sub-20s builds than M3 Max.

I suspect there might be considerable difference in developer behavior which results in these differences. Such as people with different types of laptops typically working on different things.

And a few random observations after a very cursory reading (I might be missing something):

- Go compiler seems to take little advantage from additional cores

- They are pooling data in ways that makes me fundamentally uncomfortable

- They are not consistent in their comparisons, sometimes they use histograms, sometimes they use binned density plots (with different y axis ranges), it's real unclear what is going on here...

- Macs do not throttle CPU performance on battery. If the builds are really slower on battery (which I am not convinced about btw looking at graphs), it will be because of "low power" setting activated

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#320

Post by eysi »

jayd16 wrote: post #214
eysi wrote: post #191

Earlier quoted context omitted.

My team has been developing against a fully remote environment (K8s cluster) for some years now and it makes for a really powerful DevEx. Code sits on our laptops but live syncs to the remote services without requiring a Docker build or K8s deploy. It really does feel like local. In particular it lets us do away with the commit-push-pray cycle because we can run integ tests and beyond as we code as opposed to waiting…

Kind of interesting to think that CI is significantly slower in practice and both systems need to be maintained. Is it just the overhead of pushing through git or are there other reasons as well?

The way we do things is that we build everything in the cloud and store in a central container registry. So if I trigger a build during dev, the CI runner can re-use that, e.g. if it’s needed before running a test or creating a preview env.

Similarly if another dev (or a CI runner) triggers a build of one of our services, I won’t have to next time I start my dev environment. And because it’s built in the cloud there’s no “works on my machine”.

Same applies to tests actually. They run in the cloud in an independent and trusted environment and the results are cached and stored centrally.

Garden knows all the files and config that belong to a given test suite. So the very first CI run may run tests for service A, service B, and service C. I then write code that only changes service B, open a PR and only the relevant tests get run in CI.

And because it’s all in prod-like environments, I can run integ and e2e tests from my laptop as I code, instead of only having that set up for CI.

Post reply on HN