Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tracking developer build times to decide if the M3 MacBook is worth upgrading

incident.io

Post reply on HN
Print view View original

201–210 of 432 posts

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#201

Post by drgo »

To people who are thinking about using AI for data analyses like the one described in the article:

- I think it is much easier to just load the data into R, Stata etc and interrogate the data that way. The commands to do that will be shorter and more precise and most importantly more reproducible.

- the most difficult task in data analysis is understanding the data and the mechanisms that have generated it. For that you will need a causal model of the problem domain. Not sure that AI is capable of building useful causal models unless they were somehow first trained using other data from the domain.

- it is impossible to reasonably interpret the data without reference to that model. I wonder if current AI models are capable of doing that, e.g., can they detect confounding or oversized influence of outliers or interesting effect modifiers.

Perhaps someone who knows more than I do on the state of current technology can provide a better assessment of where we are in this effort

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#202

Post by matthew-wegner »

Aurornis wrote: post #64

This is a great write-up and I love all the different ways they collected and analyzed data. That said, it would have been much easier and more accurate to simply put each laptop side by side and run some timed compilations on the exact same scenarios: A full build, incremental build of a recent change set, incremental build impacting a module that must be rebuilt, and a couple more scenarios. Or write a script that…

I would think you would want to capture what/how was built, as like:

* Repo started at this commit

* With this diff applied

* Build was run with this command

Capture that for a week. Now you have a cross section of real workloads, but you can repeat the builds on each hardware tier (and even new hardware down the road)

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#203

Post by azalemeth »

dash2 wrote: post #78

As a scientist, I'm interested how computer programmers work with data. * They drew beautiful graphs! * They used chatgpt to automate their analysis super-fast! * ChatGPT punched out a reasonably sensible t test! But: * They had variation across memory and chip type, but they never thought of using a linear regression. * They drew histograms, which are hard to compare. They could have supplemented them with simple me…

> ChatGPT punched out a reasonably sensible t test!

I think the distribution is decidedly non normal here and the difference in the medians may well have also been of substantial interest -- I'd go for a Wilcox test here to first order... Or even some type of quantile regression. Honestly the famous Jonckheere–Terpstra test for ordered medians would be _perfect_ for this bit of pseudoanalysis -- have the hypothesis that M3 > M2 > M1 and you're good to go, right?!

(Disclaimers apply!)

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#204

Post by balls187 »

> a chat interface to your ... data Generally, the process includes:

> Exporting your data to a CSV

> Create an ‘Assistant’ with a prompt explaining your purpose, and provide it the CSV file with your data.

Once MSFT build's the aforementioned process into Excel, it's going to be a major game changer.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#205

Post by orangecat »

mgrandl wrote: post #63
tomaskafka wrote: post #47

My personal research for iOS development, taking the cost into consideration, concluded: - M2 Pro is nice, but the improvement over 10 core (8 perf cores) M1 Pro is not that large (136 vs 120 s in Xcode benchmark: https://github.com/devMEremenko/XcodeBenchmark ) - M3 Pro is nerfed (only 6 perf cores) to better distinguish and sell M3 Max, basically on par with M2 Pro So, in the end, I got a slightly used 10 core M1 P…

The M3 Pro being nerfed has been parroted on the Internet since the announcement. Practically it’s a great choice. It’s much more efficient than the M2 Pro at slightly better performance. That’s what I am looking for in a laptop. I don’t really have a usecase for the memory bandwidth…

The M3 Pro and Max get virtually identical results in battery tests, e.g. https://www.tomsguide.com/news/macbook-pro-m3-and-m3-max-bat.... The Pro may be a perfectly fine machine, but Apple didn't remove cores to increase battery life; they did it to lower costs and upsell the Max.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#206

Post by vessenes »

cameroncairns wrote: post #146
lawrjone wrote: post #26

Earlier quoted context omitted.

Ahh right, I'd seen a few comments about the memory bandwidth when it was posted on LinkedIn, specifically that the M2 was much more powerful. This makes a load of sense, thanks for explaining.

I've been considering buying a Mac specifically for LLMs, and I've come across a lot of info/misinfo on the topic of bandwidth. I see you are talking about M2 bandwidth issues that you read about on linkedin, so I wanted to expand upon that in case there is any confusion on your part or someone else who is following this comment chain. M2 Ultra at 800 GB/s is for the mac studio only. So it's not quite apples to apple…

All this sounds right!

If money is no object, and you don't need a laptop, and you want a suggestion, then I'd say the M2 Ultra / Studio is the way to go. If money is still no object and you need a laptop, M3 with maxed RAM.

I have a 300GB/s M3 and a 400 GB/s M1 with more RAM, and generally the LLM difference is minimal; the extra RAM is helpful though.

If you want to try some stuff out, and don't anticipate running an LLM more than 10 hours a week, lambda labs or together.ai will save you a lot of money. :)

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#208

Post by firecall »

lawrjone wrote: post #2

Author here, thanks for posting! Lots of stuff in this from profiling Go compilations, building a hot-reloader, using AI to analyse the build dataset, etc. We concluded that it was worth upgrading the M1s to an M3 Pro (the max didn’t make much of a difference in our tests) but the M2s are pretty close to the M3s, so not (for us) worth upgrading. Happy to answer any questions if people have them.

So can we assume the M3 Max offered little benefit because the workloads couldn’t use the cores?

Or the tasks maybe finished so fast that it didn’t make a difference in real world usage?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#209

Post by Tarball10 »

This is only tangentially related, but I'm curious how other companies typically balance their endpoint management and security software with developer productivity.

The company I work for is now running 5+ background services on their developer laptops, both Mac and Windows. Endpoint management, priviledge escalation interception, TLS interception and inspection, anti-malware, and VPN clients.

This combination heavily impacts performance. You can see these services chewing up CPU and I/O performance while doing anything on the machines, and developers have complained about random lockups and hitches.

I understand security is necessary, especially with the increase in things like ransomware and IP theft, but have other companies found better ways to provide this security without impacting developer productivity as much?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#210

Post by vaxman »

moondev wrote: post #192
FireBeyond wrote: post #154

Earlier quoted context omitted.

> You can even get by with the 8GB M1 MBA because the MacOS memory compressor is really good and the SSD is really fast. I thought that general consensus was that 8GB Macs were hammering the life of the SSDs? Yeah, they're fast, but people were talking about dozens of GB a day of swapping happening. And these aren't enterprise class SSDs, despite what Apple charges for them.

I'm curious if an 8GB MacBook can run a macOS VM, and if so how much memory can be allocated to it.

Sure, it operates on an over allocation pattern. If you try to use most of the RAM, the system will begin compressing blocks of memory and will eventually begin swapping them to super fast NVMe storage. This may be fine for desktop productivity apps and web browsing, but will make the system feel sluggish when flipping between unrelated contexts.
Post reply on HN