Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tracking developer build times to decide if the M3 MacBook is worth upgrading

incident.io

Post reply on HN
Print view View original

121–130 of 432 posts

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#121

Post by hsbauauvhabzb »

lawrjone wrote: post #12
kingTug wrote: post #10

Does anyone have any anecdoctal evidence around the snappiness of VsCode with Apple Silicon? I very begrudgingly switched over from SublimeText this year (after using it as my daily driver for ~10yrs). I have a beefy 2018 MBP but VScode just drags. This is the only thing pushing me to upgrade my machine right now but I'd be bummed if there's still not a significant improvement with an m3 pro.

If you're using an Intel Mac at this point, you should 100% upgrade. The performance of the MX chips blows away the Intel chips and there's almost no friction with the arm architecture at this point. I don't use VSCode but most of my team do and I frequently pair with them. Never noticed it to be anything other than very snappy. They all have M1s or up (I am the author of this post, so the detail about their hardware…

There can be plenty of friction depending on your use case.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#122

Post by cced »

joshspankit wrote: post #99

Since RAM was a major metric, there should have been more focus on IO Wait to catch cases where OSX was being hindered by swapping to disk. (Yes, the drives are fast but you don’t know until you measure)

This. I’ve routinely got a 10-15GB page file on an M2 pro and need to justify bumping the memory up a notch or two. I’m consistently in the yellow memory and in the red while building.

How can I tell how much I would benefit from a memory bump?

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#123

Post by BlueToth »

boringuser2 wrote: post #95

I get a bit of a toxic vibe from a couple comments in that article. Chiefly, I think the problem is that the CTO solved the wrong problem: the right problem to solve includes a combination of assessing why company public opinion is generating mass movements of people wanting a new MacBook literally every year, if this is even worth responding to at all (it isn't), and keeping employees happy. Most employees are reaso…

It's really worth the money if it keeps employees happy! Besides that the conclusion was updating M1 to M3, but not every year.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#124

Post by Herring »

dash2 wrote: post #78

As a scientist, I'm interested how computer programmers work with data. * They drew beautiful graphs! * They used chatgpt to automate their analysis super-fast! * ChatGPT punched out a reasonably sensible t test! But: * They had variation across memory and chip type, but they never thought of using a linear regression. * They drew histograms, which are hard to compare. They could have supplemented them with simple me…

>They drew histograms, which are hard to compare.

Note that in some places they used boxplots, which offer clearer comparisons. It would have been more effective to present all the data using boxplots.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#125

Post by xp84 »

hk1337 wrote: post #112

I wonder why they didn't include Linux since the project they're building is Go? Most CI tools, I believe, are going to be Linux. Sure, you can explicitly select macOS in Github CI but Linux seems like it would be the better generic option? *EDIT* I guess if you needed a macOS specific build with Go you would us macOS but I would have thought you'd use Linux too. Can you build a Go project in Linux and have it run on…

I know nothing about Go, but if it's like other platforms, builds intended for production or staging environments are indeed nearly always for x86_64, but those are done somewhere besides laptops, as part of the CI process. The builds done on the laptops are to run each developer's local instance of their server-side application and its front-end components, That instance is always being updated to whatever is in-progress at the time. Then they check that code in, and eventually it gets built for prod on an Intel, Linux system elsewhere.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#126

Post by lawrjone »

gray_-_wolf wrote: post #106
lawrjone wrote: post #75

Earlier quoted context omitted.

Fwiw I've spent my whole career doing data analysis but the ease at which I was able to use OpenAI to help me for this post (am author) blew me away. The fact that I can do this type of analysis is why I appreciate it so much. It's one of the reasons I'm convinced AI engineering find its way into the average software engineer's remit ( https://blog.lawrencejones.dev/2023/#ai ) because it makes this analysis far more…

> it makes this analysis far more accessible than it was before How does the average engineer verify if the result is correct? You claim (and I believe you) to be able to do this "by hand", if required. Great, but that likely means you are able to catch when LLM makes an mistake. Any ideas on how average engineer, without much experience in this area, should validate the results?

I mentioned this in a separate comment but it may be worth bearing in mind how the AI pipeline works, in that you’re not pushing all this data into an LLM and asking it to produce graphs, which would be prone to some terrible errors.

Instead, you’re using the LLM to generate Python code that runs using normal libraries like Pandas and gnuplot. When it makes errors it’s usually generating totally the wrong graphs rather than inaccurate data, and you can quickly ask it “how many X Y Z” and use that to spot check the graphs before you proceed.

My initial version of this began in a spreadsheet so it’s not like you need sophisticated analysis to check this stuff. Hope that explains it!

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#127

Post by guax »

lawrjone wrote: post #46
orenlindsey wrote: post #31

This is pretty cool, also I love how you can use AI to read the data. Would take minutes if not hours to do it even just a year ago.

Yeah, I thought it was really cool! (am author) It's pretty cool how it works, too: the OpenAI Assistant uses the LLM to take your human instructions like "how many builds is in the dataset?" and translate that into Python code which is run in a sandbox on OpenAI compute with access to the dataset you've uploaded. Under the hood everything is just numpy, pandas and gnuplot, you're just using a human interface to a Py…

I also got surprised about using it for this kind of work. I don't have access to copilot and gpt-4 at work but my first instinct is to ask, did you double check its numbers?

Knowing how it works now makes more sense that it would make less mistakes but I'm still skeptical :P

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#128

Post by lawrjone »

boringuser2 wrote: post #95

I get a bit of a toxic vibe from a couple comments in that article. Chiefly, I think the problem is that the CTO solved the wrong problem: the right problem to solve includes a combination of assessing why company public opinion is generating mass movements of people wanting a new MacBook literally every year, if this is even worth responding to at all (it isn't), and keeping employees happy. Most employees are reaso…

Sorry you read it like this!

If it’s useful: Pete wasn’t really being combative with me on this. I suggested we should check if the M3 really was faster so we could upgrade if it was, we agreed and then I did the analysis. The game aspect of this was more for a bit of fun in the article than how things actually work.

And in terms of why we didn’t have a process for this: the company itself is about two years old, so this was the first hardware refresh we’d ever needed to schedule. So we haven’t a formal process in place yet and probably won’t until the next one either!

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#129

Post by lawrjone »

mixmastamyk wrote: post #110

A lot of the graphs near the end comparing side-to-side had different scales on the Y axis. Take results with a grain of salt. https://incident.io/_next/image?url=https%3A%2F%2Fcdn.sanity...

They’re normalised histograms so the y axis is deliberately adjusted so you can compare the shape of the distribution, as the absolute number of builds in each bucket means little when there are a different count of builds for each platform.

Re: Tracking developer build times to decide if the M3 MacBook is worth upgrading

#130

Post by lawrjone »

PaulHoule wrote: post #82
isthisreallife9 wrote: post #60

[flagged]

The medium is the message here, the macbook is just bait. The pure LLM is not effective on tabular data (so many transcripts of ChatGPT apologizing it got a calculation wrong.). To be working as well as it seems to work they must be loading results into something like a pandas data frame and having the agent write and run programs on that data frame, tap into stats and charting libraries, etc. I’d trust it more if th…

Author here!

We’re using the new OpenAI assistants with the code interpreter feature, which allows you to ask questions of the model and have OpenAI turn those into python code that they run on their infra and pipe the output back into the model chat.

It’s really impressive and removes need for you to ask it for code and then run that locally. This is what powers many of the data analysis product features that are appearing recently (we’re building one ourselves for our incident data and it works pretty great!)

Post reply on HN