Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big data on the cheapest MacBook

duckdb.org

Post reply on HN
Print view View original

211–220 of 321 posts

Re: Big data on the cheapest MacBook

#211

Post by mjlee »

ipython wrote: post #53
montroser wrote: post #26

This is as much an indictment of AWS compute as it is anything else.

Kinda comparing apples to oranges. AWS was using EBS and not local instance storage. So you’re easily looking at another order of magnitude latency when transmitting data over the network versus a local pcie bus. That’s gonna be a huge factor in what I assume is a heavy random seek load.

I wrote a longer comment already (https://news.ycombinator.com/item?id=47352526) but looking at the hot run performance and making big hand wavy guesses, the performance difference might not be as big as you'd expect.

Re: Big data on the cheapest MacBook

#212

Post by Robdel12 »

anthonySs wrote: post #95
Robdel12 wrote: post #37

I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.

most dev workflows from pre 2021 can probably run just fine on a NEO - i think once you get into conductor / 8 terminals with claude code territory that’s where things start to slow down i just got an m5 max with 128gb of ram specifically to run local llms

Heh, I also upgraded to run local LLMs. As a tiny aside, codex does not burn resources like CC does.

Re: Big data on the cheapest MacBook

#213

Post by nabakin »

dracyr wrote: post #205
Robdel12 wrote: post #37

I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.

I just spent vacation deciding not to bring a laptop, but to use my android phone (a galaxy s22) with a hdmi adapter and Bluetooth travel keyboard. Plugged it in to the TV in our accomodation and had a lot of fun. Running neovim on termux was fine. Developing elixir was no problem, the test suite took 5s on my phone, and takes 1s on my laptop. Rust and cargo compiling was slow enough that I didn't really enjoy it tho…

And nowadays we have Debian running in a VM on Android [1]

[1] https://www.zdnet.com/article/how-to-use-the-new-linux-termi...

Re: Big data on the cheapest MacBook

#214

Post by ElectricalUnion »

eru wrote: post #112
anthonySs wrote: post #95

Earlier quoted context omitted.

most dev workflows from pre 2021 can probably run just fine on a NEO - i think once you get into conductor / 8 terminals with claude code territory that’s where things start to slow down i just got an m5 max with 128gb of ram specifically to run local llms

Does Claude Code take up that many local resources? I thought the heavy lifting was in the cloud?

Claude Code still runs things on your local machine. So if you have some pretty expensive transpilation, or resolving dependency trees that needs musl recompilation, or doing something rust, you still need a reasonable ammount of local firepower. More so if you're running multiple instances of them.

Re: Big data on the cheapest MacBook

#215

Post by ElectricalUnion »

__mharrison__ wrote: post #73

When I teach, I use "big data" for data that won't fit in a single machine. "Small data" fits on a single machine in memory and medium data on disk. Having said that duckDB is awesome. I recently ported a 20 year old Python app to modern Python. I made the backend swappable, polars or duckdb. Got a 40-80x speed improvement. Took 2 days.

The funny thing is that those days you can fit 64 TB of DDR5 in a single physical system (IBM Power Server), so almost all non data-lake-class data is "Small data".

Re: Big data on the cheapest MacBook

#216

Post by sam_lowry_ »

throwawaytea wrote: post #187
embedding-shape wrote: post #138

Earlier quoted context omitted.

> I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine. Before I was a professional software developer, I used a scrawny second-hand laptop with a Norwegian keyboard (I'm not Norwegian) because that was what I could afford: https://i.imgur…

I still manage and develop my php/jquery saas product on a 2011 27" iMac running Linux Mint, with an SSD being the only upgrade. Runs better than most new windows machines. No complaints.

I switch between Thinkpad T420s and PineBook Pro for all the hobby work.

T420s has loose USB ports and the power socket is almost falling off, so I plan to replace it by a 5 years old T14 G2 in the coming months.

I can afford the latest MacBook, but I'd rather not generate more e-waste that there is, and more importantly I feel closer to my users, and my code is efficient and straight to the point.

My non-hobby laptop is an old cheap Dell from 5-6 years ago.

The best laptop I ever had was a maxed-out Thinkpad P7x, and it came with the most meaningless job ever.

I can only compare that job to the one at a unicorn that gave me the latest and greatest MacBook. Not only the job was meaningless, the whole industry made no sense to me.

Re: Big data on the cheapest MacBook

#217

Post by oldandboring »

rob wrote: post #62
Robdel12 wrote: post #37

I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.

It's starting to show its age, but I've been using a 2019 MacBook Pro with the Intel chip and 16GB of memory. Still handles multiple terminal sessions with Claude Code and Codex simultaneously, building in Xcode, running Docker in the background, etc. (Maybe the fans sometimes sound like they're a jet engine taking off…) Finally just put an order in for a new 16" MBP M5 Max with 48GB memory only because it looks like…

Former employee of mine had the 2019 MBP as well. After a few years he had the same problem with the fans -- if you haven't already, pop it open and clean the fans and vents. You'll probably need a little brush along with compressed air. Lots of stuff comes up on Google. Great machine btw. Good luck!

Re: Big data on the cheapest MacBook

#219

Post by api »

fridder wrote: post #169
api wrote: post #33

Earlier quoted context omitted.

Yeah, this is really about how ludicrously overpriced big cloud is. I’ve got a first gen M1 Max and it destroys all but the largest cloud instances (that cost its entire current market value per month!), at least in compute. It’s a laptop! A decent bare metal server in a rack will destroy any laptop. It’s staggering. Jaw dropping. Bandwidth is even worse, like 10000X markup. Yet cloud is how we do things. There’s a g…

Honestly I think the best path is hybrid with the cloud as DR and sudden load scaling.

Metal with data streamed to cloud and cloud as hot backup is something some people already do.

If the metal dies in a catastrophic way (multiple nodes at once and loss of quorum, catastrophic DC outage, etc.) you spin it up in AWS.

Re: Big data on the cheapest MacBook

#220

Post by alpaca128 »

NetMageSCW wrote: post #153
alpaca128 wrote: post #99

Earlier quoted context omitted.

A couple YouTube tabs are enough if you leave them running for long enough. Just one YT browser process will easily take up 1-4GB sooner or later.

Or it won’t because Chrome and MacOS will know how much RAM is available and manage it effectively.

I am talking from experience with an M1 & 8GB RAM. I had to restart either the browser or the YouTube browser processes at least once every couple days to stop the whole system from lagging.
Post reply on HN