Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big data on the cheapest MacBook

duckdb.org

Post reply on HN
Print view View original

281–290 of 321 posts

Re: Big data on the cheapest MacBook

#281

Post by bmd1905 »

meatmanek wrote: post #266
api wrote: post #33

Earlier quoted context omitted.

Yeah, this is really about how ludicrously overpriced big cloud is. I’ve got a first gen M1 Max and it destroys all but the largest cloud instances (that cost its entire current market value per month!), at least in compute. It’s a laptop! A decent bare metal server in a rack will destroy any laptop. It’s staggering. Jaw dropping. Bandwidth is even worse, like 10000X markup. Yet cloud is how we do things. There’s a g…

> I’ve got a first gen M1 Max and it destroys all but the largest cloud instances (that cost its entire current market value per month!) You're either underestimating how big cloud instances can get or overestimating how much it costs to rent a cloud instance that would beat an M1 Max at any multi-core processing. According to Geekbench, the M1 Max macbook pro has a single-core performance of 2374 and multicore of 12…

[dead]

Re: Big data on the cheapest MacBook

#282

Post by 3eb7988a1663 »

__mharrison__ wrote: post #73

When I teach, I use "big data" for data that won't fit in a single machine. "Small data" fits on a single machine in memory and medium data on disk. Having said that duckDB is awesome. I recently ported a 20 year old Python app to modern Python. I made the backend swappable, polars or duckdb. Got a 40-80x speed improvement. Took 2 days.

Always a classic: https://yourdatafitsinram.net/

Outside of the king's ransom you now have to pay for it, you can fit 99% of problems into RAM.

Re: Big data on the cheapest MacBook

#283

Post by vasco »

mjlee wrote: post #202
vasco wrote: post #34

Earlier quoted context omitted.

But AWS beat the laptop? And there's no cost to performance analysis? Yes AWS is overpriced but how do you make that conclusion from this specific article? Because network disks were slower than SSDs? AWS also has SSD instances with local storage.

I haven't tried the newer I7i and I8g instance types (the newest instances with local storage) for myself, but AWS claims "I7i instances offer up to 45TB of NVMe storage with up to 50% better real-time storage performance, up to 50% lower storage I/O latency, and up to 60% lower storage I/O latency variability compared to I4i instances." I benchmarked I4i at ~2GB/s read, so let's say I7i gets 3GB/s. The Verge benchma…

My point is the conclusion can't be made from the article.

Re: Big data on the cheapest MacBook

#284

Post by spike021 »

dracyr wrote: post #205
Robdel12 wrote: post #37

I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.

I just spent vacation deciding not to bring a laptop, but to use my android phone (a galaxy s22) with a hdmi adapter and Bluetooth travel keyboard. Plugged it in to the TV in our accomodation and had a lot of fun. Running neovim on termux was fine. Developing elixir was no problem, the test suite took 5s on my phone, and takes 1s on my laptop. Rust and cargo compiling was slow enough that I didn't really enjoy it tho…

I brought my 13 inch macbook pro to japan for three weeks last month for photo editing. i was able to pack up immediately by slipping it into my backpack laptop pocket.

Not sure the difference other than weight, but I wasn't carrying it day to day when i could leave it in my hotel room.

Re: Big data on the cheapest MacBook

#285

Post by billforsternz »

embedding-shape wrote: post #138
Robdel12 wrote: post #37

I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.

> I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine. Before I was a professional software developer, I used a scrawny second-hand laptop with a Norwegian keyboard (I'm not Norwegian) because that was what I could afford: https://i.imgur…

The almost regretful "I'm not Norwegian" clarification here is charming.

Re: Big data on the cheapest MacBook

#286

Post by disillusioned »

noisy_boy wrote: post #277
jorvi wrote: post #229

Earlier quoted context omitted.

People usually forget 8GB isn't 8GB. Memory compression means you can store ~2x (lz4) to 3x (zstd) as much data in memory as ordinarily. And in the worst case, reading swap from disk (writes don't matter as they can be predicted) is so much faster with NVMe SSDs. The worst corner they cut is no keyboard backlighting. That saves them what, $1 BoM per MacBook Neo? Especially because now they have to put up an entire ne…

It's not cost saving, it's deliberate differentiation.

Isn't it definitionally both?

Re: Big data on the cheapest MacBook

#287

Post by jandrewrogers »

AlotOfReading wrote: post #227
ElectricalUnion wrote: post #215

Earlier quoted context omitted.

The funny thing is that those days you can fit 64 TB of DDR5 in a single physical system (IBM Power Server), so almost all non data-lake-class data is "Small data".

And a single machine can hold petabytes of disk for medium scale. There aren't many datasets exceeding that outside fundamental physics.

> There aren't many datasets exceeding that outside fundamental physics.

Just about every physical world telemetry or sensing data source of any note will generate petabytes of analytical data model in hours to days. On the high end, there are single categories of data source that aggregate to more like an exabyte per day of high-value data.

It is a completely different standard of scale than web data. In many industrial domains the average small-to-medium sized company I come across retains tens of petabytes of data and it has been this way for many years. The prohibitive cost is the only thing keeping them for scaling even more.

The major issue is that the large-scale analytics infrastructure developed for web data are hopelessly inadequate.

Re: Big data on the cheapest MacBook

#288

Post by dracyr »

noisy_boy wrote: post #276
skeeter2020 wrote: post #271

Earlier quoted context omitted.

This doesn't really sound like a vacation.

Seems like vacation from things you have to do to things you enjoy doing.

Exactly that, I have too many ideas for side-projects and never enough time for them.

The main activity was still the traveling, hiking and enjoying some calm time. But instead of spending the usual downtime reading or something else, I had a blast coding and experimenting.

Re: Big data on the cheapest MacBook

#290

Post by compounding_it »

sheiyei wrote: post #250
dracyr wrote: post #205

Earlier quoted context omitted.

I just spent vacation deciding not to bring a laptop, but to use my android phone (a galaxy s22) with a hdmi adapter and Bluetooth travel keyboard. Plugged it in to the TV in our accomodation and had a lot of fun. Running neovim on termux was fine. Developing elixir was no problem, the test suite took 5s on my phone, and takes 1s on my laptop. Rust and cargo compiling was slow enough that I didn't really enjoy it tho…

Interesting vacation activities.

Ikr. Vacation for me would be : get away from electronics as much as possible. Maybe stay connected with family or do payments but not beyond that.
Post reply on HN