[flagged]
What does your AI agent do on it?
Big data on the cheapest MacBook
261–270 of 321 posts
Re: Big data on the cheapest MacBook
#262Earlier quoted context omitted.
> I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine. Before I was a professional software developer, I used a scrawny second-hand laptop with a Norwegian keyboard (I'm not Norwegian) because that was what I could afford: https://i.imgur…
Your hardware matters quite a bit if you're doing lower level things and the architecture is not the same as you're developing for. But apparently HN is all web devs
[deleted]
Re: Big data on the cheapest MacBook
#263Earlier quoted context omitted.
Yes you're right. I meaned a different video, but I can't find it right now. I've looked it up, and back then MacOS had a bug which exacerbated that issue. Here is an article https://www.macrumors.com/2021/02/23/m1-mac-users-report-exc...
You originally stated "Also there are countless reports of bricked M1 8GB MacBook Airs that are bricked because the SSD used up it's write cycles" Do you have a source for these "countless bricked SSD's"?
Here was the Video I meant back then.
https://m.youtube.com/watch?v=MZuv4TIjk-I&pp=ygURZGVhZCBNYWN...
Re: Big data on the cheapest MacBook
#264I’ve been tempted to buy one and do “real dev work” on it just to show people it’s not this handicapped little machine. I built multiple iOS apps and went through two start up acquisitions with my M1 MBA as my primary computer, as a developer. And the neo is better than the M1 MBA. I edited my 30-45 min long 4k race videos in FCP on that air just fine.
cool humble brag
Re: Big data on the cheapest MacBook
#265Re: Big data on the cheapest MacBook
#266This is as much an indictment of AWS compute as it is anything else.
Yeah, this is really about how ludicrously overpriced big cloud is. I’ve got a first gen M1 Max and it destroys all but the largest cloud instances (that cost its entire current market value per month!), at least in compute. It’s a laptop! A decent bare metal server in a rack will destroy any laptop. It’s staggering. Jaw dropping. Bandwidth is even worse, like 10000X markup. Yet cloud is how we do things. There’s a g…
> I’ve got a first gen M1 Max and it destroys all but the largest cloud instances (that cost its entire current market value per month!)
You're either underestimating how big cloud instances can get or overestimating how much it costs to rent a cloud instance that would beat an M1 Max at any multi-core processing.
According to Geekbench, the M1 Max macbook pro has a single-core performance of 2374 and multicore of 12257; AWS's c8i.4xlarge (16 vCPUs) has 2034 and 12807, so relatively equivalent.
That c8i.4xlarge would cost you $246/mo at current spot pricing of $0.3425/hr, which is, what, 20% of the cost of that M1 Max MBP?
As discussed recently in https://news.ycombinator.com/item?id=47291906, Geekbench is underestimating the multi-core performance of very large machines for parallelizable tasks -- the benchmark's performance peaks at around 12x single-core performance. (I might've picked a different benchmark but I couldn't find another benchmark that had results for both the M1 Max and the Xeon Scalable 6 family.)
If your tasks are _not_ like that, then even a mid-range cloud instance like a 64-vCPU c8i.16xlarge (which currently costs $0.95/hour on the spot market) will handily beat the M1 Max, by a factor of about 4. The largest cloud instances from AWS have 896 vCPUs, so I'd expect they'd outperform the M1 Max by about 50-to-1 for trivially parallelizable workloads. Even if you stay away from the exotic instances like the `u7i-12tb.224xlarge` and stick to the standard c/m/r families, the c8i.96xlarge has 384 vCPUs (so at least 24x the compute power of that M1 Max) and costs $3.76/hr.
Re: Big data on the cheapest MacBook
#267When I teach, I use "big data" for data that won't fit in a single machine. "Small data" fits on a single machine in memory and medium data on disk. Having said that duckDB is awesome. I recently ported a 20 year old Python app to modern Python. I made the backend swappable, polars or duckdb. Got a 40-80x speed improvement. Took 2 days.
The funny thing is that those days you can fit 64 TB of DDR5 in a single physical system (IBM Power Server), so almost all non data-lake-class data is "Small data".
There's nothing wrong with that. Small data is relative, and my clients often find it useful to rent or get access to beefy machines to process it with "small" techniques rather than use clusters...
Re: Big data on the cheapest MacBook
#268[flagged]
Your comment is fully AI generated which is against HN guidelines https://www.pangram.com/history/6e9e9161-73d8-4860-88cf-2424...
> Don't post generated comments or AI-edited comments. HN is for conversation between humans.
Re: Big data on the cheapest MacBook
#269Earlier quoted context omitted.
“Your data isn’t big” is a good working definition of big data. Google has big data. You are not google.
I think the definition of big is smaller than that. Mine was "too big to fit on a maxed-out laptop", effectively >8TB. Our photo collection is bigger than that, it's not 'big data'. Or one could define it as too big to fit on a single SSD/HDD, maybe >30TB. Still within the reach of a hobbyist, but too large to process in memory and needs special tools to work with. It doesn't have to be petabyte scale to need 'big da…
“Your data is not big” comes from this thread…https://news.ycombinator.com/item?id=7192839
8TB is a couple hundred hours of 4k RAW video assets.
Re: Big data on the cheapest MacBook
#270I adore DuckDB. Did a PoC on a AWS Lambda for data that was GZ'ed in a s3 bucket. It was able to replace about 400 C# LoC with about 10 lines. Amazing little bit of kit.
It really is one of the greatest open source gifts of recent years.