Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big data on the cheapest MacBook

duckdb.org

Post reply on HN
Print view View original

21–30 of 321 posts

Re: Big data on the cheapest MacBook

#21

Post by speedgoose »

michalc wrote: post #3
TutleCpt wrote: post #2

Oh great, the term "big data" is back.

So my definition of big data was data so big it cannot be processed on a single machine in a reasonable amount of time. I guess they’re using a different definition?

Computers got bigger and software got smarter.

You have phones that are faster than cloud VMs of the past. You can use bare metal servers with up to 344 cores and 16TB of ram.

I used to share your definition too, but I now say that if it doesn’t open in Microsoft Excel, it’s big data.

Re: Big data on the cheapest MacBook

#22

Post by speedgoose »

rrr_oh_man wrote: post #19
michalc wrote: post #3

Earlier quoted context omitted.

So my definition of big data was data so big it cannot be processed on a single machine in a reasonable amount of time. I guess they’re using a different definition?

In my former life as a soulless consultant mid-level IT managers really liked to hear the 3 "V"s mentioned: Velocity, Volume, Variety

The V of Value is very important in some circles.

Re: Big data on the cheapest MacBook

#23

Post by LeifCarrotson »

Schiendelman wrote: post #17
BoredPositron wrote: post #15

Queue the endless blog posts about running tech on the potato macbook and being stunned it’s functional with massive trade-offs. Groundbreaking stuff.

That usage is "Cue", not "queue".

Cue the queue of blogs! Trigger the formation of a line of posts to be published sequentially.

Re: Big data on the cheapest MacBook

#24

Post by SteveNuts »

post-it wrote: post #11
Jeffrin-dev wrote: post #10

[flagged]

Did you use AI to write a one-paragraph comment?

> MacBook Neo's NVMe being slower than the Air/Pro isn't just a benchmark footnote — it compounds with file count in a non-linear way.

The “isn’t just” part is a dead giveaway almost always.

Re: Big data on the cheapest MacBook

#25

Post by g947o »

opentokix wrote: post #18

Mind blown, if you need to handle "big" data on the move - the macbook neo is not the right choice. - Who would have guessed that outcome?

It occurs to me that there is near zero overlap between people who use a Macbook Neo and people who run DuckDB locally.

It would be a surprise if more than 0.1% of Macbook Neo users have even heard of DuckDB.

Which means that this article is probably just riding the hype.

Re: Big data on the cheapest MacBook

#27

Post by Zambyte »

speedgoose wrote: post #21
michalc wrote: post #3

Earlier quoted context omitted.

So my definition of big data was data so big it cannot be processed on a single machine in a reasonable amount of time. I guess they’re using a different definition?

Computers got bigger and software got smarter. You have phones that are faster than cloud VMs of the past. You can use bare metal servers with up to 344 cores and 16TB of ram. I used to share your definition too, but I now say that if it doesn’t open in Microsoft Excel, it’s big data.

Processing data that cannot be processed on a single machine is fundamentally a different problem than processing data that can be processed on a single machine. It's useful to have a term for that.

As you say, single machines can scale up incredibly far. That just means 16 TB datasets no longer demand big data solutions.

Re: Big data on the cheapest MacBook

#29

Post by speedgoose »

Zambyte wrote: post #27
speedgoose wrote: post #21

Earlier quoted context omitted.

Computers got bigger and software got smarter. You have phones that are faster than cloud VMs of the past. You can use bare metal servers with up to 344 cores and 16TB of ram. I used to share your definition too, but I now say that if it doesn’t open in Microsoft Excel, it’s big data.

Processing data that cannot be processed on a single machine is fundamentally a different problem than processing data that can be processed on a single machine. It's useful to have a term for that. As you say, single machines can scale up incredibly far. That just means 16 TB datasets no longer demand big data solutions.

I get your point, but I don’t know if big data is the right term anymore.

Many people like to think they have big data, and you kinda have to agree with them if you want their money. At least in consulting.

Also you could go well beyond a 16TB dataset on a single machine. You assume that the whole uncompressed dataset has to fit in memory, but many workloads don’t need that.

How many people in the world have such big datasets to analyse within reasonable time?

Some people say extreme data.

Re: Big data on the cheapest MacBook

#30

Post by brudgers »

michalc wrote: post #3
TutleCpt wrote: post #2

Oh great, the term "big data" is back.

So my definition of big data was data so big it cannot be processed on a single machine in a reasonable amount of time. I guess they’re using a different definition?

“Your data isn’t big” is a good working definition of big data.

Google has big data. You are not google.

Post reply on HN