Live data from Hacker News

Big data is dead

motherduck.com

1–10 of 444 posts

Re: Big data is dead

#4
There is literally a post on front page on ChatGPT, and Microsoft and Google are preparing to duke it out starting in the _next 2 days_ over big-data generated 'chat' result.

Big data was never going to be useful to even medium size enterprises, unless anyone can get public access to PBs of data, but that doesn't mean big data is dead. ChatGPT is literally changing how school will test their students, for a start.

Maybe what the author is trying to say is 'small-scale big data is dead, but big data chugs on.'

Re: Big data is dead

#6
I agree with many of the points here.

My cheap no-name old laptop SSD writes with 170MB/s.

A customer has a name, address, email and order. Let's say 200 bytes for each. That means I can write 844000 new customers per second, far outside my personal marketing reach.

My disk is 240GB, which means I can store data for 1.2 billion customers. It'll take a while until I become that successful.

Re: Big data is dead

#9

There is literally a post on front page on ChatGPT, and Microsoft and Google are preparing to duke it out starting in the _next 2 days_ over big-data generated 'chat' result. Big data was never going to be useful to even medium size enterprises, unless anyone can get public access to PBs of data, but that doesn't mean big data is dead. ChatGPT is literally changing how school will test their students, for a start. Ma…

I kind of doubt they trained chatgpt on petabytes of application logs and web server logs. Is keeping all of this crap even useful for more than a small amount of time at this scale?

Actual good information will always be useful, most of this "big data" seems to be the equivalent of recording background static.

Re: Big data is dead

#10

I am writing an essay series on this topic: last-mile analytics and how an abundance of data must be ultimately converted into (measurably correct) action. If anyone wants to follow along, the series is here! https://alexpetralia.com/2023/01/19/working-with-data-from-s...

This sounds like "sane planning, sensible tomorrow." Book for Al gore
Post reply on HN