Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big data is dead

motherduck.com

Post reply on HN
Print view View original

11–20 of 444 posts

Re: Big data is dead

#11

Post by miguelazo »

guardiangod wrote: post #4

There is literally a post on front page on ChatGPT, and Microsoft and Google are preparing to duke it out starting in the _next 2 days_ over big-data generated 'chat' result. Big data was never going to be useful to even medium size enterprises, unless anyone can get public access to PBs of data, but that doesn't mean big data is dead. ChatGPT is literally changing how school will test their students, for a start. Ma…

>ChatGPT is literally changing how school will test their students, for a start.

Sure, instead of schools checking for plagiarism from other students' papers using turnitin.com, they'll check for plagiarism using ChatGPT tools that scan for known output from their industrial-scale amalgamation of plagiarized materials. Big whoop.

Re: Big data is dead

#13

Post by lucidguppy »

Some of mongo's leveling off is the adoption of good jsonb columns in postgres.

mongo's got sharding out of the box - which is nice - but you have to get your key right or it will suck.

Also no one should want to host a mongo db - unless that's your business.

Re: Big data is dead

#14

Post by fdgsdfogijq »

"For more than a decade now, the fact that people have a hard time gaining actionable insights from their data has been blamed on its size."

The real issue is that business people usually ignore what the data says. Wading through data takes a huge amount of thought, which is in short supply. Data Scientists are commonly disregarded by VPs in large corporations, despite the claims about being "data driven". Most corporate decision making is highly political, the needs of/whats best for the business is just one parameter in a complex equation.

Re: Big data is dead

#16

Post by revolvingocelot »

fdgsdfogijq wrote: post #14

"For more than a decade now, the fact that people have a hard time gaining actionable insights from their data has been blamed on its size." The real issue is that business people usually ignore what the data says. Wading through data takes a huge amount of thought, which is in short supply. Data Scientists are commonly disregarded by VPs in large corporations, despite the claims about being "data driven". Most corpo…

It's absolutely this. "Decision-based evidence-making" is what I've seen it called.

Re: Big data is dead

#18

Post by capableweb »

fdgsdfogijq wrote: post #14

"For more than a decade now, the fact that people have a hard time gaining actionable insights from their data has been blamed on its size." The real issue is that business people usually ignore what the data says. Wading through data takes a huge amount of thought, which is in short supply. Data Scientists are commonly disregarded by VPs in large corporations, despite the claims about being "data driven". Most corpo…

Is that what's happening at Amazon as well? As they seemingly is loosing more and more track of the "Customer Obsession" schtick.

Re: Big data is dead

#19

Post by epicureanideal »

guardiangod wrote: post #4

There is literally a post on front page on ChatGPT, and Microsoft and Google are preparing to duke it out starting in the _next 2 days_ over big-data generated 'chat' result. Big data was never going to be useful to even medium size enterprises, unless anyone can get public access to PBs of data, but that doesn't mean big data is dead. ChatGPT is literally changing how school will test their students, for a start. Ma…

I wonder how long until training today’s ChatGPT will cost $1000 of AWS compute. 10 years?

At that point, does it keep scaling or is there an S curve where 100x more data and compute only leads to a 2x improvement?

Re: Big data is dead

#20

Post by glogla »

I agree with a lot of the sentiments of the MotherDuck people, but boy are they loud and proud for someone who never delivered anything more than blogposts and vague promise to somehow exploit the MIT licensed DuckDB.

Meanwhile for example boilingdata.com seems to have already done that - by using AWS Lambda + DuckDB as distributed compute engine which I can't decide if its awesome, deranged or both.

Post reply on HN