Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most data isn’t “big,” and businesses are wasting money pretending it is

qz.com

Post reply on HN
Print view View original

31–40 of 160 posts

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#31

Post by joseph_cooney »

eurleif wrote: post #28
SeoxyS wrote: post #24

Earlier quoted context omitted.

I know that's not what you mean, but I find it quite amusing that you describe Oracle (a 30 year old company)'s business as convincing people they need to do things the same way Google (a 15 year old company) does it.

He said they have the same business model. Business models are abstract.

dewitt said "Oracle has used exactly this business model to great success, and obscene profit, for over 30 years". You claim that business models are abstract. Perhaps, except in the case where the words "exact" are used, and a specific company is named. I, like SeoxyS find the paradox of Oracle being accused of helping businesses to be "me-too" copies of Google - a company half of Oracle's age - amusing.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#32

Post by eksith »

In an effort to Store All the Things a lot of companies have talked themselves into a rhetorical corner of poorly fitting shoes. At this point, we've stopped wasting time when asked about Big Table and NoSQL and instead demo their storage stack on a different framework until their eyes widen.

Then, when questions about how much engineering went into this "thing" that does such a good job of keeping data secure, and so much of it, we say it's built on Postgres.

As DevOps Borat says : https://twitter.com/DEVOPS_BORAT/status/313322958997295104

How well you can utilize it and how quickly is just as important as what kind of data you store in the first place.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#33

Post by snaky »

dewitt wrote: post #19

If I ever want to get rich, I'll set up shop convincing small businesses they need to do things the way Google does, if only they want to remain competitive. Oracle has used exactly this business model to great success, and obscene profit, for over 30 years.

They need to do it if the cost is affordable. And the fact the cost of Big Data processing became cheap enough to do it en masse is a driver.

Take a simple example. Every small offline shop will not refuse to count every customers head turn (with direction, angle and frequency), calculate averages, and get sales floor attention heatmap divided by day, hour, age and sex.

If that would cost them $500 one-time fee for 10 cameras and $5/day for the cloud service, and produced by pressing one big green key.

That's the Big Data driver, businesses are opening their eyes to the ability to analyze (cheap!) a huge number of small (and even smaller) factors and make better decisions.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#34

Post by jwilliams »

For me, "big" data is increasing the linkage between your data. It's not simply more data, but much richer, less formal data relationships. It's taking your sales data and linking it to your website clicks, linking that to the weather (or whatever). Or you take something traditionally static and add a temporal dimension.

This kind of deep linking you can't measure with straight megabytes. A few gig doesn't seem that large, but if it's a complex graph with a complex hypothesis - then, sure - that's big.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#35

Post by pfarrell »

There are two ways to define BigData.

1. The accumulation, integration and analysis of a larger number of data sources.

2. A volume of data that presents challenges running analysis functions across them... Due to the limits of the tools available.

1 is fraught with the kind of statistical pitfalls that are mentioned in the posted article. 2 describes a set of problems and boundaries that are time sensitive. What was BigData in 2006 (to, say LiveJournal or Digg) may not longer hold. As a data engineer, its important to keep a skeptical eye on marketing and make sure we're delivering valuable solutions that increase the bottom line for our business, not just produce "ain't it cool" type correlations.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#36

Post by jshen »

christopheraden wrote: post #30
philip1209 wrote: post #11

I think that big data has made math sexy, and selling applied statistics and operations research to small and medium-sized businesses under the guise of "big data" with the intention of providing applied mathematical tools is what is happening in the market.

Statistics involves checking modeling assumptions. A lot of what I've seen with the big data people is the repetition of algorithms to the exclusion of understanding and checking modeling assumptions. While it's nice that the big data craze is making statistics more popular in the mainstream press, it is important that statistics does not become just an application of numerical methods without consideration of underl…

Your comment reminded me of this recent Krugman post which makes a similar point about economics.

http://krugman.blogs.nytimes.com/2013/05/11/harpooning-ben-b...

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#37

Post by dvse »

joseph_cooney wrote: post #31
eurleif wrote: post #28

Earlier quoted context omitted.

He said they have the same business model. Business models are abstract.

dewitt said "Oracle has used exactly this business model to great success, and obscene profit, for over 30 years". You claim that business models are abstract. Perhaps, except in the case where the words "exact" are used, and a specific company is named. I, like SeoxyS find the paradox of Oracle being accused of helping businesses to be "me-too" copies of Google - a company half of Oracle's age - amusing.

The model in question is "convince small and medium businesses that they need to buy my software in order to do things the same way as large companies X and Y and have a hope of remaining competitive". For Oracle X and Y were banks, retail and logistics companies, for the new generation of "big data" vendors it is Google and Facebook.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#38

Post by yen223 »

SeoxyS wrote: post #24
dewitt wrote: post #19

If I ever want to get rich, I'll set up shop convincing small businesses they need to do things the way Google does, if only they want to remain competitive. Oracle has used exactly this business model to great success, and obscene profit, for over 30 years.

I know that's not what you mean, but I find it quite amusing that you describe Oracle (a 30 year old company)'s business as convincing people they need to do things the same way Google (a 15 year old company) does it.

To be pedantic, he said Oracle follows the same business model, not that Oracle follows Google.

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#39

Post by GigabyteCoin »

What is the author of this article trying to say here?

>it appears that for both Facebook and Yahoo, those same clusters are unnecessary for many of the tasks which they’re handed. In the case of Facebook, most of the jobs engineers ask their clusters to perform are in the “megabyte to gigabyte” range (pdf), which means they could easily be handled on a single computer—even a laptop.

That facebook or yahoo could be run from a laptop?

Re: Most data isn’t “big,” and businesses are wasting money pretending it is

#40

Post by userbee »

christopheraden wrote: post #30
philip1209 wrote: post #11

I think that big data has made math sexy, and selling applied statistics and operations research to small and medium-sized businesses under the guise of "big data" with the intention of providing applied mathematical tools is what is happening in the market.

Statistics involves checking modeling assumptions. A lot of what I've seen with the big data people is the repetition of algorithms to the exclusion of understanding and checking modeling assumptions. While it's nice that the big data craze is making statistics more popular in the mainstream press, it is important that statistics does not become just an application of numerical methods without consideration of underl…

"Big data" also checks model assumptions, if only if by monitoring whether or not acting on the information moves a business metric.

Statistics involves inference over prediction, but either one when done right validates assumptions.

By the way big data will sit on your face for days.

Post reply on HN