Most data isn’t “big,” and businesses are wasting money pretending it is
1–10 of 160 posts
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#2God says...
8:8 And there was great joy in that city.
8:9 But there was a certain man, called Simon, which beforetime in the same city used sorcery, and bewitched the people of Samaria, giving out that himself was some great one: 8:10 To whom they all gave heed, from the least to the greatest, saying, This man is the great power of God.
8:11 And to him they had regard, because that of long time he had bewitched them with sorceries.
8:12 But when they believed Philip preaching the things concerning the kingdom of God, and the name of Jesus Christ, they were baptized, both men and women.
8:13 Then Simon himself believed also: and when he was baptized, he continued with Philip, and wondered, beholding the miracles and signs which were done.
God says... son's obeyed interrupt fruit fervently treasure-house unsought overflowed deeper cold student fruit-trees dissolvest readily exceedingly wear assuring re-collected reform pierced
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#3Said this before, still want to see it fixed, as I can't stand to read the page with the huge grey box on the right side that disables scrolling without being moused over content. Last time though, I didn't have my environment info.
Windows 7 Ultimate SP1 Chrome Version 26.0.1410.64 m
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#4It's a buzzword, not a quantifiable thing.
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#5There's an important distinction to be made between the storage layer and the analysis layer. Something like HDFS can make sense as a storage layer once you hit the > 10TB range even if your average dataset for analysis is reasonably small (and it should be; 99% of the time you can get by with sampling down to single-machine size). That doesn't mean you need to be setting up all your analysis jobs to run via map-reduce; you can usually dump the dataset to a dedicated machine and do it all in one go with sequential algorithms. As a side benefit, you have access to algorithms that are really difficult to express efficiently as map-reduce (eg, computations over ordered time series).
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#6I am grateful to finally see this in an article. The "big data" craze is being pushed in areas where it really doesn't make sense. We've been bit by the Big Data bug where I'm at, but it's not coming from the statisticians. It's usually the executives proposing a shift to big data.
People underestimate how much work it would be to shift an old server onto modern technologies and tell the statisticians to use MapReduce and NoSQL instead of SAS and SQL. If the Fortune 500 world has taken this long to catch on to R, imagine how long it'll take to completely change the DBMS and analysis software!
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#7Related discussion from three weeks ago: https://news.ycombinator.com/item?id=5602727
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#8Most blogs don't need javascript, and publishers are pissing off their readers by pretending they do.
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#9Sure if you're dealing with 1GB of data it probably isn't worth spinning up a Hadoop cluster to run your analysis. However, if you already have Hadoop up an running for something that genuinely requires it, that 1GB job might make sense there. The data may already be in HDFS, and you already have the infrastructure there to manage and monitor jobs.
The references to Facebook & Yahoo running small jobs on huge clusters may be a little misleading. It may be simply the easiest place for them to deploy those jobs consistently.
But yeah... "Big Data" is a total meaningless buzzard.
Re: Most data isn’t “big,” and businesses are wasting money pretending it is
#10It's a buzzword, not a quantifiable thing.
The fact that so many people are calling things "big data" when the data is not high volume (the most popular definition I've seen is the 5 V's definition--big seems to be a misnomer in this case, as only volume could really be called a measure of "big") lends credence to your statement.