So this is essentially analogous to division of labour efficiencies. Or am I missing something?
Map Reduce: A simple introduction (2010)
31–37 of 37 posts
Re: Map Reduce: A simple introduction (2010)
#32So this is essentially analogous to division of labour efficiencies. Or am I missing something?
It's more a technique/architecture for parallelizing highly parallelizable tasks.
Re: Map Reduce: A simple introduction (2010)
#33Am I right in thinking MapReduce seems to be going out of fashion somewhat as even Google themselves have moved towards using something they term Millwheel. Which is a stream based processing system. I have seen one paper on a streaming MapReduce solution though.
Hadoop is busy replacing MapReduce with Yez. MapReduce was a great first implementation but we need something that performs better.
Re: Map Reduce: A simple introduction (2010)
#34Earlier quoted context omitted.
Unless you're already familiar with the material, spelling and grammar lends credibility to the content of the article. Similar to a code smell, it makes you ask "are you sure you know what you're doing?"
While I agree that numerous spelling and grammar mistakes can indicate a lack of maturity in writing skills, I also know that some of the most brilliant people I've ever worked with were absolutely terrible at spelling. One guy in particular was an amazing programmer. He produced libraries of code that were fast, efficient, and easy to read and understand. They were also very well documented. I know, because I went t…
Just my personal opinion, no disrespect intended. I have personally authored posts that needed a bit more time on the proofreading table and I did worry that people would question my authority on the subject due to grammatical errors in the article.
Re: Map Reduce: A simple introduction (2010)
#35Earlier quoted context omitted.
While I agree that numerous spelling and grammar mistakes can indicate a lack of maturity in writing skills, I also know that some of the most brilliant people I've ever worked with were absolutely terrible at spelling. One guy in particular was an amazing programmer. He produced libraries of code that were fast, efficient, and easy to read and understand. They were also very well documented. I know, because I went t…
Just my personal opinion, no disrespect intended. I have personally authored posts that needed a bit more time on the proofreading table and I did worry that people would question my authority on the subject due to grammatical errors in the article.
Fair enough.
Just to be clear, I don't want to give the impression that I don't think spelling/grammar matter (even for blog posts). I just think it is easy to get so pedantic about it that you place far too much weight on them.
I have my own set of pet peeves, and am probably guilty of allowing the violation of one of them to taint my view of an article too quickly and too often.
And of course, there are always those articles that are so bad grammatically that it looks like a first-grader wrote them (but I don't think that's the kind of article we were discussing).
Re: Map Reduce: A simple introduction (2010)
#36Map Reduce seems very interesting, but every example I have seen explains it in terms of counting frequency of words in documents. I would love to have someone explain it with an actual business example. I can't think of many real world uses where counting the frequency of words would matter to most businesses. (besides maybe some analysis of log files)
In realworld, people barely use "MapReduce", they use a high-level wrapper, that provides a more friendly user interface similar to SQL, like Hive, and to a lesser extent, Pig.
Re: Map Reduce: A simple introduction (2010)
#37Map Reduce seems very interesting, but every example I have seen explains it in terms of counting frequency of words in documents. I would love to have someone explain it with an actual business example. I can't think of many real world uses where counting the frequency of words would matter to most businesses. (besides maybe some analysis of log files)
I could not agree more. That is why I published an introductory example of map reduce that is still simple enough for educational purposes yet of a more real world nature.
glennengstrand.info/analytics/oss
Here you will see a blog, paper, and github repo of some map reduce jobs that take openly available San Francisco crime data and load it into an OLAP cube.