Home:ALL Converter>Hadoop MapReduce WordCount example flaw?

Hadoop MapReduce WordCount example flaw?

Ask Time:2016-02-16T13:41:52         Author:Felice Pollano

Json Formatter

With reference to the basic WordCount example: https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html I know that HDFS divide files in blocks, and maps tasks works on a single block. So there is no guarantee the block analyzed by a map task would not contain a word continuing in the next block, causing a mistake ( one word counted twice ). I know this is an example, and is always shown with small file, but wouldn't be a problem in real world scenarios?

Author:Felice Pollano,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/35424651/hadoop-mapreduce-wordcount-example-flaw
yy