Home > Software > BIGDATA > HADOOP
Interview Questions   Tutorials   Discussions   Programs   Videos   Discussion   

HADOOP - Explain the Reducer?s Sort phase?

asked SRVMTrainings November 11, 2012 02:28 AM  

Explain the Reducer?s Sort phase?


1 Answers

answered By   0  
Reducer has 3 primary phases: shuffle, sort and reduce.

The framework groups Reducer inputs by keys (since different mappers may have output the same key) in this stage.

The shuffle and sort phases occur simultaneously; while map-outputs are being fetched they are merged.

Secondary Sort

If equivalence rules for grouping the intermediate keys are required to be different from those for grouping keys before reduction, then one may specify a Comparator viaJob.setGroupingComparatorClass(Class). Since this can be used to control how intermediate keys are grouped, these can be used in conjunction to simulate secondary sort on values.

   add comment

Your answer

Join with account you already have



Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!