Home > Software > BIGDATA > HADOOP
Interview Questions   Tutorials   Discussions   Programs   Videos   Discussion   

HADOOP - What are combiners? When should I use a combiner in my MapReduce Job?




1169
views
asked SRVMTrainings March 4, 2013 03:46 AM  

What are combiners? When should I use a combiner in my MapReduce Job?


           

1 Answers



 
answered By   0  
Combiner can minimize the data transferred between mapper and reducer.
We can write combiner code  same as reducer since instead of shuffling at a time at reducer, we can shuffle the same in mapper and can send the out put to the reducer.

For suppose your data may like this.

First mapper output without combiner:
(1950, 0)
(1950, 20)
(1950, 10)
And the second mapper output without combiner:
(1950, 25)
(1950, 15)

The reduce function would be called with a list of all the values without reducer :
(1950, [0, 20, 10, 25, 15])

With combiner
first mapper output  (1950 , 20)
Second mapper output (1950, 25)
Reducer output
(1950, 25)

flag   
   add comment

Your answer

Join with account you already have

FF

Preview


Ready to start your tutorial with us? That's great! Send us an email and we will get back to you as soon as possible!

Alert