org.apache.hadoop.mrunit.mapreduce

Class ReduceFeeder<K,V>

java.lang.Object

org.apache.hadoop.mrunit.mapreduce.ReduceFeeder<K,V>

Type Parameters:

K - The type of the keys.

V - The type of the values.

public class ReduceFeeder<K,V>
extends Object

This Class provides some methods to get inputs compatible with new API reducer.

Constructor Summary

Constructors

Constructor and Description
ReduceFeeder(org.apache.hadoop.conf.Configuration conf) Constructor

Method Summary

Methods

Modifier and Type	Method and Description
List<KeyValueReuseList<K,V>>	sortAndGroup(List<Pair<K,V>> mapOutputs) This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to their natural order.
List<KeyValueReuseList<K,V>>	sortAndGroup(List<Pair<K,V>> mapOutputs, Comparator<K> keyOrderAndGroupComparator) This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to given comparator.
List<KeyValueReuseList<K,V>>	sortAndGroup(List<Pair<K,V>> mapOutputs, Comparator<K> keyValueOrderComparator, Comparator<K> keyGroupComparator) This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to given comparators.
List<KeyValueReuseList<K,V>>	updateAll(List<Pair<K,List<V>>> inputs) This method takes a list of (k, v*) and return a list of (k*, v*) where k have been duplicated.
KeyValueReuseList<K,V>	updateInput(K key, List<V> values) This method takes a (k, v*) input and return a (k*, v*) where k have been duplicated.
KeyValueReuseList<K,V>	updateInput(Pair<K,List<V>> input) This method takes a (k, v*) input and return a (k*, v*) where k have been duplicated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReduceFeeder

public ReduceFeeder(org.apache.hadoop.conf.Configuration conf)

Constructor

Parameters:

conf - The driver's configuration.

Method Detail

sortAndGroup

public List<KeyValueReuseList<K,V>> sortAndGroup(List<Pair<K,V>> mapOutputs,
                                        Comparator<K> keyValueOrderComparator,
                                        Comparator<K> keyGroupComparator)

This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to given comparators. If at least one comparator is null, Keys have to implements Comparable.

Parameters:

mapOutputs - The outputs from mapper

keyValueOrderComparator - The comparator for ordering keys.

keyGroupComparator - The comparator for grouping keys

Returns:

key values sorted and grouped for the reducer

sortAndGroup

public List<KeyValueReuseList<K,V>> sortAndGroup(List<Pair<K,V>> mapOutputs,
                                        Comparator<K> keyOrderAndGroupComparator)

This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to given comparator. If the comparator is null, Keys have to implements Comparable.

Parameters:

mapOutputs - The outputs from mapper

keyOrderAndGroupComparator - The comparator for grouping and ordering keys

Returns:

key values sorted and grouped for the reducer

sortAndGroup

public List<KeyValueReuseList<K,V>> sortAndGroup(List<Pair<K,V>> mapOutputs)

This method takes the outputs from a mapper and return the corresponding reducer input where keys have been sorted and grouped according to their natural order. Keys have to implements Comparable.

Parameters:

mapOutputs - The outputs from mapper

Returns:

key values sorted and grouped for the reducer

updateAll

public List<KeyValueReuseList<K,V>> updateAll(List<Pair<K,List<V>>> inputs)

This method takes a list of (k, v*) and return a list of (k*, v*) where k have been duplicated.

Parameters:

inputs - The list of key values pair with one key and multiples values

Returns:

Reducer inputs compatible with the new API

updateInput

public KeyValueReuseList<K,V> updateInput(Pair<K,List<V>> input)

This method takes a (k, v*) input and return a (k*, v*) where k have been duplicated.

Parameters:

input - The key values pair with one key for multiple values

Returns:

Reducer input compatible with the new API

updateInput

public KeyValueReuseList<K,V> updateInput(K key,
                                 List<V> values)

This method takes a (k, v*) input and return a (k*, v*) where k have been duplicated.

Parameters:

key - the key for all the values

values - multiples values for the same key

Returns:

Reducer input compatible with the new API