org.apache.hadoop.mrunit.mapreduce

Class ReduceDriver<K1,V1,K2,V2>

java.lang.Object

org.apache.hadoop.mrunit.TestDriver<K2,V2,T>

org.apache.hadoop.mrunit.ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>

org.apache.hadoop.mrunit.mapreduce.ReduceDriver<K1,V1,K2,V2>

All Implemented Interfaces:

org.apache.hadoop.mrunit.internal.mapreduce.ContextDriver

@PrepareForTest(value=org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.class)
public class ReduceDriver<K1,V1,K2,V2>
extends ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>
implements org.apache.hadoop.mrunit.internal.mapreduce.ContextDriver

Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

ReduceDriverBase.ValueClassInstanceReuseList<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected List<KeyValueReuseList<K1,V1>>	groupedInputs
static org.apache.commons.logging.Log	LOG
protected org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2>	myReducer

Fields inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

inputKey, inputs, mockOutputCreator

Fields inherited from class org.apache.hadoop.mrunit.TestDriver

counterWrapper, expectedEnumCounters, expectedMultipleOutputs, expectedOutputs, expectedPathOutputs, expectedStringCounters, mos

Constructor Summary

Constructors

Constructor and Description
ReduceDriver()
ReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Method Summary

Methods

Modifier and Type	Method and Description
void	addAllElements(List<KeyValueReuseList<K1,V1>> inputs) Adds input to send to the Reducer
void	addInput(KeyValueReuseList<K1,V1> input) Add input (K*, V*) to send to the Reducer
void	clearInput() Clears the input to be sent to the Reducer
org.apache.hadoop.mapreduce.Reducer.Context	getContext() Obtain Context object for furthering mocking with Mockito.
org.apache.hadoop.mapreduce.Counters	getCounters()
List<Pair<K1,V1>>	getInputs(K1 firstKey)
org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2>	getReducer()
static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2>	newReduceDriver() Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2>	newReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> reducer) Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
protected void	preRunChecks(Object reducer) Handle inputKey and inputValues and inputs for backwards compatibility.
protected void	printPreTestDebugLog() Overridable hook for printing pre-test debug information
List<Pair<K2,V2>>	run() Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)
void	setCounters(org.apache.hadoop.mapreduce.Counters ctrs) Sets the counters object to use for this test.
void	setReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r) Sets the reducer object to use for this test
String	toString()
ReduceDriver<K1,V1,K2,V2>	withAllElements(List<KeyValueReuseList<K1,V1>> inputs) Identical to addAllElements() but returns self for fluent programming style
ReduceDriver<K1,V1,K2,V2>	withCounters(org.apache.hadoop.mapreduce.Counters ctrs) Sets the counters to use and returns self for fluent style
ReduceDriver<K1,V1,K2,V2>	withInput(KeyValueReuseList<K1,V1> input) Identical to addInput() but returns self for fluent programming style
ReduceDriver<K1,V1,K2,V2>	withOutputFormat(Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass, Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass) Configure Reducer to output with a real OutputFormat.
ReduceDriver<K1,V1,K2,V2>	withReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r) Identical to setReducer(), but with fluent programming style

Methods inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

addAll, addInput, addInput, addInputValue, addInputValues, getInputValues, getInputValues, setInput, setInputFromString, setInputKey, setInputValues, withAll, withInput, withInput, withInputFromString, withInputKey, withInputValue, withInputValues

Methods inherited from class org.apache.hadoop.mrunit.TestDriver

addAllOutput, addCacheArchive, addCacheArchive, addCacheFile, addCacheFile, addMultiOutput, addMultiOutput, addOutput, addOutput, addOutputFromString, cleanupDistributedCache, copy, copyPair, driverReused, formatPairList, formatValueList, getConfiguration, getExpectedEnumCounters, getExpectedOutputs, getExpectedStringCounters, getKeyComparator, getOutputSerializationConfiguration, getValueComparator, initDistributedCache, parseCommaDelimitedList, parseTabbedPair, resetExpectedCounters, resetOutput, run, runTest, runTest, setCacheArchives, setCacheFiles, setConfiguration, setKeyComparator, setOutputSerializationConfiguration, setUsedOnceStatus, setValueComparator, thisAsTestDriver, validate, validate, validate, validateOutputList, withAllOutput, withCacheArchive, withCacheArchive, withCacheFile, withCacheFile, withConfiguration, withCounter, withCounter, withMultiOutput, withMultiOutput, withOutput, withOutput, withOutputFromString, withOutputSerializationConfiguration, withPathOutput, withPathOutput, withStrictCounterChecking

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.mrunit.internal.mapreduce.ContextDriver

getConfiguration, getOutputSerializationConfiguration

Field Detail

groupedInputs

protected List<KeyValueReuseList<K1,V1>> groupedInputs

LOG

public static final org.apache.commons.logging.Log LOG

myReducer

protected org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> myReducer

Constructor Detail

ReduceDriver

public ReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

ReduceDriver

public ReduceDriver()

Method Detail

getInputs

public List<Pair<K1,V1>> getInputs(K1 firstKey)

clearInput

public void clearInput()

Clears the input to be sent to the Reducer

Overrides:

clearInput in class ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>

addInput

public void addInput(KeyValueReuseList<K1,V1> input)

Add input (K*, V*) to send to the Reducer

Parameters:

input -

withInput

public ReduceDriver<K1,V1,K2,V2> withInput(KeyValueReuseList<K1,V1> input)

Identical to addInput() but returns self for fluent programming style

Parameters:

input -

Returns:

this

withAllElements

public ReduceDriver<K1,V1,K2,V2> withAllElements(List<KeyValueReuseList<K1,V1>> inputs)

Identical to addAllElements() but returns self for fluent programming style

Parameters:

inputs -

Returns:

this

addAllElements

public void addAllElements(List<KeyValueReuseList<K1,V1>> inputs)

Adds input to send to the Reducer

Parameters:

inputs - list of (K*, V*) pairs

printPreTestDebugLog

protected void printPreTestDebugLog()

Description copied from class: TestDriver

Overridable hook for printing pre-test debug information

Overrides:

printPreTestDebugLog in class ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>

setReducer

public void setReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Sets the reducer object to use for this test

Parameters:

r - The reducer object to use

withReducer

public ReduceDriver<K1,V1,K2,V2> withReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Identical to setReducer(), but with fluent programming style

Parameters:

r - The Reducer to use

Returns:

this

getReducer

public org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> getReducer()

getCounters

public org.apache.hadoop.mapreduce.Counters getCounters()

Specified by:

getCounters in interface org.apache.hadoop.mrunit.internal.mapreduce.ContextDriver

Returns:

the counters used in this test

setCounters

public void setCounters(org.apache.hadoop.mapreduce.Counters ctrs)

Sets the counters object to use for this test.

Parameters:

ctrs - The counters object to use.

withCounters

public ReduceDriver<K1,V1,K2,V2> withCounters(org.apache.hadoop.mapreduce.Counters ctrs)

Sets the counters to use and returns self for fluent style

withOutputFormat

public ReduceDriver<K1,V1,K2,V2> withOutputFormat(Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass,
                                         Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass)

Configure Reducer to output with a real OutputFormat. Set InputFormat to read output back in for use with run* methods

Parameters:

outputFormatClass -

inputFormatClass -

Returns:

this for fluent style

preRunChecks

protected void preRunChecks(Object reducer)

Handle inputKey and inputValues and inputs for backwards compatibility.

Overrides:

preRunChecks in class ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>

run

public List<Pair<K2,V2>> run()
                      throws IOException

Description copied from class: TestDriver

Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)

Specified by:

run in class ReduceDriverBase<K1,V1,K2,V2,ReduceDriver<K1,V1,K2,V2>>

Returns:

the list of (k, v) pairs returned as output from the test

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

getContext

public org.apache.hadoop.mapreduce.Reducer.Context getContext()

Obtain Context object for furthering mocking with Mockito. For example, causing write() to throw an exception:

 import static org.mockito.Matchers.*;
 import static org.mockito.Mockito.*;
 doThrow(new IOException()).when(context).write(any(), any());
 

Or implement other logic:

 import static org.mockito.Matchers.*;
 import static org.mockito.Mockito.*;
 doAnswer(new Answer