org.apache.hadoop.mrunit

Class ReduceDriverBase<K1,V1,K2,V2,T extends ReduceDriverBase<K1,V1,K2,V2,T>>

java.lang.Object

org.apache.hadoop.mrunit.TestDriver<K2,V2,T>

org.apache.hadoop.mrunit.ReduceDriverBase<K1,V1,K2,V2,T>

Direct Known Subclasses:

ReduceDriver, ReduceDriver

public abstract class ReduceDriverBase<K1,V1,K2,V2,T extends ReduceDriverBase<K1,V1,K2,V2,T>>
extends TestDriver<K2,V2,T>

Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ReduceDriverBase.ValueClassInstanceReuseList<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected K1	inputKey Deprecated.
protected List<Pair<K1,List<V1>>>	inputs
protected org.apache.hadoop.mrunit.internal.output.MockOutputCreator<K2,V2>	mockOutputCreator

Fields inherited from class org.apache.hadoop.mrunit.TestDriver

counterWrapper, expectedEnumCounters, expectedMultipleOutputs, expectedOutputs, expectedPathOutputs, expectedStringCounters, LOG, mos

Constructor Summary

Constructors

Constructor and Description
ReduceDriverBase()

Method Summary

Methods

Modifier and Type	Method and Description
void	addAll(List<Pair<K1,List<V1>>> inputs) Adds input to send to the Reducer
void	addInput(K1 key, List<V1> values) Add input (K, V*) to send to the Reducer
void	addInput(Pair<K1,List<V1>> input) Add input (K, V*) to send to the Reducer
void	addInputValue(V1 val) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)
void	addInputValues(List<V1> values) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)
void	clearInput() Clears the input to be sent to the Reducer
List<V1>	getInputValues() Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by getInputValues(Object)
List<V1>	getInputValues(K1 key) Returns a list of values for the given key
protected void	preRunChecks(Object reducer) Handle inputKey and inputValues for backwards compatibility.
protected void	printPreTestDebugLog() Overridable hook for printing pre-test debug information
abstract List<Pair<K2,V2>>	run() Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)
void	setInput(K1 key, List<V1> values) Sets the input to send to the reducer
void	setInputFromString(String input) Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
void	setInputKey(K1 key) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)
void	setInputValues(List<V1> values) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)
T	withAll(List<Pair<K1,List<V1>>> inputs) Identical to addAll() but returns self for fluent programming style
T	withInput(K1 key, List<V1> values) Identical to setInput() but returns self for fluent programming style
T	withInput(Pair<K1,List<V1>> input) Identical to addInput() but returns self for fluent programming style
T	withInputFromString(String input) Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
T	withInputKey(K1 key) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)
T	withInputValue(V1 val) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)
T	withInputValues(List<V1> values) Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)

Methods inherited from class org.apache.hadoop.mrunit.TestDriver

addAllOutput, addCacheArchive, addCacheArchive, addCacheFile, addCacheFile, addMultiOutput, addMultiOutput, addOutput, addOutput, addOutputFromString, cleanupDistributedCache, copy, copyPair, driverReused, formatPairList, formatValueList, getConfiguration, getExpectedEnumCounters, getExpectedOutputs, getExpectedStringCounters, getKeyComparator, getOutputSerializationConfiguration, getValueComparator, initDistributedCache, parseCommaDelimitedList, parseTabbedPair, resetExpectedCounters, resetOutput, run, runTest, runTest, setCacheArchives, setCacheFiles, setConfiguration, setKeyComparator, setOutputSerializationConfiguration, setUsedOnceStatus, setValueComparator, thisAsTestDriver, validate, validate, validate, validateOutputList, withAllOutput, withCacheArchive, withCacheArchive, withCacheFile, withCacheFile, withConfiguration, withCounter, withCounter, withMultiOutput, withMultiOutput, withOutput, withOutput, withOutputFromString, withOutputSerializationConfiguration, withPathOutput, withPathOutput, withStrictCounterChecking

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inputs

protected List<Pair<K1,List<V1>>> inputs

inputKey

@Deprecated
protected K1 inputKey

Deprecated.

mockOutputCreator

protected final org.apache.hadoop.mrunit.internal.output.MockOutputCreator<K2,V2> mockOutputCreator

Constructor Detail

ReduceDriverBase

public ReduceDriverBase()

Method Detail

getInputValues

@Deprecated
public List<V1> getInputValues()

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by getInputValues(Object)

Returns a list of values.

Returns:

List of values

getInputValues

public List<V1> getInputValues(K1 key)

Returns a list of values for the given key

Parameters:

key -

Returns:

List for the given key, or null if key does not exist

setInputKey

@Deprecated
public void setInputKey(K1 key)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)

Sets the input key to send to the Reducer

addInputValue

@Deprecated
public void addInputValue(V1 val)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)

adds an input value to send to the reducer

Parameters:

val -

setInputValues

@Deprecated
public void setInputValues(List<V1> values)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)

Sets the input values to send to the reducer; overwrites existing ones

Parameters:

values -

addInputValues

@Deprecated
public void addInputValues(List<V1> values)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by setInput(K1, java.util.List<V1>), addInput(K1, java.util.List<V1>), and addAll(java.util.List<org.apache.hadoop.mrunit.types.Pair<K1, java.util.List<V1>>>)

Adds a set of input values to send to the reducer

Parameters:

values -

setInput

public void setInput(K1 key,
            List<V1> values)

Sets the input to send to the reducer

Parameters:

key -

values -

clearInput

public void clearInput()

Clears the input to be sent to the Reducer

addInput

public void addInput(K1 key,
            List<V1> values)

Add input (K, V*) to send to the Reducer

Parameters:

key - The key too add

values - The list of values to add

addInput

public void addInput(Pair<K1,List<V1>> input)

Add input (K, V*) to send to the Reducer

Parameters:

input - input pair

addAll

public void addAll(List<Pair<K1,List<V1>>> inputs)

Adds input to send to the Reducer

Parameters:

inputs - list of (K, V*) pairs

setInputFromString

@Deprecated
public void setInputFromString(String input)

Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Expects an input of the form "key \t val, val, val..." Forces the Reducer input types to Text.

Parameters:

input - A string of the form "key \t val,val,val". Trims any whitespace.

withInputKey

@Deprecated
public T withInputKey(K1 key)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)

Identical to setInputKey() but with fluent programming style

Returns:

this

withInputValue

@Deprecated
public T withInputValue(V1 val)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)

Identical to addInputValue() but with fluent programming style

Parameters:

val -

Returns:

this

withInputValues

@Deprecated
public T withInputValues(List<V1> values)

Deprecated. MRUNIT-64. Moved to list implementation to support multiple input (k, v*)*. Replaced by withInput(Object, List), withAll(List), and withInput(Pair)

Identical to addInputValues() but with fluent programming style

Parameters:

values -

Returns:

this

withInput

public T withInput(K1 key,
          List<V1> values)

Identical to setInput() but returns self for fluent programming style

Returns:

this

withInputFromString

@Deprecated
public T withInputFromString(String input)

Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Identical to setInput, but with a fluent programming style

Parameters:

input - A string of the form "key \t val". Trims any whitespace.

Returns:

this

withInput

public T withInput(Pair<K1,List<V1>> input)

Identical to addInput() but returns self for fluent programming style

Parameters:

input -

Returns:

this

withAll

public T withAll(List<Pair<K1,List<V1>>> inputs)

Identical to addAll() but returns self for fluent programming style

Parameters:

inputs -

Returns:

this

preRunChecks

protected void preRunChecks(Object reducer)

Handle inputKey and inputValues for backwards compatibility.

run

public abstract List<Pair<K2,V2>> run()
                               throws IOException

Description copied from class: TestDriver

Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)

Specified by:

run in class TestDriver<K2,V2,T extends ReduceDriverBase<K1,V1,K2,V2,T>>

Returns:

the list of (k, v) pairs returned as output from the test

Throws:

IOException

printPreTestDebugLog

protected void printPreTestDebugLog()

Description copied from class: TestDriver

Overridable hook for printing pre-test debug information

Overrides:

printPreTestDebugLog in class TestDriver<K2,V2,T extends ReduceDriverBase<K1,V1,K2,V2,T>>