org.apache.hadoop.mrunit.mapreduce
Class ReduceDriver<K1,V1,K2,V2>

java.lang.Object
  org.apache.hadoop.mrunit.TestDriver<K1,V1,K2,V2>
      org.apache.hadoop.mrunit.ReduceDriverBase<K1,V1,K2,V2>
          org.apache.hadoop.mrunit.mapreduce.ReduceDriver<K1,V1,K2,V2>

public class ReduceDriver<K1,V1,K2,V2>
extends ReduceDriverBase<K1,V1,K2,V2>

Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results. This is designed to handle a single (k, v*) -> (k, v)* case from the Reducer, representing a single unit test. Multiple input (k, v*) sets should go in separate unit tests.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

ReduceDriverBase.ValueClassInstanceReuseList<T>

Field Summary

static org.apache.commons.logging.Log

LOG

Fields inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

inputKey

Fields inherited from class org.apache.hadoop.mrunit.TestDriver

configuration, counterWrapper, expectedEnumCounters, expectedOutputs, expectedStringCounters

Constructor Summary

ReduceDriver()

ReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Method Summary

org.apache.hadoop.mapreduce.Counters	getCounters()
org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2>	getReducer()
static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2>	newReduceDriver() Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2>	newReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> reducer) Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
List<Pair<K2,V2>>	run() Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)
void	setCounters(org.apache.hadoop.mapreduce.Counters ctrs) Sets the counters object to use for this test.
void	setReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r) Sets the reducer object to use for this test
String	toString()
ReduceDriver<K1,V1,K2,V2>	withConfiguration(org.apache.hadoop.conf.Configuration configuration)
ReduceDriver<K1,V1,K2,V2>	withCounter(Enum e, long expectedValue) Register expected enumeration based counter value
ReduceDriver<K1,V1,K2,V2>	withCounter(String g, String n, long e) Register expected name based counter value
ReduceDriver<K1,V1,K2,V2>	withCounters(org.apache.hadoop.mapreduce.Counters ctrs) Sets the counters to use and returns self for fluent style
ReduceDriver<K1,V1,K2,V2>	withInput(K1 key, List<V1> values) Identical to setInput() but returns self for fluent programming style
ReduceDriver<K1,V1,K2,V2>	withInputFromString(String input) Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
ReduceDriver<K1,V1,K2,V2>	withInputKey(K1 key) Identical to setInputKey() but with fluent programming style
ReduceDriver<K1,V1,K2,V2>	withInputValue(V1 val) Identical to addInputValue() but with fluent programming style
ReduceDriver<K1,V1,K2,V2>	withInputValues(List<V1> values) Identical to addInputValues() but with fluent programming style
ReduceDriver<K1,V1,K2,V2>	withOutput(K2 key, V2 val) Works like addOutput(), but returns self for fluent style
ReduceDriver<K1,V1,K2,V2>	withOutput(Pair<K2,V2> outputRecord) Works like addOutput(), but returns self for fluent style
ReduceDriver<K1,V1,K2,V2>	withOutputFromString(String output) Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
ReduceDriver<K1,V1,K2,V2>	withReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r) Identical to setReducer(), but with fluent programming style

Methods inherited from class org.apache.hadoop.mrunit.ReduceDriverBase

addInputValue, addInputValues, addOutput, addOutput, addOutputFromString, getInputValues, runTest, setInput, setInputFromString, setInputKey, setInputValues

Methods inherited from class org.apache.hadoop.mrunit.TestDriver

formatValueList, getConfiguration, getExpectedEnumCounters, getExpectedOutputs, getExpectedStringCounters, parseCommaDelimitedList, parseTabbedPair, resetExpectedCounters, resetOutput, runTest, setConfiguration, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Constructor Detail

ReduceDriver

public ReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

ReduceDriver

public ReduceDriver()

Method Detail

setReducer

public void setReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Sets the reducer object to use for this test

Parameters:

r - The reducer object to use

withReducer

public ReduceDriver<K1,V1,K2,V2> withReducer(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> r)

Identical to setReducer(), but with fluent programming style

Parameters:

r - The Reducer to use

Returns:

this

getReducer

public org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> getReducer()

getCounters

public org.apache.hadoop.mapreduce.Counters getCounters()

Returns:

the counters used in this test

setCounters

public void setCounters(org.apache.hadoop.mapreduce.Counters ctrs)

Sets the counters object to use for this test.

Parameters:

ctrs - The counters object to use.

withCounters

public ReduceDriver<K1,V1,K2,V2> withCounters(org.apache.hadoop.mapreduce.Counters ctrs)

Sets the counters to use and returns self for fluent style

withInputKey

public ReduceDriver<K1,V1,K2,V2> withInputKey(K1 key)

Identical to setInputKey() but with fluent programming style

Returns:

this

withInputValue

public ReduceDriver<K1,V1,K2,V2> withInputValue(V1 val)

Identical to addInputValue() but with fluent programming style

Parameters:

val -

Returns:

this

withInputValues

public ReduceDriver<K1,V1,K2,V2> withInputValues(List<V1> values)

Identical to addInputValues() but with fluent programming style

Parameters:

values -

Returns:

this

withInput

public ReduceDriver<K1,V1,K2,V2> withInput(K1 key,
                                           List<V1> values)

Identical to setInput() but returns self for fluent programming style

Parameters:

key -

values -

Returns:

this

withOutput

public ReduceDriver<K1,V1,K2,V2> withOutput(Pair<K2,V2> outputRecord)

Works like addOutput(), but returns self for fluent style

Parameters:

outputRecord -

Returns:

this

withOutput

public ReduceDriver<K1,V1,K2,V2> withOutput(K2 key,
                                            V2 val)

Works like addOutput(), but returns self for fluent style

Parameters:

key - The key part of a (k, v) pair to add

val - The val part of a (k, v) pair to add

Returns:

this

withInputFromString

@Deprecated
public ReduceDriver<K1,V1,K2,V2> withInputFromString(String input)

Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Identical to setInput, but with a fluent programming style

Parameters:

input - A string of the form "key \t val". Trims any whitespace.

Returns:

this

withOutputFromString

@Deprecated
public ReduceDriver<K1,V1,K2,V2> withOutputFromString(String output)

Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Identical to addOutput, but with a fluent programming style

Parameters:

output - A string of the form "key \t val". Trims any whitespace.

Returns:

this

run

public List<Pair<K2,V2>> run()
                      throws IOException

Description copied from class: TestDriver

Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)

Specified by:

run in class ReduceDriverBase<K1,V1,K2,V2>

Returns:

the list of (k, v) pairs returned as output from the test

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

withConfiguration

public ReduceDriver<K1,V1,K2,V2> withConfiguration(org.apache.hadoop.conf.Configuration configuration)

Parameters:

configuration - The configuration object that will given to the reducer associated with the driver

Returns:

this object for fluent coding

withCounter

public ReduceDriver<K1,V1,K2,V2> withCounter(Enum e,
                                             long expectedValue)

Description copied from class: TestDriver

Register expected enumeration based counter value

Overrides:

withCounter in class TestDriver<K1,V1,K2,V2>

Parameters:

e - Enumeration based counter

expectedValue - Expected value

Returns:

withCounter

public ReduceDriver<K1,V1,K2,V2> withCounter(String g,
                                             String n,
                                             long e)

Description copied from class: TestDriver

Register expected name based counter value

Overrides:

withCounter in class TestDriver<K1,V1,K2,V2>

Parameters:

g - Counter group

n - Counter name

e - Expected value

Returns:

newReduceDriver

public static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2> newReduceDriver()

Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.

Returns:

new ReduceDriver

newReduceDriver

public static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2> newReduceDriver(org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2> reducer)

Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.

Parameters:

reducer - passed to ReduceDriver constructor

Returns:

new ReduceDriver