org.apache.hadoop.mrunit
Class ReduceDriver<K1,V1,K2,V2>

java.lang.Object
  extended by org.apache.hadoop.mrunit.TestDriver<K1,V1,K2,V2>
      extended by org.apache.hadoop.mrunit.ReduceDriverBase<K1,V1,K2,V2>
          extended by org.apache.hadoop.mrunit.ReduceDriver<K1,V1,K2,V2>

public class ReduceDriver<K1,V1,K2,V2>
extends ReduceDriverBase<K1,V1,K2,V2>

Harness that allows you to test a Reducer instance. You provide a key and a set of intermediate values for that key that represent inputs that should be sent to the Reducer (as if they came from a Mapper), and outputs you expect to be sent by the Reducer to the collector. By calling runTest(), the harness will deliver the input to the Reducer and will check its outputs against the expected results. This is designed to handle a single (k, v*) -> (k, v)* case from the Reducer, representing a single unit test. Multiple input (k, v*) sets should go in separate unit tests.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.mrunit.ReduceDriverBase
ReduceDriverBase.ValueClassInstanceReuseList<T>
 
Field Summary
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from class org.apache.hadoop.mrunit.ReduceDriverBase
inputKey
 
Fields inherited from class org.apache.hadoop.mrunit.TestDriver
configuration, counterWrapper, expectedEnumCounters, expectedOutputs, expectedStringCounters
 
Constructor Summary
ReduceDriver()
           
ReduceDriver(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)
           
 
Method Summary
 org.apache.hadoop.mapred.Counters getCounters()
           
 org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> getReducer()
           
static
<K1,V1,K2,V2>
ReduceDriver<K1,V1,K2,V2>
newReduceDriver()
          Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
static
<K1,V1,K2,V2>
ReduceDriver<K1,V1,K2,V2>
newReduceDriver(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> reducer)
          Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.
 List<Pair<K2,V2>> run()
          Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)
 void setCounters(org.apache.hadoop.mapred.Counters ctrs)
          Sets the counters object to use for this test.
 void setReducer(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)
          Sets the reducer object to use for this test
 String toString()
           
 ReduceDriver<K1,V1,K2,V2> withConfiguration(org.apache.hadoop.conf.Configuration configuration)
           
 ReduceDriver<K1,V1,K2,V2> withCounter(Enum e, long expectedValue)
          Register expected enumeration based counter value
 ReduceDriver<K1,V1,K2,V2> withCounter(String g, String n, long e)
          Register expected name based counter value
 ReduceDriver<K1,V1,K2,V2> withCounters(org.apache.hadoop.mapred.Counters ctrs)
          Sets the counters to use and returns self for fluent style
 ReduceDriver<K1,V1,K2,V2> withInput(K1 key, List<V1> values)
          Identical to setInput() but returns self for fluent programming style
 ReduceDriver<K1,V1,K2,V2> withInputFromString(String input)
          Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
 ReduceDriver<K1,V1,K2,V2> withInputKey(K1 key)
          Identical to setInputKey() but with fluent programming style
 ReduceDriver<K1,V1,K2,V2> withInputValue(V1 val)
          Identical to addInputValue() but with fluent programming style
 ReduceDriver<K1,V1,K2,V2> withInputValues(List<V1> values)
          Identical to addInputValues() but with fluent programming style
 ReduceDriver<K1,V1,K2,V2> withOutput(K2 key, V2 val)
          Works like addOutput(), but returns self for fluent style
 ReduceDriver<K1,V1,K2,V2> withOutput(Pair<K2,V2> outputRecord)
          Works like addOutput(), but returns self for fluent style
 ReduceDriver<K1,V1,K2,V2> withOutputFromString(String output)
          Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables
 ReduceDriver<K1,V1,K2,V2> withReducer(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)
          Identical to setReducer(), but with fluent programming style
 
Methods inherited from class org.apache.hadoop.mrunit.ReduceDriverBase
addInputValue, addInputValues, addOutput, addOutput, addOutputFromString, getInputValues, runTest, setInput, setInputFromString, setInputKey, setInputValues
 
Methods inherited from class org.apache.hadoop.mrunit.TestDriver
formatValueList, getConfiguration, getExpectedEnumCounters, getExpectedOutputs, getExpectedStringCounters, parseCommaDelimitedList, parseTabbedPair, resetExpectedCounters, resetOutput, runTest, setConfiguration, validate, validate
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Constructor Detail

ReduceDriver

public ReduceDriver(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)

ReduceDriver

public ReduceDriver()
Method Detail

getCounters

public org.apache.hadoop.mapred.Counters getCounters()
Returns:
the counters used in this test

setCounters

public void setCounters(org.apache.hadoop.mapred.Counters ctrs)
Sets the counters object to use for this test.

Parameters:
ctrs - The counters object to use.

withCounters

public ReduceDriver<K1,V1,K2,V2> withCounters(org.apache.hadoop.mapred.Counters ctrs)
Sets the counters to use and returns self for fluent style


setReducer

public void setReducer(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)
Sets the reducer object to use for this test

Parameters:
r - The reducer object to use

withReducer

public ReduceDriver<K1,V1,K2,V2> withReducer(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> r)
Identical to setReducer(), but with fluent programming style

Parameters:
r - The Reducer to use
Returns:
this

getReducer

public org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> getReducer()

withInputKey

public ReduceDriver<K1,V1,K2,V2> withInputKey(K1 key)
Identical to setInputKey() but with fluent programming style

Returns:
this

withInputValue

public ReduceDriver<K1,V1,K2,V2> withInputValue(V1 val)
Identical to addInputValue() but with fluent programming style

Parameters:
val -
Returns:
this

withInputValues

public ReduceDriver<K1,V1,K2,V2> withInputValues(List<V1> values)
Identical to addInputValues() but with fluent programming style

Parameters:
values -
Returns:
this

withInput

public ReduceDriver<K1,V1,K2,V2> withInput(K1 key,
                                           List<V1> values)
Identical to setInput() but returns self for fluent programming style

Returns:
this

withOutput

public ReduceDriver<K1,V1,K2,V2> withOutput(Pair<K2,V2> outputRecord)
Works like addOutput(), but returns self for fluent style

Parameters:
outputRecord -
Returns:
this

withOutput

public ReduceDriver<K1,V1,K2,V2> withOutput(K2 key,
                                            V2 val)
Works like addOutput(), but returns self for fluent style

Parameters:
key - The key part of a (k, v) pair to add
val - The val part of a (k, v) pair to add
Returns:
this

withInputFromString

@Deprecated
public ReduceDriver<K1,V1,K2,V2> withInputFromString(String input)
Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Identical to setInput, but with a fluent programming style

Parameters:
input - A string of the form "key \t val". Trims any whitespace.
Returns:
this

withOutputFromString

@Deprecated
public ReduceDriver<K1,V1,K2,V2> withOutputFromString(String output)
Deprecated. No replacement due to lack of type safety and incompatibility with non Text Writables

Identical to addOutput, but with a fluent programming style

Parameters:
output - A string of the form "key \t val". Trims any whitespace.
Returns:
this

withCounter

public ReduceDriver<K1,V1,K2,V2> withCounter(Enum e,
                                             long expectedValue)
Description copied from class: TestDriver
Register expected enumeration based counter value

Overrides:
withCounter in class TestDriver<K1,V1,K2,V2>
Parameters:
e - Enumeration based counter
expectedValue - Expected value
Returns:

withCounter

public ReduceDriver<K1,V1,K2,V2> withCounter(String g,
                                             String n,
                                             long e)
Description copied from class: TestDriver
Register expected name based counter value

Overrides:
withCounter in class TestDriver<K1,V1,K2,V2>
Parameters:
g - Counter group
n - Counter name
e - Expected value
Returns:

run

public List<Pair<K2,V2>> run()
                      throws IOException
Description copied from class: TestDriver
Runs the test but returns the result set instead of validating it (ignores any addOutput(), etc calls made before this)

Specified by:
run in class ReduceDriverBase<K1,V1,K2,V2>
Returns:
the list of (k, v) pairs returned as output from the test
Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object

withConfiguration

public ReduceDriver<K1,V1,K2,V2> withConfiguration(org.apache.hadoop.conf.Configuration configuration)
Parameters:
configuration - The configuration object that will given to the reducer associated with the driver
Returns:
this object for fluent coding

newReduceDriver

public static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2> newReduceDriver()
Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.

Returns:
new ReduceDriver

newReduceDriver

public static <K1,V1,K2,V2> ReduceDriver<K1,V1,K2,V2> newReduceDriver(org.apache.hadoop.mapred.Reducer<K1,V1,K2,V2> reducer)
Returns a new ReduceDriver without having to specify the generic types on the right hand side of the object create statement.

Parameters:
reducer - passed to ReduceDriver constructor
Returns:
new ReduceDriver


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.