org.apache.hadoop.examples

Class RandomWriter

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.examples.RandomWriter

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class RandomWriter
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

This program uses map/reduce to just run a distributed job where there is no interaction between the tasks and each task write a large unsorted random binary sequence file of BytesWritable. In order for this program to generate data for terasort with 10-byte keys and 90-byte values, have the following config: <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>test.randomwrite.min_key</name> <value>10</value> </property> <property> <name>test.randomwrite.max_key</name> <value>10</value> </property> <property> <name>test.randomwrite.min_value</name> <value>90</value> </property> <property> <name>test.randomwrite.max_value</name> <value>90</value> </property> <property> <name>test.randomwrite.total_bytes</name> <value>1099511627776</value> </property> </configuration> Equivalently, RandomWriter also supports all the above options and ones supported by GenericOptionsParser via the command-line.

Constructor Summary

Constructors

Constructor and Description
RandomWriter()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(String[] args)
int	run(String[] args) This is the main routine for launching a distributed random write job.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

RandomWriter

public RandomWriter()

Method Detail

run

public int run(String[] args)
        throws Exception

This is the main routine for launching a distributed random write job. It runs 10 maps/node and each node writes 1 gig of data to a DFS file. The reduce doesn't do anything.

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

IOException

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception