org.apache.hadoop.contrib.failmon

Class LocalStore

java.lang.Object

org.apache.hadoop.contrib.failmon.LocalStore

public class LocalStore
extends Object

This class takes care of the temporary local storage of gathered metrics before they get uploaded into HDFS. It writes Serialized Records as lines in a temporary file and then compresses and uploads it into HDFS.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMPRESSION_SUFFIX
static char	FIELD_SEPARATOR
static char	RECORD_SEPARATOR
static int	UPLOAD_INTERVAL

Constructor Summary

Constructors

Constructor and Description
LocalStore() Create an instance of the class and read the configuration file to determine some output parameters.

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close the temporary local file
static void	copyToHDFS(String localFile, String hdfsFile) Copy a local file to HDFS
void	insert(EventRecord er) Insert an EventRecord to the local storage, after it gets serialized and anonymized.
void	insert(EventRecord[] ers) Insert an array of EventRecords to the local storage, after they get serialized and anonymized.
static StringBuffer	pack(SerializedRecord sr) Pack a SerializedRecord into an array of bytes
void	upload() Upload the local file store into HDFS, after it compressing it.
static void	zipCompress(String filename) Compress a text file using the ZIP compressing algorithm.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FIELD_SEPARATOR

public static final char FIELD_SEPARATOR

See Also:

Constant Field Values

RECORD_SEPARATOR

public static final char RECORD_SEPARATOR

See Also:

Constant Field Values

COMPRESSION_SUFFIX

public static final String COMPRESSION_SUFFIX

See Also:

Constant Field Values

UPLOAD_INTERVAL

public static final int UPLOAD_INTERVAL

See Also:

Constant Field Values

Constructor Detail

LocalStore

public LocalStore()

Create an instance of the class and read the configuration file to determine some output parameters. Then, initiate the structured needed for the buffered I/O (so that smal appends can be handled efficiently).

Method Detail

insert

public void insert(EventRecord er)

Insert an EventRecord to the local storage, after it gets serialized and anonymized.

Parameters:

er - the EventRecord to be inserted

insert

public void insert(EventRecord[] ers)

Insert an array of EventRecords to the local storage, after they get serialized and anonymized.

Parameters:

ers - the array of EventRecords to be inserted

pack

public static StringBuffer pack(SerializedRecord sr)

Pack a SerializedRecord into an array of bytes

Parameters:

sr - the SerializedRecord to be packed

upload

public void upload()

Upload the local file store into HDFS, after it compressing it. Then a new local file is created as a temporary record store.

zipCompress

public static void zipCompress(String filename)
                        throws IOException

Compress a text file using the ZIP compressing algorithm.

Parameters:

filename - the path to the file to be compressed

Throws:

IOException

copyToHDFS

public static void copyToHDFS(String localFile,
              String hdfsFile)
                       throws IOException

Copy a local file to HDFS

Parameters:

localFile - the filename of the local file

hdfsFile - the HDFS filename to copy to

Throws:

IOException

close

public void close()

Close the temporary local file