org.apache.hadoop.mapred

Class DefaultTaskController

java.lang.Object

org.apache.hadoop.mapred.TaskController

org.apache.hadoop.mapred.DefaultTaskController

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class DefaultTaskController
extends TaskController

The default implementation for controlling tasks. This class provides an implementation for launching and killing tasks that need to be run as the tasktracker itself. Hence, many of the initializing or cleanup methods are not required here.
NOTE: This class is internal only class and not intended for users!!

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.TaskController

TaskController.DeletionContext

Field Summary

Fields inherited from class org.apache.hadoop.mapred.TaskController

allocator, COMMAND_FILE, localStorage, TASK_LAUNCH_SCRIPT_PERMISSION

Constructor Summary

Constructors

Constructor and Description
DefaultTaskController()

Method Summary

Methods

Modifier and Type	Method and Description
void	createLogDir(TaskAttemptID taskID, boolean isCleanup) Creates task log dir
void	deleteAsUser(String user, String subDir) Delete the user's files under all of the task tracker root directories.
void	deleteLogAsUser(String user, String subDir) Delete the user's files under the userlogs directory.
void	initializeJob(String user, String jobid, org.apache.hadoop.fs.Path credentials, org.apache.hadoop.fs.Path jobConf, TaskUmbilicalProtocol taskTracker, InetSocketAddress ttAddr) This routine initializes the local file system for running a job.
int	launchTask(String user, String jobId, String attemptId, List<String> setup, List<String> jvmArguments, File currentWorkDirectory, String stdout, String stderr) Create all of the directories for the task and launches the child jvm.
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setup(org.apache.hadoop.fs.LocalDirAllocator allocator, org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage) Does initialization and setup.
void	signalTask(String user, int taskPid, ProcessTree.Signal signal) Send a signal to a task pid as the user.
void	truncateLogsAsUser(String user, List<Task> allAttempts) Run the passed command as the user

Methods inherited from class org.apache.hadoop.mapred.TaskController

getConf, getLocalDirs, getRunAsUser, logOutput, writeCommand

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultTaskController

public DefaultTaskController()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class TaskController

createLogDir

public void createLogDir(TaskAttemptID taskID,
                boolean isCleanup)
                  throws IOException

Description copied from class: TaskController

Creates task log dir

Specified by:

createLogDir in class TaskController

Parameters:

taskID - ID of the task

isCleanup - If the task is cleanup task or not

Throws:

IOException

launchTask

public int launchTask(String user,
             String jobId,
             String attemptId,
             List<String> setup,
             List<String> jvmArguments,
             File currentWorkDirectory,
             String stdout,
             String stderr)
               throws IOException

Create all of the directories for the task and launches the child jvm.

Specified by:

launchTask in class TaskController

Parameters:

user - the user name

attemptId - the attempt id

jobId - the jobId in question

setup - list of shell commands to execute before the jvm

jvmArguments - list of jvm arguments

currentWorkDirectory - the full path of the cwd for the task

stdout - the file to redirect stdout to

stderr - the file to redirect stderr to

Returns:

the exit code for the task

Throws:

IOException

initializeJob

public void initializeJob(String user,
                 String jobid,
                 org.apache.hadoop.fs.Path credentials,
                 org.apache.hadoop.fs.Path jobConf,
                 TaskUmbilicalProtocol taskTracker,
                 InetSocketAddress ttAddr)
                   throws IOException,
                          InterruptedException

This routine initializes the local file system for running a job. Details:

• Copies the credentials file from the TaskTracker's private space to the job's private space
• Creates the job work directory and set TaskTracker.JOB_LOCAL_DIR in the configuration
• Downloads the job.jar, unjars it, and updates the configuration to reflect the localized path of the job.jar
• Creates a base JobConf in the job's private space
• Sets up the distributed cache
• Sets up the user logs directory for the job

This method must be invoked in the access control context of the job owner user. This is because the distributed cache is also setup here and the access to the hdfs files requires authentication tokens in case where security is enabled.

Specified by:

initializeJob in class TaskController

Parameters:

user - the user in question (the job owner)

jobid - the ID of the job in question

credentials - the path to the credentials file that the TaskTracker downloaded

jobConf - the path to the job configuration file that the TaskTracker downloaded

taskTracker - the connection to the task tracker

ttAddr - the tasktracker's RPC address

Throws:

IOException

InterruptedException

signalTask

public void signalTask(String user,
              int taskPid,
              ProcessTree.Signal signal)

Description copied from class: TaskController

Send a signal to a task pid as the user.

Specified by:

signalTask in class TaskController

Parameters:

user - the user name

taskPid - the pid of the task

signal - the id of the signal to send

deleteAsUser

public void deleteAsUser(String user,
                String subDir)
                  throws IOException

Delete the user's files under all of the task tracker root directories.

Specified by:

deleteAsUser in class TaskController

Parameters:

user - the user name

subDir - the path relative to the user's subdirectory under the task tracker root directories.

Throws:

IOException

deleteLogAsUser

public void deleteLogAsUser(String user,
                   String subDir)
                     throws IOException

Delete the user's files under the userlogs directory.

Specified by:

deleteLogAsUser in class TaskController

Parameters:

user - the user to work as

subDir - the path under the userlogs directory.

Throws:

IOException

truncateLogsAsUser

public void truncateLogsAsUser(String user,
                      List<Task> allAttempts)
                        throws IOException

Description copied from class: TaskController

Run the passed command as the user

Specified by:

truncateLogsAsUser in class TaskController

allAttempts - the list of attempts that the JVM ran

Throws:

IOException

setup

public void setup(org.apache.hadoop.fs.LocalDirAllocator allocator,
         org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage)

Description copied from class: TaskController

Does initialization and setup.

Specified by:

setup in class TaskController

Parameters:

allocator - the local dir allocator to use

localStorage - TaskTracker's LocalStorage object