MapReduce Job Execution Process – Map Task Execution

Hi Hadoopers,

The user had submitted his job. He has permissions. We have slots in the cluster. Job setup is completed. We look at 4th circle given below – The Map Task Execution

hadoop037-job-submission-1

 

The below given diagram depicts the Map Task Execution.

hadoop041-map-task-execution

  1. The task tracker launches the Map  Task
  2. The Map task read the jar file given the user. This is what we write in Eclipse. In the entire frameworks, this is what our contribution 🙂
    The Map task also reads the job config (input path, output path etc). It gets everything from HDFS, as all these are already uploaded to HDFS initially.
  3. The Map task reads the input splits from HDFS
  4. From the input splits, Map task creates the record.
  5. The Map task invokes the user Mapper with the record
  6. The mapper writes intermediate output
  7. The task sort them based on key and flush them to disk.
  8. Map task informs Task Tracker about the completion of the job.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s