Illegal partition for Big Data (3)


I was writing a sample MR program with Partitioner for 6th lab exercise. My job failed with below given teasing exception.

16/09/10 18:28:40 WARN mapred.LocalJobRunner: job_local938197532_0001
java.lang.Exception: Illegal partition for Big Data (3)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(
        at org.apache.hadoop.mapred.LocalJobRunner$
Caused by: Illegal partition for Big Data (3)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(
        at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(
        at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(
        at org.apache.hadoop.mapred.MapTask.runNewMapper(
        at org.apache.hadoop.mapred.LocalJobRunner$Job$
        at java.util.concurrent.Executors$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$
16/09/10 18:28:41 INFO mapreduce.Job: Job job_local938197532_0001 failed with state FAILED due to: NA


Actually there were only 3 reducers 0, 1 and 2. I mentioned 3 in my partitioner instead of 2. Correcting the right reducer number in the partitioner solved this problem.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s