Speculative execution is a mechanism of running
multiple copies of same map or reduce tasks on different slave nodes to cope
with individual Machine performance.
In large clusters of hundreds of machines, there may
be machines which are not performing as fast as others. This may result in
delays in a full job due to only one machine not performing well. To avoid
this, speculative execution in hadoop can run multiple copies of same map or
reduce task on different slave nodes. The results from first node to finish are
used.
If other copies were executing speculatively, Hadoop
tells the Task Trackers to abandon the tasks and discard their outputs.
