hat are the configuration parameters in a “MapReduce” program ?

Mapreduce Example
  • The main configuration parameters in “MapReduce” framework are:
    • Input location of Jobs in the distributed file system
    • Output location of Jobs in the distributed file system
    • The input format of data
    • The output format of data
    • The class which contains the map function
    • The class which contains the reduce function
    • JAR file which contains the mapper, reducer and the driver classes
Mapreduce configuration parameters

Sample Code

public static void main(String [] args) throws Exception
{
Configuration c=new Configuration();
String[] files=new GenericOptionsParser(c,args).getRemainingArgs();
Path input=new Path(files[0]);
Path output=new Path(files[1]);
Job j=new Job(c,"wordcount");
j.setJarByClass(WordCount.class);
j.setMapperClass(MapForWordCount.class);
j.setReducerClass(ReduceForWordCount.class);
j.setOutputKeyClass(Text.class);
j.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(j, input);
FileOutputFormat.setOutputPath(j, output);
System.exit(j.waitForCompletion(true)?0:1);
}

Categorized in:

Tagged in:

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,