mapreduce code - Interview Questions https://www.wikitechy.com/interview-questions/tag/mapreduce-code/ Interview Questions Tue, 14 Sep 2021 07:11:33 +0000 en-US hourly 1 https://wordpress.org/?v=6.1.1 What are the configuration parameters in a “MapReduce” program ? https://www.wikitechy.com/interview-questions/big-data/what-are-the-configuration-parameters-in-a-mapreduce-program/ https://www.wikitechy.com/interview-questions/big-data/what-are-the-configuration-parameters-in-a-mapreduce-program/#respond Mon, 12 Jul 2021 18:08:35 +0000 https://www.wikitechy.com/interview-questions/?p=283

hat are the configuration parameters in a “MapReduce” program ?

Mapreduce Example
  • The main configuration parameters in “MapReduce” framework are:
    • Input location of Jobs in the distributed file system
    • Output location of Jobs in the distributed file system
    • The input format of data
    • The output format of data
    • The class which contains the map function
    • The class which contains the reduce function
    • JAR file which contains the mapper, reducer and the driver classes
Mapreduce configuration parameters

Sample Code

[pastacode lang=”java” manual=”public%20static%20void%20main(String%20%5B%5D%20args)%20throws%20Exception%0A%7B%0A%20%20%20%20Configuration%20c%3Dnew%20Configuration()%3B%0A%20%20%20%20String%5B%5D%20files%3Dnew%20GenericOptionsParser(c%2Cargs).getRemainingArgs()%3B%0A%20%20%20%20Path%20input%3Dnew%20Path(files%5B0%5D)%3B%0A%20%20%20%20Path%20output%3Dnew%20Path(files%5B1%5D)%3B%0A%20%20%20%20Job%20j%3Dnew%20Job(c%2C%22wordcount%22)%3B%0A%20%20%20%20j.setJarByClass(WordCount.class)%3B%0A%20%20%20%20j.setMapperClass(MapForWordCount.class)%3B%0A%20%20%20%20j.setReducerClass(ReduceForWordCount.class)%3B%0A%20%20%20%20j.setOutputKeyClass(Text.class)%3B%0A%20%20%20%20j.setOutputValueClass(IntWritable.class)%3B%0A%20%20%20%20FileInputFormat.addInputPath(j%2C%20input)%3B%0A%20%20%20%20FileOutputFormat.setOutputPath(j%2C%20output)%3B%0A%20%20%20%20System.exit(j.waitForCompletion(true)%3F0%3A1)%3B%0A%7D” message=”” highlight=”” provider=”manual”/]
]]> https://www.wikitechy.com/interview-questions/big-data/what-are-the-configuration-parameters-in-a-mapreduce-program/feed/ 0