• Hadoop distributed file system (HDFS) uses a specific permissions model for files and directories. Following user levels are used in HDFS
  • Owner
  • Group
  • Others

• For each of the user mentioned above following permissions are applicable,
  • read (r)
  • write (w)
  • execute(x)

Above mentioned permissions work differently for files and directories.

• For files
  • read (r) permission – Reading a file.
  • write (w) permission – Writing a file.

• For directories.
  • The r permission lists the contents of a specific directory.
  • The w permission creates or deletes a directory.
  • The X permission accessing a child directory.

• Datasets in HDFS store as blocks in DataNodes the Hadoop cluster.
• During the execution of a MapReduce job the individual Mapper processes the blocks (Input Splits).
• If the data does not reside in the same node where the Mapper is executing the job, the data needs to be copied from the DataNode over the network to the mapper DataNode.

• Now if a MapReduce job has more than 100 Mapper and each Mapper tries to copy the data from other DataNode in the cluster simultaneously, it would cause serious network congestion which is a big performance issue of the overall system.
• Hence, data proximity to the computation is an effective and cost-effective solution which is technically termed as Data locality in Hadoop. It helps to increase the overall throughput of the system.

• Data local
  • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.

• Rack Local
  • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.
  • In this scenarios mapper and data reside on the same rack but on the different data nodes.

• Different Rack
  • In this scenario mapper and data reside on the different racks.