• Hadoop distributed file system (HDFS) uses a specific permissions model for files and directories. Following user levels are used in HDFS
  • Owner
  • Group
  • Others

• For each of the user mentioned above following permissions are applicable,
  • read (r)
  • write (w)
  • execute(x)

Above mentioned permissions work differently for files and directories.

• For files
  • read (r) permission – Reading a file.
  • write (w) permission – Writing a file.

• For directories.
  • The r permission lists the contents of a specific directory.
  • The w permission creates or deletes a directory.
  • The X permission accessing a child directory.

• NFS (Network File System) is one of the oldest and popular distributed file storage systems. Whereas HDFS (Hadoop Distributed File System) is the recently used and popular one to handle big data.

• Datasets in HDFS store as blocks in DataNodes the Hadoop cluster.
• During the execution of a MapReduce job the individual Mapper processes the blocks (Input Splits).
• If the data does not reside in the same node where the Mapper is executing the job, the data needs to be copied from the DataNode over the network to the mapper DataNode.

• Now if a MapReduce job has more than 100 Mapper and each Mapper tries to copy the data from other DataNode in the cluster simultaneously, it would cause serious network congestion which is a big performance issue of the overall system.
• Hence, data proximity to the computation is an effective and cost-effective solution which is technically termed as Data locality in Hadoop. It helps to increase the overall throughput of the system.

• Data local
  • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.

• Rack Local
  • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.
  • In this scenarios mapper and data reside on the same rack but on the different data nodes.

• Different Rack
  • In this scenario mapper and data reside on the different racks.

• Hadoop is an open source framework which means it is available free of cost.
• Also, the users are allowed to change the source code as per their requirements.

• Hadoop supports distributed processing of data i.e. faster processing.
• The data in Hadoop HDFS is stored in a distributed manner and MapReduce is responsible for the parallel processing of data.

• Hadoop is highly fault-tolerant. It creates three replicas for each block at different nodes, by default.
• This number can be changed according to the requirement. So, we can recover the data from another node if one node fails.
• The detection of node failure and recovery of data is done automatically.

• Hadoop stores data on the cluster in a reliable manner that is independent of machine.
• So, the data stored in Hadoop environment is not affected by the failure of the machine.

• Another important feature of Hadoop is the scalability. It is compatible with the other hardware and we can easily ass the new hardware to the nodes.

• Apache Hadoop is not very expensive as it runs on a cluster of commodity hardware.
• Hadoop also provides huge cost saving also as it is very easy to add more nodes on the fly here. So if requirement increases, then you can increase nodes as well without any downtime and without requiring much of pre-planning.

• No need of client to deal with distributed computing, the framework takes care of all the things. So this feature of Hadoop is easy to use.

• This one is a unique features of Hadoop that made it easily handle the Big Data. Hadoop works on data locality principle which states that move computation to data instead of data to computation.
• When a client submits the MapReduce algorithm, this algorithm is moved to data in the cluster rather than bringing data to the location where the algorithm is submitted and then processing it.

• The data stored in Hadoop is available to access even after the hardware failure. In case of hardware failure, the data can be accessed from another path.

• NameNode is the master node for processing metadata information for data blocks within the HDFS.

• DataNode is the node which acts as slave node to store the data, for processing and use by the NameNode.
• In addition to serving the client requests, the NameNode executes either of two following roles –
  • CheckpointNode.
  • BackupNode.

• It runs on a different host from the NameNode

• It is a read-only NameNode which contains file system metadata information excluding the block locations.

• This component receives processing requests and accordingly allocates to respective NodeManagers depending on processing needs.

• It executes tasks on each single Data Node.