• Hadoop distributed file system (HDFS) uses a specific permissions model for files and directories. Following user levels are used in HDFS
  • Owner
  • Group
  • Others

• For each of the user mentioned above following permissions are applicable,
  • read (r)
  • write (w)
  • execute(x)

Above mentioned permissions work differently for files and directories.

• For files
  • read (r) permission – Reading a file.
  • write (w) permission – Writing a file.

• For directories.
  • The r permission lists the contents of a specific directory.
  • The w permission creates or deletes a directory.
  • The X permission accessing a child directory.

• Apache Spark is an open-source distributed cluster-computing framework.
• Spark is a data processing engine developed to provide faster and ease-of-use analytics than Hadoop MapReduce.
• Before Apache Software Foundation took possession of Spark, it was under the control of University of California, Berkeley’s AMP Lab.

• Apache Hadoop is an open-source framework written in Java that allows us to store and process Big Data in a distributed environment, across various clusters of computers using simple programming constructs.
• To do this, Hadoop uses an algorithm called MapReduce, which divides the task into small parts and assigns them to a set of computers.
• Hadoop also has its own file system, Hadoop Distributed File System (HDFS), which is based on the Google File System (GFS).
• HDFS is designed to run on low-cost hardware.