hadoop architecture components - Wikitechy https://www.wikitechy.com/interview-questions/tag/hadoop-architecture-components/ Interview Questions Tue, 14 Sep 2021 09:49:28 +0000 en-US hourly 1 https://wordpress.org/?v=6.9 https://www.wikitechy.com/interview-questions/wp-content/uploads/2025/10/cropped-wikitechy-icon-32x32.png hadoop architecture components - Wikitechy https://www.wikitechy.com/interview-questions/tag/hadoop-architecture-components/ 32 32 What are the core components of Hadoop ? https://www.wikitechy.com/interview-questions/big-data/what-are-the-core-components-of-hadoop/ https://www.wikitechy.com/interview-questions/big-data/what-are-the-core-components-of-hadoop/#respond Mon, 12 Jul 2021 18:01:53 +0000 https://www.wikitechy.com/interview-questions/?p=281

What are the core components of Hadoop ?

  • Hadoop is an open source framework that is meant for storage and processing of big data in a distributed manner.
What are the core components of Hadoop

HDFS (Hadoop Distributed File System)

  • HDFS is the basic storage system of Hadoop.
  • The large data files running on a cluster of commodity hardware are stored in HDFS.
  • It can store data in a reliable manner even when hardware fails.
HDFS (Hadoop Distributed File System)

Core Components of Hadoop

  • Hadoop MapReduce
  • YARN

Hadoop MapReduce

  • MapReduce is the Hadoop layer that is responsible for data processing. It writes an application to process unstructured and structured data stored in HDFS.
  • It is responsible for the parallel processing of high volume of data by dividing data into independent tasks.
  • The processing is done in two phases Map and Reduce.
  • The Map is the first phase of processing that specifies complex logic code and the Reduce is the second phase of processing that specifies light-weight operations.
Hadoop MapReduce

YARN

  • The processing framework in Hadoop is YARN.
  • It is used for resource management and provides multiple data processing engines i.e. data science, real-time streaming, and batch processing.
YARN
]]> https://www.wikitechy.com/interview-questions/big-data/what-are-the-core-components-of-hadoop/feed/ 0 What are the components of HDFS and YARN ? https://www.wikitechy.com/interview-questions/big-data/what-are-the-components-of-hdfs-and-yarn/ https://www.wikitechy.com/interview-questions/big-data/what-are-the-components-of-hdfs-and-yarn/#respond Mon, 12 Jul 2021 16:57:48 +0000 https://www.wikitechy.com/interview-questions/?p=266

What are the components of HDFS and YARN ?

components of HDFS and YARN

NameNode

  • NameNode is the master node for processing metadata information for data blocks within the HDFS.

DataNode/SlaveNode

  • DataNode is the node which acts as slave node to store the data, for processing and use by the NameNode.
  • In addition to serving the client requests, the NameNode executes either of two following roles –
    • CheckpointNode.
    • BackupNode.
components of HDFS and YARN

CheckpointNode

  • It runs on a different host from the NameNode

BackupNode

  • It is a read-only NameNode which contains file system metadata information excluding the block locations.
Components of HDFS

Components of YARN

ResourceManager

  • This component receives processing requests and accordingly allocates to respective NodeManagers depending on processing needs.

NodeManager

  • It executes tasks on each single Data Node.
Components of YARN
]]> https://www.wikitechy.com/interview-questions/big-data/what-are-the-components-of-hdfs-and-yarn/feed/ 0