what is big data hadoop - Wikitechy https://www.wikitechy.com/interview-questions/tag/what-is-big-data-hadoop/ Interview Questions Wed, 22 Sep 2021 05:53:05 +0000 en-US hourly 1 https://wordpress.org/?v=6.9 https://www.wikitechy.com/interview-questions/wp-content/uploads/2025/10/cropped-wikitechy-icon-32x32.png what is big data hadoop - Wikitechy https://www.wikitechy.com/interview-questions/tag/what-is-big-data-hadoop/ 32 32 What is the difference between Apache Hadoop and Cloudera in big data ? https://www.wikitechy.com/interview-questions/cloudera-impala/what-is-the-difference-between-apache-hadoop-and-cloudera-in-big-data/ https://www.wikitechy.com/interview-questions/cloudera-impala/what-is-the-difference-between-apache-hadoop-and-cloudera-in-big-data/#respond Mon, 19 Jul 2021 06:38:24 +0000 https://www.wikitechy.com/interview-questions/?p=1023

Difference between Apache Hadoop and Cloudera in big data

  • Apache Hadoop is the Hadoop distribution from Apache group.
  • Cloudera Hadoop has its own supply of Hadoop which is designed on top of Apache Hadoop. so it does not have latest release of Hadoop.
  • Cloudera Hadoop contains extra tools. while Hadoop distributions like Cloudera Search, Impala, Cloudera Navigator and Cloudera Manager these are not involved in Cloudera Hadoop.
  • These additional tools turns out Cloudera Hadoop to be slightly open source and proprietary.
Distribution hadoop
  • Apache Hadoop is very important to use to cope up with any data, no matter how large or what type or complex it is. It works like a fluid way or can say it is very faster, cheaper and reliable analytic tool than others.
  • Even, it can easily work on unstructured, and schemaless data and people can easily ground their data into a format without reformating or putting any other efforts.
  • Apart from this, via Apache Hadoop various programming model has been simplified to allow you to quickly write and test the software in distributed systems.
  • This is an open-source Apache Hadoop distribution usually, focus on enterprise-class deployments.
  • Cloudera Inc is a very popular American software company, known for providing Apache Hadoop-based software, services, and full support for the same.
  • With Cloudera, it is said that Hadoop is just a starting to create your data management strategy, and you can easily use the same to add the security and other various functions to create an enterprise-grade foundation for your data.
]]> https://www.wikitechy.com/interview-questions/cloudera-impala/what-is-the-difference-between-apache-hadoop-and-cloudera-in-big-data/feed/ 0 Why do we need Data Locality in Hadoop ? https://www.wikitechy.com/interview-questions/big-data/why-do-we-need-data-locality-in-hadoop/ https://www.wikitechy.com/interview-questions/big-data/why-do-we-need-data-locality-in-hadoop/#respond Mon, 12 Jul 2021 18:21:39 +0000 https://www.wikitechy.com/interview-questions/?p=287

Why do we need Data Locality in Hadoop ?

Data Locality in Hadoop
  • Datasets in HDFS store as blocks in DataNodes the Hadoop cluster.
  • During the execution of a MapReduce job the individual Mapper processes the blocks (Input Splits).
  • If the data does not reside in the same node where the Mapper is executing the job, the data needs to be copied from the DataNode over the network to the mapper DataNode.
Datasets in HDFS - Data Locality in Hadoop
  • Now if a MapReduce job has more than 100 Mapper and each Mapper tries to copy the data from other DataNode in the cluster simultaneously, it would cause serious network congestion which is a big performance issue of the overall system.
  • Hence, data proximity to the computation is an effective and cost-effective solution which is technically termed as Data locality in Hadoop. It helps to increase the overall throughput of the system.

Types of data locality

  • Data local
    • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.
  • Rack Local
    • In this type data and the mapper resides on the same node. This is the closest proximity of data and the most preferred scenario.
    • In this scenarios mapper and data reside on the same rack but on the different data nodes.
  • Different Rack
    • In this scenario mapper and data reside on the different racks.
Types of data locality
]]> https://www.wikitechy.com/interview-questions/big-data/why-do-we-need-data-locality-in-hadoop/feed/ 0 What are the features of Hadoop ? https://www.wikitechy.com/interview-questions/big-data/what-are-the-features-of-hadoop/ https://www.wikitechy.com/interview-questions/big-data/what-are-the-features-of-hadoop/#respond Mon, 12 Jul 2021 17:55:46 +0000 https://www.wikitechy.com/interview-questions/?p=279

What are the features of Hadoop ?

Hadoop supports the storage and processing of big data. It is the best solution for handling big data challenges. Some important features of Hadoop are –

What are the features of Hadoop

Open Source

  • Hadoop is an open source framework which means it is available free of cost.
  • Also, the users are allowed to change the source code as per their requirements.

Distributed Processing

  • Hadoop supports distributed processing of data i.e. faster processing.
  • The data in Hadoop HDFS is stored in a distributed manner and MapReduce is responsible for the parallel processing of data.

Fault Tolerance

  • Hadoop is highly fault-tolerant. It creates three replicas for each block at different nodes, by default.
  • This number can be changed according to the requirement. So, we can recover the data from another node if one node fails.
  • The detection of node failure and recovery of data is done automatically.

Reliability

  • Hadoop stores data on the cluster in a reliable manner that is independent of machine.
  • So, the data stored in Hadoop environment is not affected by the failure of the machine.

Scalability

  • Another important feature of Hadoop is the scalability. It is compatible with the other hardware and we can easily ass the new hardware to the nodes.
Scalability in hadoop

Economic

  • Apache Hadoop is not very expensive as it runs on a cluster of commodity hardware.
  • Hadoop also provides huge cost saving also as it is very easy to add more nodes on the fly here. So if requirement increases, then you can increase nodes as well without any downtime and without requiring much of pre-planning.
Economic in hadoop

Easy to use

  • No need of client to deal with distributed computing, the framework takes care of all the things. So this feature of Hadoop is easy to use.

Data Locality

  • This one is a unique features of Hadoop that made it easily handle the Big Data. Hadoop works on data locality principle which states that move computation to data instead of data to computation.
  • When a client submits the MapReduce algorithm, this algorithm is moved to data in the cluster rather than bringing data to the location where the algorithm is submitted and then processing it.
Data Locality

High Availability

  • The data stored in Hadoop is available to access even after the hardware failure. In case of hardware failure, the data can be accessed from another path.
High Availability
]]> https://www.wikitechy.com/interview-questions/big-data/what-are-the-features-of-hadoop/feed/ 0 What is Big Data ? https://www.wikitechy.com/interview-questions/big-data/what-is-big-data/ https://www.wikitechy.com/interview-questions/big-data/what-is-big-data/#respond Mon, 12 Jul 2021 16:21:38 +0000 https://www.wikitechy.com/interview-questions/?p=256

What is Big Data ?

  • Big Data is a term associated with complex and large datasets.
  • A relational database cannot handle big data, and that’s why special tools and methods are used to perform operations on a vast collection of data.
  • Big data enables companies to understand their business better and helps them derive meaningful information from the unstructured and raw data collected on a regular basis.
  • Big data also allows companies to make better business decisions backed by data.
What is Big Data

Characteristics Of Big Data

  • Volume – The Size of data plays a very essential role in determining value out of data.
  • Variety – The next aspect of Big Data is its variety. Variety refers to heterogeneous sources and the nature of data, both structured and unstructured.
  • Velocity – The term ‘velocity’ refers to the speed of generation of data.
  • Variability – The process of being able to handle and manage the data effectively.
]]> https://www.wikitechy.com/interview-questions/big-data/what-is-big-data/feed/ 0