hive hadoop - Wikitechy https://www.wikitechy.com/interview-questions/tag/hive-hadoop/ Interview Questions Wed, 15 Sep 2021 05:14:15 +0000 en-US hourly 1 https://wordpress.org/?v=6.9.4 https://www.wikitechy.com/interview-questions/wp-content/uploads/2025/10/cropped-wikitechy-icon-32x32.png hive hadoop - Wikitechy https://www.wikitechy.com/interview-questions/tag/hive-hadoop/ 32 32 What is workflow when using Pig & R ? https://www.wikitechy.com/interview-questions/apache-pig/what-is-workflow-when-using-pig-r/ https://www.wikitechy.com/interview-questions/apache-pig/what-is-workflow-when-using-pig-r/#respond Mon, 12 Jul 2021 06:37:19 +0000 https://www.wikitechy.com/interview-questions/?p=185

Workflow when using Pig & R

  • Pig programming language is used for obtaining and manipulating data perhaps doing otherwise with UDFs.
  • Pig is used for filtering, cleaning, transforming and otherwise preparing large data sets.
  • Depending on the workflow, user might throw a Hive table on top of the Pig output and then use Hive to prepare subsets of data for R.
  • R is used for analysis and modeling and it’s also act as prototype models.
  • There are ways to connect directly to Hive from R using the Hive JDBC drivers.
  • We use R for early data exploration and examination of results.
  • In a normal workflow, Pig is used to perform the ETL(Extract, Transform, Load) functionality and transform it load the data into HDFS and analysts utilizes HIVE or extensions like RHIVE to perform the analysis on large volume of data available in HDFS.
  • The Comparison between Pig & R is invalid for ever because the pig is for large scale data and R definitely is not.
]]> https://www.wikitechy.com/interview-questions/apache-pig/what-is-workflow-when-using-pig-r/feed/ 0 What is the history of Pig in Hadoop ? https://www.wikitechy.com/interview-questions/apache-pig/what-is-the-history-of-pig-in-hadoop/ https://www.wikitechy.com/interview-questions/apache-pig/what-is-the-history-of-pig-in-hadoop/#respond Mon, 12 Jul 2021 05:12:12 +0000 https://www.wikitechy.com/interview-questions/?p=153

History of Pig in Hadoop

  • In 2006 Pig was developed by Yahoo Research for particular way of creating and executing MapReduce jobs on very large data sets.
  • Apache Software Foundation was moved into 2007.
  • Pig started a project in Yahoo! Research, where Yahoo! scientists designed it and produced an implementation.
  • The MapReduce paradigm presented by Hadoop β€œis too low-level and rigid, and leads to a great deal of custom user code that is hard to maintain and reuse.”
  • Users were not comfortable with MapReduce languages such as SQL.
  • Sort the problem a new language called Pig Latin that we have designed to fit between the declarative style of SQL, and the low-level, procedural style of MapReduce.”
  • Hadoop users adopt Pig. So, a team of development engineers was assembled to take the research prototype and build into a production-quality product. About this same time, in fall 2007, Pig was open sourced via the Apache Incubator.
  • The first Pig release in September 2008, were Pig graduated from the Incubator and became a subproject of Apache Hadoop.
Bigdata history
]]> https://www.wikitechy.com/interview-questions/apache-pig/what-is-the-history-of-pig-in-hadoop/feed/ 0