History of Pig in Hadoop
- In 2006 Pig was developed by Yahoo Research for particular way of creating and executing MapReduce jobs on very large data sets.
- Apache Software Foundation was moved into 2007.
- Pig started a project in Yahoo! Research, where Yahoo! scientists designed it and produced an implementation.
- The MapReduce paradigm presented by Hadoop “is too low-level and rigid, and leads to a great deal of custom user code that is hard to maintain and reuse.”
- Users were not comfortable with MapReduce languages such as SQL.
- Sort the problem a new language called Pig Latin that we have designed to fit between the declarative style of SQL, and the low-level, procedural style of MapReduce.”
- Hadoop users adopt Pig. So, a team of development engineers was assembled to take the research prototype and build into a production-quality product. About this same time, in fall 2007, Pig was open sourced via the Apache Incubator.
- The first Pig release in September 2008, were Pig graduated from the Incubator and became a subproject of Apache Hadoop.
