[Solved-1 Solution] A way to read table data from Mysql to Pig ?

What is Mysql:

  • MySQL is an open source relational database management system (RDBMS) based on Structured Query Language (SQL).
  • MySQL runs on virtually all platforms, including Linux, UNIX, and Windows. Although it can be used in a wide range of applications, MySQL is most often associated with web-based applications.

What is read table data

  • Reads a file in table format and creates a data frame from it, with cases corresponding to lines and variables to fields in the file.

Problem :

How to read table data from mysql to pig ?

Solution 1:

  • DBStorage only supports saving results to a database.
  • To load data from MySQL we could look into a project called sqoop.
  • Sqoop helps to copy a data from a database to HDFS, or we could perform a mysql dump and then copy the file into HDFS.
  • Both ways required some interaction and cannot be directly used from inside Pig.
  • A third option look into writing a Pig LoadFunc.
  • Its not much difficult, we need to pass much the same options as DBStorage (driver, connection credentials and a SQL query to execute), and we can probably use some result set metadata inspection to auto generate a schema too.

Related Searches to A way to read table data from Mysql to Pig