sqoop - Starting Sqoop - apache sqoop - sqoop tutorial - sqoop hadoop
How to Start Sqoop?
- Sqoop is a command line tool with following structure
sqoop TOOL PROPERTY_ARGS SQOOP_ARGS [-- EXTRA_ARGS]Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- TOOL indicates the operation eg: "import", "export".
- PROPERTY_ARGS are Java properties in the format "-Dname=value"
- SQOOP_ARGS mention various Sqoop parameters
- EXTRA_ARGS are for specialized connectors, separated from the SQOOP_ARGS with a "--"
Example:
% sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets -m 1
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- Type "sqoop help" to get all the tools accessible:
Learn Sqoop - Sqoop tutorial - Starting Sqoop - Sqoop examples - Sqoop programs
What happens in backend
- When you have definite to move data from RDBMS to HDFS then the first product which comes into usage is Apache Sqoop.
- When you demand to bring the data to HDFS then the following things happen.
- Sqoop asks for metadata information from Relation DB.
- Relational DB returns the required request.
- Based on metadata information Sqoop generates java classes.
- Based on primary id partitioning happens in table as multiple mappers will importing data as the same time.
Sqoop related tags : sqoop import , sqoop interview questions , sqoop export , sqoop commands , sqoop user guide , sqoop documentation
Starting, Stopping, and Accessing the Sqoop 2 Server
- Starting the Sqoop 2 Server
- After you have completed all of the required configuration steps, you can start Sqoop 2 server:
$ sudo /sbin/service sqoop2-server start
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- Starting the Sqoop 2 Server
- After you have completed all of the required configuration steps, you can start Sqoop 2 server:
- Stopping the Sqoop 2 Server
$ sudo /sbin/service sqoop2-server stop
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- Checking that the Sqoop 2 Server has Started
- You can verify whether the server has started correctly by connecting to its HTTP interface. The simplest way is to get the server version using following command:
$ wget -qO - localhost:12000/sqoop/version
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- You should get a text fragment in JSON format similar to the following:
{"version":"1.99.2-cdh5.0.0",...}
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Accessing the Sqoop 2 Server with the Sqoop 2 Client
- You should get a text fragment in JSON format similar to the following:
- Start the Sqoop 2 client:
sqoop2
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- Identify the host where your server is running (we will use localhost in this example):
sqoop:000> set server --host localhost
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
- Test the connection by running the command show version --all to obtain the version number from server.
Output
sqoop:000> show version --all
server version:
Sqoop 1.99.2-cdh5.0.0 revision ...
Compiled by jenkins on ...
client version:
Sqoop 1.99.2-cdh5.0.0 revision ...
Compiled by jenkins on ...
Protocol version:
[1]
