Sqoop

Sqoop

·          Install and configure Sqoop on cluster

·          Connecting to RDBMS

·          Installing Mysql

·          Import data from Mysql to hive

·          Export data to Mysql

·          Internal mechanism of import/export

Best Reference  document  at  http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html

Importing  data  from  to  HDFS from RDBMS

From HDFS edge node or from client -  to see list of all available databases

Some  familiar commands

• sqoop list-databases  --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db"  --username=retail_dba  --password=cloudera    --warehouse-dir=/user/cloudera/sqoop_import

mysql -u retail_dba -p

sqoop list-databases  --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db"  --username=retail_dba  --password=clouderamysql -u retail_dba -p

CREATE TABLE employees_export (

emp_no int(11) NOT NULL,

birth_date date NOT NULL,

first_name varchar(14) NOT NULL,

last_name varchar(16) NOT NULL,

gender enum('M','F') NOT NULL,

hire_date date NOT NULL,

PRIMARY KEY (emp_no)

sqoop import  --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db"  --username=retail_dba  --password=cloudera --table employees_export1  --warehouse-dir=/user/cloudera/sqoop_import -m 1