解压
将sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz拷贝到linux下，然后解压
tar –zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
然后进入新生成的目录
cd sqoop-1.4.6.bin__hadoop-2.0.4-alpha
cd /conf
cp sqoop-env-template.sh sqoop-env.sh
修改sqoop-env.sh(下面的目录换成你自己的安装目录)
|
#Set path to where bin/hadoop is available export HADOOP_COMMON_HOME=/usr/tools/hadoop-2.6.4
#Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/usr/tools/hadoop-2.6.4
#set the path to where bin/hbase is available export HBASE_HOME=/usr/tools/hbase-1.2.0
#Set the path to where bin/hive is available export HIVE_HOME=/usr/tools/apache-hive-2.0.0-bin
#Set the path for where zookeper config dir is export ZOOCFGDIR=/usr/zookeeper-3.4.5
|
修改环境变量：
vi /etc/profile
#sqoop
export SQOOP_HOME=/usr/tools/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
export PATH=$PATH:$SQOOP_HOME/bin
保存后使其生效
source /etc/profile
检查sqoop是否安装成功：sqoop help
简单测试：
开启mysql远程访问：
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123qwe';
FLUSH PRIVILEGES;
列出链接数据所有表
sqoop list-tables -connect jdbc:mysql://localhost:3306/test --username root --password 123qwe
|
这一步可能会报错： Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 20/05/15 17:16:25 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 20/05/15 17:16:25 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 20/05/15 17:16:25 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 20/05/15 17:16:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59) at org.apache.sqoop.manager.CatalogQueryManager.listTables(CatalogQueryManager.java:102) at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) 原因：/opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/lib目录下确实mysql的jar驱动 解决办法：把hive的lib下的mysql-connector-java-5.1.47.jar复制到sqoop的lib目录下即可 |
查看结果与MySQL的show tables;结果一致
注意，这里会报警告：
|
Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /opt/module/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. |
这主要是因为集群中没有安装HCatal和accumulo环境变量，但sqoop默认会检测这两个组件。若非要去掉这两个检测，可以修改bin/config-sqoop文件，很容易找到这两个组件的检测位置。把下面两行注释掉即可：（你也可以选择安装HCatal和accumulo这两个组件）