【问题标题】:sqoop imported data but with empty part-m-00000 files?sqoop 导入的数据,但有空的 part-m-00000 文件?
【发布时间】:2018-08-13 11:22:44
【问题描述】:

使用 Apache sqoop 将数据从 oracle 数据库导入 HDFS 时。它已导入但为空文件。

sqoop import --connect jdbc:oracle:thin:@192.168.0.15:1521:XE --username system --password system --table EMP -m 1 --target-dir /user/sinha

在没有任何数据的情况下运行它创建的 part-m-00000 文件后...在运行查询时

    Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/03/05 09:43:57 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.12.0
18/03/05 09:43:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/03/05 09:44:00 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
18/03/05 09:44:58 INFO mapreduce.JobSubmitter: number of splits:1
18/03/05 09:45:01 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1520229051986_0016
18/03/05 09:45:03 INFO impl.YarnClientImpl: Submitted application application_1520229051986_0016
18/03/05 09:45:03 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1520229051986_0016/
18/03/05 09:45:03 INFO mapreduce.Job: Running job: job_1520229051986_0016
18/03/05 09:45:54 INFO mapreduce.Job: Job job_1520229051986_0016 running in uber mode : false
18/03/05 09:45:54 INFO mapreduce.Job:  map 0% reduce 0%
18/03/05 09:46:35 INFO mapreduce.Job:  map 100% reduce 0%
18/03/05 09:46:36 INFO mapreduce.Job: Job job_1520229051986_0016 completed successfully
18/03/05 09:46:36 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=151209
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=87
        HDFS: Number of bytes written=0
        HDFS: Number of read operations=4
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=2
    Job Counters 
        Launched map tasks=1
        Other local map tasks=1
        Total time spent by all maps in occupied slots (ms)=37383
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=37383
        Total vcore-milliseconds taken by all map tasks=37383
        Total megabyte-milliseconds taken by all map tasks=38280192
    Map-Reduce Framework
        Map input records=0
        Map output records=0
        Input split bytes=87
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=546
        CPU time spent (ms)=5110
        Physical memory (bytes) snapshot=143175680
        Virtual memory (bytes) snapshot=1509150720
        Total committed heap usage (bytes)=74973184
    File Input Format Counters 
        Bytes Read=0
    File Output Format Counters 
        Bytes Written=0
18/03/05 09:46:36 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 108.9264 seconds (0 bytes/sec)
18/03/05 09:46:36 INFO mapreduce.ImportJobBase: Retrieved 0 records

不知道是什么问题? 即使我使用“eval”命令进行检查,它也只显示表的列名。

【问题讨论】:

    标签: hadoop import hdfs sqoop hadoop2


    【解决方案1】:

    查看日志,您的源表根本没有任何记录。在您的 oracle 表上执行select * 以进行验证。将一些记录添加到您的 oracle 表中,然后再次尝试 sqoop 操作。您应该能够获取数据。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2016-08-05
      • 1970-01-01
      • 2012-05-26
      • 1970-01-01
      • 2013-07-22
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多