日期:2014-05-16  浏览次数:20423 次

Sqoop 1.99.3 with hadoop-2.3.0 使用 2

连接建好 就开始建Job了

sqoop:000> create job --xid 1 --type import

Creating job for connection with id 1

Please fill following values to create new job object

Name: dimDate

?

Database configuration

?

Schema name: dbo

Table name: dimDate

Table SQL statement:

Table column names:

Partition column name:

Nulls in partition column:

Boundary query:

?

Output configuration

?

Storage type:

? 0 : HDFS

Choose: 0

Output format:

? 0 : TEXT_FILE

? 1 : SEQUENCE_FILE

Choose: 0

Compression format:

? 0 : NONE

? 1 : DEFAULT

? 2 : DEFLATE

? 3 : GZIP

? 4 : BZIP2

? 5 : LZO

? 6 : LZ4

? 7 : SNAPPY

Choose: 0

Output directory: /home/dimDate

?

Throttling resources

?

Extractors:

Loaders:

New job was successfully created with validation status FINE ?and persistent id 1

?

-------------------------------------------------------------

跑job?

sqoop:000> start job --jid 1

Submission details

Job ID: 1

Server URL: http://localhost:12000/sqoop/

Created by: root

Creation date: 2014-03-20 07:01:12 PDT

Lastly updated by: root

External ID: job_1395223907193_0001

? ? ? ? http://localhost.localdomain:8088/proxy/application_1395223907193_0001/

2014-03-20 07:01:12 PDT: BOOTING ?- Progress is not available

?

?查看?http://XXXXXXX:8088/proxy/application_1395223907193_0001/

发觉job 在跑

使用命令也可以查看状态

?

sqoop:000> status job --jid 1

Submission details

Job ID: 1

Server URL: http://localhost:12000/sqoop/

Created by: root

Creation date: 2014-03-20 07:01:12 PDT

Lastly updated by: root

External ID: job_1395223907193_0001

? ? ? ? http://localhost.localdomain:8088/proxy/application_1395223907193_0001/

2014-03-20 07:03:31 PDT: RUNNING ?- 0.00 %

?

过些时候再查?

sqoop:000> status job --jid 1

Submission details

Job ID: 1

Server URL: http://localhost:12000/sqoop/

Created by: root

Creation date: 2014-03-20 07:01:12 PDT

Lastly updated by: root

External ID: job_1395223907193_0001

? ? ? ? http://localhost.localdomain:8088/proxy/application_1395223907193_0001/

2014-03-20 07:04:46 PDT: SUCCEEDED

Counters:

? ? ? ? org.apache.hadoop.mapreduce.JobCounter

? ? ? ? ? ? ? ? SLOTS_MILLIS_MAPS: 772203

? ? ? ? ? ? ? ? MB_MILLIS_MAPS: 790735872

? ? ? ? ? ? ? ? TOTAL_LAUNCHED_MAPS: 10

? ? ? ? ? ? ? ? MILLIS_MAPS: 772203

? ? ? ? ? ? ? ? VCORES_MILLIS_MAPS: 772203

? ? ? ? ? ? ? ? SLOTS_MILLIS_REDUCES: 0

? ? ? ? ? ? ? ? OTHER_LOCAL_MAPS: 10

? ? ? ? org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter

? ? ? ? ? ? ? ? BYTES_WRITTEN: 129332

? ? ? ? org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter

? ? ? ? ? ? ? ? BYTES_READ: 0

? ? ? ? org.apache.hadoop.mapreduce.TaskCounter

? ? ? ? ? ? ? ? MAP_INPUT_RECORDS: 0

? ? ? ? ? ? ? ? MERGED_MAP_OUTPUTS: 0

? ? ? ? ? ? ? ? PHYSICAL_MEMORY_BYTES: 612765696

? ? ? ? ? ? ? ? SPILLED_RECORDS: 0

? ? ? ? ? ? ? ? COMMITTED_HEAP_BYTES: 161021952

? ? ? ? ? ? ? ? CPU_MILLISECONDS: 8390

? ? ? ? ? ? ? ? FAILED_SHUFFLE: 0

? ? ? ? ? ? ? ? VIRTUAL_MEMORY_BYTES: 3890085888

? ? ? ? ? ? ? ? SPLIT_RAW_BYTES: 1391

? ? ? ? ? ? ? ? MAP_OUTPUT_RECORDS: 1188

? ? ? ? ? ? ? ? GC_TIME_MILLIS: 2962

? ? ? ? org.apache.hadoop.mapreduce.FileSystemCounter

? ? ? ? ? ? ? ? FILE_WRITE_OPS: 0

? ? ? ? ? ? ? ? FILE_READ_OPS: 0

? ? ? ?