12/05/28 09:33:37 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-Hadoop/mapred/staging/Hadoop-1983717839/.staging/job_local_0001
12/05/28 09:33:37 ERROR security.UserGroupInformation: PriviledgedActionException as:Hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://localhost:54310/user/Hadoop/input/cxu_index already exists
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://localhost:54310/user/Hadoop/input/cxu_index already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:132)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:872)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:476)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)
at com.payegis.caesar.hdfsindex.Main.main(Main.java:58)
==================================================================================================
hadoop 由于进行的是耗费资源的计算,生产的结果默认是不能被覆盖的,
因此中间结果输出目录一定不能存在,否则出现这个错误。
就是这句代码:
FileOutputFormat.setOutputPath( job, new Path(args[1]) );