日期:2014-05-16  浏览次数:21682 次

hive Required table missing : "`DBS`" in Catalog "" Schema "

最近需要提取一些数据,故开始使用hive,本机搭建了一个hive客户端环境,但是始终有问题,在本机装好了mysql以后,老是报

?

?

Caused by: org.datanucleus.store.rdbms.exceptions.MissingTableException: Required table missing : "`DBS`" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables"
        at org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractTable.java:455)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2689)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148)
        at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952)
        at org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919)
        at org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356)
        at org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332)
        at org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149)

?

?

依据堆栈提示,然后下得jar文件主要是看 ?AbstractTable.exists(AbstractTable.java:455)?

?

?

 if ((type == null) || ((allowDDLOutput()) && (this.storeMgr.getDdlWriter() != null) && (this.storeMgr.getCompleteDDL())))
    {
      if (!auto_create)
      {
        this.existsInDatastore = Boolean.FALSE;
        throw new MissingTableException(getCatalogName(), getSchemaName(), toString());
      }

?

? 455 行 便是 异常抛出之地。

?依据 hive错误提示

?

?"datanucleus.autoCreateTables" = true

? 配置了hive-site.xml不行,故结合上面的代码看,猜测是autoCreate没有传递进去,故开始跟踪这个值的设置的地方,最终跟踪到org.datanucleus.store.mapped.MappedStoreManager,其中有一段很关键的代码如下

?

?

 if ((this.readOnlyDatastore) || (this.fixedDatastore))
    {
      this.autoCreateTables = false;
      this.autoCreateColumns = false;
      this.autoCreateConstraints = false;
    }
    else
    {
      boolean autoCreateSchema = conf.getBooleanProperty("datanucleus.autoCreateSchema");
      if (autoCreateSchema)
      {
        this.autoCreateTables = true;
        this.autoCreateColumns = true;
        this.autoCreateConstraints = true;
      }
      else
      {
        this.autoCreateColumns = conf.getBooleanProperty("datanucleus.autoCreateColumns");
        this.autoCreateTables = conf.getBooleanProperty("datanucleus.autoCreateTables");
        this.autoCreateConstraints = conf.getBooleanProperty("datanucleus.autoCreateConstraints");
      }
    }

?

? ?看来关键是?this.readOnlyDatastore ? ? ? ?this.fixedDatastore 这2个字段

?

? 而且autoCreateSchema 这个设置为true 就可以决定了其他的设置,所以其他设置在此都无效了。

?

? 继续追踪org.datanucleus.store.AbstractStoreManager 发现了这2个字段的设置代码

?

?

?

 this.readOnlyDatastore = conf.getBooleanProperty("datanucleus.readOnlyDatastore");
 this.fixedDatastore = conf.getBooleanProperty("datanucleus.fixedDatastore");
?

? 原来问题再这里,再次修改hive-site.xml文件,ok,错误没有了。

?

?