日期:2014-05-16  浏览次数:20513 次

OpenStack从数据库恢复Volume状态

问题

OpenStack中很容易导致数据库和真实状态不一致的情况。因为OpenStack中操作基本都是分步完成的,从api接受请求到调度再到具体的操作节点,每一步都有可能更新数据库状态,如果哪一个出错就会直接抛出异常导致整个操作链中断,然后数据库就处于上一个操作后的更新状态。比较典型的就是删除实例,如果在nova-compute出错那这个实例的状态就可能永远处于deleting状态了。

现在我遇到这样一个问题,我有一个Volume挂载在一个实例上,但是不知道什么原因,这个Volume与这个实例的联系断了,在nova-volume通过tgtadm查看发现已经没有客户端连接到该Volume了。但是数据库中该记录还在,这导致以下结果:
1) 删除实例时无法删除,提示“Stderr: 'iscsiadm: No records found'”
2) 无法从实例卸载Volume 。于是只能直接操作数据库了。

与Volume相关的表

数据库中与Volume直接相关的几个表如下所示

操作Volume时数据库的相关数据变化

新建Volume:

select * from volumes where id = 40\G
*************************** 1. row ***************************
         created_at: 2012-10-29 07:00:23
         updated_at: 2012-10-29 07:00:25
         deleted_at: NULL
            deleted: 0
                 id: 40
             ec2_id: NULL
            user_id: 397dd3be88b6492caa88521502b07617
         project_id: c6159a4f3dd34a2b83527499a40dbd2b
               host: store2.sigsit.org
               size: 20
  availability_zone: nova
        instance_id: NULL
         mountpoint: NULL
        attach_time: NULL
             status: available
      attach_status: detached
       scheduled_at: 2012-10-29 07:00:23
        launched_at: 2012-10-29 07:00:25
      terminated_at: NULL
       display_name: test
display_description: 
  provider_location: 10.61.2.14:3260,5 iqn.2010-10.org.openstack:volume-00000028 1
      provider_auth: NULL
        snapshot_id: NULL
     volume_type_id: NULL

select * from volume_metadata where volume_id = 40\G

select * from iscsi_targets where volume_id = 40\G
*************************** 1. row ***************************
created_at: 2012-09-24 09:00:36
updated_at: 2012-10-29 07:00:24
deleted_at: NULL
   deleted: 0
        id: 205
target_num: 5
      host: store2.sigsit.org
 volume_id: 40

select * from block_device_mapping where volume_id = 40\G

select * from sm_volume where id = 40\G
    

将该Volume挂载到一个实例后:

select * from volumes where id = 40\G
*************************** 1. row ***************************
         created_at: 2012-10-29 07:00:23
         updated_at: 2012-10-29 11:55:36
         deleted_at: NULL
            deleted: 0
                 id: 40
             ec2_id: NULL
            user_id: 397dd3be88b6492caa88521502b07617
         project_id: c6159a4f3dd34a2b83527499a40dbd2b
               host: store2.sigsit.org
               size: 20
  availability_zone: nova
        instance_id: 70
         mountpoint: /dev/vdc
        attach_time: NULL
             status: in-use
      attach_status: attached
       scheduled_at: 2012-10-29 07:00:23
        launched_at: 2012-10-29 07:00:25
      terminated_at: NULL
       display_name: test
display_description: 
  provider_location: 10.61.2.14:3260,5 iqn.2010-10.org.openstack:volume-00000028 1
      provider_auth: NULL
        snapshot_id: NULL
     volume_type_id: NULL

select * from volume_metadata where volume_id = 40\G

select * from iscsi_targets where volume_id = 40\G
*************************** 1. row ***************************
created_at: 2012-09-24 09:00:36
updated_at: 2012-10-29 07:00:24
deleted_at: NULL
   deleted: 0
        id: 205
target_num: 5
      host: store2.sigsit.org
 volume_id: 40

select * from block_device_mapping where volume_id = 40\G
*************************** 1. row ***************************
           created_at: 2012-10-29 11:55:36
           updated_at: NULL
           deleted_at: NULL
              deleted: 0
                   id: 49
          instance_id: 70
          device_name: /dev/vdc
delete_on_termination: 0
         virtual_name: NULL
          snapshot_id: NULL
            volume_id: 40
          volume_size: NULL
            no_device: NULL
      connection_info: {"driver_volume_type&q