Terraform-provider-aws: 无法销毁具有以下卷的AWS节点:* aws_volume_attachment.jenkins_disk_attachment:等待卷(vol-XXXX)与实例分离的错误:i-XXXXX

创建于 2017-10-20  ·  3评论  ·  资料来源: hashicorp/terraform-provider-aws

_此问题最初由@evgeniagusakova以hashicorp / terraform#16167开头。 由于提供者的分裂,它被迁移到了这里。 问题的原始内容如下。


嗨,团队!

当我无法使用terraform创建的卷销毁实例时遇到了问题

地形版本

bash-4.3$ uname  -s
Darwin
bash-4.3$ terraform version
Terraform v0.10.6



md5-58db0e4bdc67a27c4b9068659851f4f4



data "aws_ebs_volume" "jenkins_disk" {
  most_recent = true

  filter {
    name   = "volume-id"
    values = ["vol-XXXXX"]
  }
}
resource "aws_volume_attachment" "jenkins_disk_attachment" {
  device_name = "/dev/sdx"
  volume_id   = "${data.aws_ebs_volume.jenkins_disk.id}"
  instance_id = "${aws_instance.jenkins_server.id}"
}

resource "aws_instance" "jenkins_server" {
  connection {
    type        = "ssh"
    user        = "${var.instance_user}"
    host        = "${aws_instance.jenkins_server.public_ip}"
    private_key = "${file("${var.jenkins_server_ssh_key_path}")}"
  }
...
}



md5-46cb7666285c5936e8fec21a7d22aef1



terraform  destroy   -var secret_key=<KEY> -var access_key=<KEY>
aws_security_group.jenkins_server_security_group: Refreshing state... (ID: sg-XXXXX)
<other resources here>
data.aws_ebs_volume.jenkins_disk: Refreshing state...
<other resources here>
aws_instance.jenkins_server: Refreshing state... (ID: i-XXXXX)
aws_volume_attachment.jenkins_disk_attachment: Refreshing state... (ID: vai-XXXXX)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:
<full and correct list of resources to delete here>
Plan: 0 to add, 0 to change, 12 to destroy.
Do you really want to destroy?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.
  Enter a value: yes

<deleting resource>
aws_volume_attachment.jenkins_disk_attachment: Destroying... (ID: vai-XXXXX)
<deleting resources>
aws_volume_attachment.jenkins_disk_attachment: Still destroying... (ID: vai-XXXXX, 10s elapsed)
aws_volume_attachment.jenkins_disk_attachment: Still destroying... (ID: vai-XXXXX, 20s elapsed)
aws_volume_attachment.jenkins_disk_attachment: Still destroying... (ID: vai-XXXXX, 30s elapsed)
aws_volume_attachment.jenkins_disk_attachment: Still destroying... (ID: vai-XXXXX, 40s elapsed)
Error applying plan:
1 error(s) occurred:
* aws_volume_attachment.jenkins_disk_attachment (destroy): 1 error(s) occurred:
* aws_volume_attachment.jenkins_disk_attachment: Error waiting for Volume (vol-XXXXX) to detach from Instance: i-XXXXX

调试输出

预期行为

实例已删除,卷已取消附加但未删除。

实际行为

EIP已解除连接
秒组未删除
实例未删除
音量未断开

解决方法

在节点上:

  1. 使用磁盘安装点停止所有进程( lsof帮助查找所有进程)
  2. 同步磁盘并卸载
  3. 使用Amazon Console取消附加音量
  4. 从状态terraform state rm aws_volume_attachment.jenkins_disk_attachment删除附件
  5. 销毁实例: terraform destroy -var ....

额外细节

我非常确定该磁盘没有取消连接,因为它已安装并正在使用中。 在执行磁盘卸载后,我才可以从Amazon Console拆下它。

但是我希望在一段时间(可配置?)超时后,强制从terraform脱离。

因此,如果这是预期的行为,我想我们需要将此错误报告转换为功能请求:)

参考文献

不确定,但是这个问题看起来与我很接近。
它是旧的terraform版本,所以我决定创建新的发行版本。

bug servicec2

最有用的评论

在销毁附加到实例的aws_ebs_volume时,我遇到了类似的问题。
为了使应用程序能够在实例上使用卷(磁盘空间),必须执行以下步骤:

  • 创建aws卷
  • 将其附加到实例
  • 在该磁盘上创建文件系统
  • 创建一个安装点文件夹
  • 将挂载点添加到fstab
  • 安装它
  • 开始使用它(通常运行一个应用程序)

因此,我尝试手动从实例中分离一个已装入的卷,并在一段时间后失败,并带有注释:
.... /dev/xvdg (busy)
基于此,我得出一个结论,即terraform无法从实例中强制删除已装载的卷。 哪个是正确的。 因此,为了分离该卷,我创建了一个所谓的过程,该过程在分离该卷之前安全地卸载该卷:

  • 停止使用音量(停止应用程序)
  • 卸下它
  • 从fstab中删除安装点(可选,如果是永久的)
  • 删除安装文件夹(可选,如果是永久的)
  • 与实例分离
  • 移除aws音量

代码中的内容如下所示:

# Commands in variable to create 
# a filesystem and add to fstab
variable "dataCreate" {
  default = [ "echo 'BEGIN: mkfs.ext4 /dev/xvdg'",
              "sudo mkfs.ext4 /dev/xvdg > /dev/null 2>&1 && echo 'END: mkfs.ext4 /dev/xvdg done'",
              "echo 'BEGIN: create /data folder'",
              "sudo mkdir -p /data > /dev/null 2>&1 && echo 'END: create /data folder done'",
              "echo 'BEGIN: Add /dev/xvdg to fstab /data'",
              "sudo sh -c \"echo '/dev/xvdg /data ext4 defaults 1 2' >> /etc/fstab\" && echo 'END: Added /dev/xvdg to fstab /data'",
              "echo 'BEGIN: Mount /dev/xvdg as /data'",
              "sudo mount /bitcoin && echo 'END: Mount /data done'" ]
}
## Variable holding commands to stop application, 
## unmount and remove from fstab
variable "dataUmount" {
  default = [ "echo 'BEGIN: Stop application using /data''",
              "sudo systemctl stop {{APP}} && echo 'END: Application using /data stopped'",
              "echo 'BEGIN: unmount /data'",
              "sudo umount /data && echo 'END: unmount /data done'",
              "echo 'BEGIN: Remove /data from fstab''",
              "sudo sed -i '/data/d' /etc/fstab && echo 'END: Remove /data from fstab'",
              "echo 'BEGIN: Remove /data folder",
              "sudo rm -rfv /data && echo 'END: Added /dev/xvdg to fstab /data'"
            ]
}
## Create a data volume
resource "aws_ebs_volume" "data" {
  availability_zone = "eu-central-1a"
  size = "10"
}
## Attach to instance
resource "aws_volume_attachment" "data_attach" {
  device_name = "/dev/xvdg"
  volume_id   = "${aws_ebs_volume.data.id}"
  instance_id = "${aws_instance.instance.id}"
}

## Configure volume
resource "null_resource" "data_config" {
  depends_on = ["aws_volume_attachment.data_attach", "aws_instance.instance"]

 ## commands to run when creating
  provisioner "remote-exec" {
    connection {
      type = "ssh"
      agent = false
      timeout = "60s"
      host = "${aws_instance.instance.public_ip}"
      user = "ubuntu"
      private_key = "${file(var.privateKey)}"
    }
    inline = ["${var.dataCreate}"]
    }

  ## run unmount commands when destroying the data volume
  provisioner "remote-exec" {
    when = "destroy"
    on_failure = "continue"
    connection {
      type = "ssh"
      agent = false
      timeout = "60s"
      host = "${aws_instance.instance.public_ip}"
      user = "ubuntu"
      private_key = "${file(var.privateKey)}"
    }
    inline = ["${var.dataUmount}"]
  }
}

## Create instance
resource "aws_instance" "instance" {
  depends_on = ["aws_ebs_volume.data"]
  ami = .....
......
}

因此,想法是在分离卷之前释放该卷。
希望它能帮助某人。

所有3条评论

@evgeniagusakova您可能想要尝试将skip_destroy为true。 https://www.terraform.io/docs/providers/aws/r/volume_attachment.html#skip_destroy

1017值得一看。

我面临着同样的问题,它确实有助于设置skip_destroy

在销毁附加到实例的aws_ebs_volume时,我遇到了类似的问题。
为了使应用程序能够在实例上使用卷(磁盘空间),必须执行以下步骤:

  • 创建aws卷
  • 将其附加到实例
  • 在该磁盘上创建文件系统
  • 创建一个安装点文件夹
  • 将挂载点添加到fstab
  • 安装它
  • 开始使用它(通常运行一个应用程序)

因此,我尝试手动从实例中分离一个已装入的卷,并在一段时间后失败,并带有注释:
.... /dev/xvdg (busy)
基于此,我得出一个结论,即terraform无法从实例中强制删除已装载的卷。 哪个是正确的。 因此,为了分离该卷,我创建了一个所谓的过程,该过程在分离该卷之前安全地卸载该卷:

  • 停止使用音量(停止应用程序)
  • 卸下它
  • 从fstab中删除安装点(可选,如果是永久的)
  • 删除安装文件夹(可选,如果是永久的)
  • 与实例分离
  • 移除aws音量

代码中的内容如下所示:

# Commands in variable to create 
# a filesystem and add to fstab
variable "dataCreate" {
  default = [ "echo 'BEGIN: mkfs.ext4 /dev/xvdg'",
              "sudo mkfs.ext4 /dev/xvdg > /dev/null 2>&1 && echo 'END: mkfs.ext4 /dev/xvdg done'",
              "echo 'BEGIN: create /data folder'",
              "sudo mkdir -p /data > /dev/null 2>&1 && echo 'END: create /data folder done'",
              "echo 'BEGIN: Add /dev/xvdg to fstab /data'",
              "sudo sh -c \"echo '/dev/xvdg /data ext4 defaults 1 2' >> /etc/fstab\" && echo 'END: Added /dev/xvdg to fstab /data'",
              "echo 'BEGIN: Mount /dev/xvdg as /data'",
              "sudo mount /bitcoin && echo 'END: Mount /data done'" ]
}
## Variable holding commands to stop application, 
## unmount and remove from fstab
variable "dataUmount" {
  default = [ "echo 'BEGIN: Stop application using /data''",
              "sudo systemctl stop {{APP}} && echo 'END: Application using /data stopped'",
              "echo 'BEGIN: unmount /data'",
              "sudo umount /data && echo 'END: unmount /data done'",
              "echo 'BEGIN: Remove /data from fstab''",
              "sudo sed -i '/data/d' /etc/fstab && echo 'END: Remove /data from fstab'",
              "echo 'BEGIN: Remove /data folder",
              "sudo rm -rfv /data && echo 'END: Added /dev/xvdg to fstab /data'"
            ]
}
## Create a data volume
resource "aws_ebs_volume" "data" {
  availability_zone = "eu-central-1a"
  size = "10"
}
## Attach to instance
resource "aws_volume_attachment" "data_attach" {
  device_name = "/dev/xvdg"
  volume_id   = "${aws_ebs_volume.data.id}"
  instance_id = "${aws_instance.instance.id}"
}

## Configure volume
resource "null_resource" "data_config" {
  depends_on = ["aws_volume_attachment.data_attach", "aws_instance.instance"]

 ## commands to run when creating
  provisioner "remote-exec" {
    connection {
      type = "ssh"
      agent = false
      timeout = "60s"
      host = "${aws_instance.instance.public_ip}"
      user = "ubuntu"
      private_key = "${file(var.privateKey)}"
    }
    inline = ["${var.dataCreate}"]
    }

  ## run unmount commands when destroying the data volume
  provisioner "remote-exec" {
    when = "destroy"
    on_failure = "continue"
    connection {
      type = "ssh"
      agent = false
      timeout = "60s"
      host = "${aws_instance.instance.public_ip}"
      user = "ubuntu"
      private_key = "${file(var.privateKey)}"
    }
    inline = ["${var.dataUmount}"]
  }
}

## Create instance
resource "aws_instance" "instance" {
  depends_on = ["aws_ebs_volume.data"]
  ami = .....
......
}

因此,想法是在分离卷之前释放该卷。
希望它能帮助某人。

此页面是否有帮助?
0 / 5 - 0 等级