Forum Discussion

Rick_Miller's avatar
Rick_Miller
Icon for Nimbostratus rankNimbostratus
Sep 18, 2019

vdsk migration fails

We had a blade failure which required we migrate all guests to the working blade. We did the RMA sucessfully and the blade came up and booted. I(t took over as the primary. We then wanted to migrate some of the guests back to the (now) unused blade. Method we used before was to drop the guest to provisioned and change the assigned slot from 3 to 1. (When it did it the other way it migrated the vdsk as anticipated). In this case, the migration of the vdsk failed.

 

This is the log

 

Aug 30 13:27:21 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Migration of VDisk (TEST.img) from slot (3) to /shared/vmdisks/TEST.img FAILED: Child exited with non-zero exit code: 2

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Deleting image file - /shared/vmdisks/TEST.img

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Deleting info file - /shared/vmdisks/TEST.info

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Removing.

Aug 30 13:27:21 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Failure - VDisk migration process from slot 3 failed.

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_MIGRATING_VDISK->VS_FAILED

Aug 30 13:27:21 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Failure - VDisk migration process from slot 3 failed.

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_FAILED->VS_ACQUIRING_VDISK

Aug 30 13:27:21 slot1/f5etdc notice vcmpd[5419]: 01510006:5: Guest (TEST): Acquiring a deployable virtual-disk.

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_ACQUIRING_VDISK->VS_WAITING_MIGRATE

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_WAITING_MIGRATE->VS_MIGRATING_VDISK

Aug 30 13:27:21 slot1/f5etdc notice vcmpd[5419]: 01510006:5: Guest (TEST): Migrating VDisk TEST.img from slot 3.

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Adding.

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Syncing with MCP - [filename:TEST.img slot:1 installed_os:0 state:0]

Aug 30 13:27:21 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Syncing with MCP - [filename:TEST.img slot:1 installed_os:0 state:3]

Aug 30 13:36:00 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Migration of VDisk (TEST.img) from slot (3) to /shared/vmdisks/TEST.img FAILED: Child exited with non-zero exit code: 2

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Deleting image file - /shared/vmdisks/TEST.img

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Deleting info file - /shared/vmdisks/TEST.info

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: VDisk (TEST.img/1): Removing.

Aug 30 13:36:00 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Failure - VDisk migration process from slot 3 failed.

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_MIGRATING_VDISK->VS_FAILED

Aug 30 13:36:00 slot1/f5etdc err vcmpd[5419]: 01510004:3: Guest (TEST): Failure - VDisk migration process from slot 3 failed.

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_FAILED->VS_ACQUIRING_VDISK

Aug 30 13:36:00 slot1/f5etdc notice vcmpd[5419]: 01510006:5: Guest (TEST): Acquiring a deployable virtual-disk.

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_ACQUIRING_VDISK->VS_WAITING_MIGRATE

Aug 30 13:36:00 slot1/f5etdc info vcmpd[5419]: 01510007:6: Guest (TEST): VS_WAITING_MIGRATE->VS_MIGRATING_VDISK

Aug 30 13:36:00 slot1/f5etdc notice vcmpd[5419]: 01510006:5: Guest (TEST): Migrating VDisk TEST.img from slot 3.

 

According to F5

 

And as per your symptoms looks like you are hitting the BUG ID 700888. https://cdn.f5.com/product/bugtracker/ID700888.html. Unfortunately there is no workaround for that bug, but it is fixed in version 14.1.0. So our recommendation is to upgrade to that version to fix that BUG.

 

So, my question is can anyone add more information. It worked for 12.1.2 (which is supposed to be one of the versions called out by the bug). It did not work for 12.1.2 HF2.

 

(WE will be migrating the host to 14.1.2 pretty quickly but I still would like to see if there is any further information on the cause/solution)

 

 

Rick

 

1 Reply

  • As I answer my own question. It turned out that MGMT/3 had come out so the management interface for slot 3 was not connected. This caused an interesting set of errors. Once we plugged it in vdsk migration would work. I think the issue is that vdsk migration is done thru the management interface and no diagnostic was kicked when the mgmt interface was down