How to deal with failed SVMotions/DMotions

The VMware Communities can be a source of awesome information. And today I found a gem from a user called Argyle where he first states his problem, then finds the solution – all by himself – in one thread!

—

Scenario:

If a SVmotion fails for whatever reason the VM still runs OK, but it unmanageable from a VMware perspective (edit setting etc greyed out). The below needs to be completed to remove the DMotion files.

Basically here’s what the below instructions do:

• commits the delta data to the original vmdk files
• resets DMotion state
• keeps the VM online

Description:

Create a snapshot of the VM and then remove the snapshots. This will commit the dmotion deltas too. Note that you can’t remove/commit the dmotion delta files directly since they don’t count as a normal snapshot. Running vmware-cmd with hassnapshot parameter doesn’t return a value of “1″.

After that edit the .vmx settings to remove entries for the DMotionParent parameters or it will restart the migration at next reboot. You will have no options like “Edit settings” etc in Virtual Center unless you do this.

Having data left in the DMotionParent parameters will result in only having one option available in Virtual Center GUI at next reboot called “Complete migration”.

You do not want to use this option though since it completes the migration on the source LUNs, ignoring your previous destination LUNs. You need double the space to do this if you still want to perform it. In my case I had a 100 GB VM disk on a 150 GB LUN and it will fail.

Once you remove the value in the DMotionParent parameters via vmware-cmd in ESX, Virtual Center will display all normal options again. Note that editing the .vmx file directly will not trigger a reload of the .vmx config.

Step by step:

• You have a VM with two disks on LUN1 and LUN2 that you want to migrate to LUN3 and LUN4
• Storage vmotion fails midway for reason X
• We have a running VM with .vmx and swap on LUN3 and vmdk and dmotion delta files on LUN1 and LUN2. VM is running on the delta files.
• Log in to the Service Cosnole of the ESX server that has the VM to create a snapshot on VM (its not available via Virtual Center GUI in this state), make sure there is room on LUN3 that holds vmx files.
• Find the UUID path to your VM with: vmware-cmd -l | grep MYSERVER
• Create a snapshot (of all disks) with:  vmware-cmd /vmfs/volumes/datastore/MYSERVER/MYSERVER.vmx createsnapshot snapname snapdescrition 1 1
• You get files like this on LUN3:

DMotion-scsi0:00_MYSERVER-000001.vmdk
DMotion-scsi0:00_MYSERVER-000001-delta.vmdk
DMotion-scsi0:01_MYSERVER-000001.vmdk
DMotion-scsi0:01_MYSERVER-000001-delta.vmdk

• Remove (Commit) the snapshots: vmware-cmd /vmfs/volumes/datastore/MYSERVER/MYSERVER.vmx removesnapshots
• The above commit all delta files, including dmotion files.
• We have a server running on the original disks with all data intact.
• Virtual Center still thinks its in dmotion state so you can’t edit settings, perform vmotion or anything via Virtual Center. To fix we need to clear the DMotionParent parameters in the .vmx file with the following command from ESX:

vmware-cmd /vmfs/volumes/datastore/MYSERVER/MYSERVER.vmx setconfig scsi0:0.DMotionParent “”

vmware-cmd /vmfs/volumes/datastore/MYSERVER/MYSERVER.vmx setconfig scsi0:1.DMotionParent “”

• If we do not do this the only option after VM shut down will be “Complete Migration” in Virtual Center. If you select this option it will try to rerun storage vmotion again (offline) but it will use same destination as the source disks. Not good if we don’t have space on those LUNs.

Cheers,

Leo