VHD files not getting deleted after removing VM and disk using XenCenter

[Kohl Adams]

Hello,

I'm currently using XenCenter and XenServer version 6.0. Using XenCenter gui, I went ahead and removed a VM and the following 2 disks attached to it:

uuid ( RO) : d7d556d8-99d1-4d68-8b9b-d1cf4ef095f0
name-label ( RW): server01 opt disk
name-description ( RW): server01 opt disk

uuid ( RO) : 2e8b6e37-a728-49f4-a087-dda943d98836
name-label ( RW): server01 root disk
name-description ( RW): server01 root disk

The 2e8b6e37-a728-49f4-a087-dda943d98836.vhd* 8GB file was removed from the SR location without any issues but the *d7d556d8-99d1-4d68-8b9b-d1cf4ef095f0.vhd 50GB is still sitting inside the SR location.

I ran xe vdi-list and both disks don't show anymore. Why is this happening and can I just manually remove the d7d556d8-99d1-4d68-8b9b-d1cf4ef095f0.vhd fle ?

Thanks in advanced.

[Gert Jensen]

Hello,

Please check /var/log/SMlog for errors...
is the xenserver patched up to date ?

do a "tail -f /var/log/SMlog" then to a rescan of the sr, that trigger a coalece..

Kind regards
Gert

[DEAN CARRERA]

I have the same thing going on with xen server 5.6.2. I run snap shots using phdvirtual for backups. I'm wondering if this is causing the issue. The properties on the SR are showing 704 of 709gig used. This is incorrect there is only 400 gig of disk used. I tried using the coalesce utility and this is not correcting the issue. I'm not seeing any snapshots either. Thoughts.

[horizon limit]

the storage issue and the lack of space when the PHD is in action is a legacy now. I am not sure when there will be a fix for such problems

the only workaround to deal with this is to move/copy the vm to another storage which will coalesce the disks.
Coalesce what i have seen fails in the cases where the vdis are corrupt or the chains consisting of more than 32 vdis.

[DEAN CARRERA]

This is scary. How can one tell if there is corruption? Also are the vms in danger because of this?

[horizon limit]

well SM logs can tell if there is corruption or not. you can grep the keyword corrupt incase there are nay, that will tell you if the disks re corrupt or not.

[DEAN CARRERA]

Well it kind of fixed itself when I move a vm off the SR to another location. Now it shows the correct usage but the allocation does not seem right unless I'm not understand something here:

350.7 GB used of 709 GB total (2450.7 GB allocated)

[horizon limit]

can you check the sr type xe sr-list params=all uuid= sr-uuid ? i am sure it is lvm and the lvms are more than it should be . seems like the space is still not released by the snapshots.

[DEAN CARRERA]

Here is the output from the command:

: 6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e
name-label ( RW): iSCSI virtual SAS 700Gig
name-description ( RW): iSCSI SR [172.31.2.161 (iqn.1992-04.com.emc:cx.sl7e2100300037.b1)]
host ( RO): <shared>
allowed-operations (SRO): forget; VDI.create; VDI.snapshot; plug; update; destroy; VDI.destroy; scan; VDI.clone; VDI.resize; unplug
current-operations (SRO):
VDIs (SRO): ba5341f8-f801-4d7a-a19f-9eda2a865468
PBDs (SRO): 2121e7be-9d3a-f828-7216-176fd0b9db4d; 40fb71d6-6b5c-865a-0285-e43b4dade7c3; 8aefef09-a48b-bd2c-ae86-7ff2d55ada71; 3934bcfd-821e-4936-5529-45b51b49e5d8
virtual-allocation ( RO): 2631409860608
physical-utilisation ( RO): 376556224512
physical-size ( RO): 761270370304
type ( RO): lvmoiscsi
content-type ( RO):
shared ( RW): true
other-config (MRW): auto-scan: false
sm-config (MRO): allocation: thick; use_vhd: true; multipathable: true; devserial: scsi-3600601604f3028007c5f93d3cc3ee211
blobs ( RO):
local-cache-enabled ( RO): false
tags (SRW):

[horizon limit]

cool.. this is lvmoiscsi as i was suspecting so it is thick provisioned. Now can you please run the this script as
perl FindOrpahnVDI.pl <sr-uuid>. this will tell you what are the orphan vdi and can be deleted.
Make sure it will list the HA disks as orphan as well. You can confirm by running xe vbd-list vdi-uuid=<> and if it returns to prompt, you can delete the vdi.

FindOrphanVDI.pl

[DEAN CARRERA]

Thanks for the script, will this just show me the vhd's that are no longer needed or will it also delete them? How do I know that the script is accurately identifying them. I want to make sure it will not delete a live needed vhd.

Thanks

[horizon limit]

it will list all the vdis which do not have a vbd connection with them. A vbd is block device and a connector to the vm and the vdi, it wille xists even if the vm is shutdown.

it will only list the vdi-uuid and would not delete them, you would need to delete them using xe vdi-destroy uuid=<>
now the big question how can we judge if it is listing the correct vdi, simply run xe vbd-list vdi-uuid=<> and xe vdi-list uuid=<vdi-uuid>( this will give info about the vdi in xapi DB, but the main is the former command), if returns to the prompt in xe vbd-list then simply don;t think much, go ahead and delete it.

hope this helps.

[DEAN CARRERA]

Hello Sir,

I'm receiving the following error when running the script:

Use of uninitialized value in concatenation (.) or string at FindOrphanVDI.pl line 10.

thank you!

[horizon limit]

yes because the sr-uuid has not been provided. Please go through the post above, it clealry says

perl FindOrpahnVDI.pl <sr-uuid>.

[DEAN CARRERA]

Ok. I ran the script and it came back with nothing. Strange.

[horizon limit]

That means there are no orphan disk, i tested the same script in my lab server and it gave the expected results. Another conclusion is that there re no orphan disks attached. Please attach the output of the vhd-util scan -f -c -m "VHD-*" -l VG_XenStroage-<sr-uuid> -p and lvscan | grep <sr-uuid>. here. let me have a look.

[DEAN CARRERA]

Here is the result of : vhd-util scan -f -c -m "VHD-*" -l VG_XenStroage-6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e -p

scan failed: -22

and here are the results of the 2nd command:

lvscan | grep 6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e
/dev/sdq: read failed after 0 of 4096 at 0: Input/output error
/dev/sdag: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdt: read failed after 0 of 4096 at 0: Input/output error
/dev/sde: read failed after 0 of 4096 at 0: Input/output error
/dev/sdu: read failed after 0 of 4096 at 0: Input/output error
/dev/sdf: read failed after 0 of 4096 at 0: Input/output error
/dev/sdv: read failed after 0 of 4096 at 0: Input/output error
/dev/sdg: read failed after 0 of 4096 at 0: Input/output error
/dev/sdw: read failed after 0 of 4096 at 0: Input/output error
/dev/sdh: read failed after 0 of 4096 at 0: Input/output error
/dev/sdx: read failed after 0 of 4096 at 0: Input/output error
/dev/sdj: read failed after 0 of 4096 at 0: Input/output error
/dev/sdz: read failed after 0 of 4096 at 0: Input/output error
/dev/sdk: read failed after 0 of 4096 at 0: Input/output error
/dev/sdaa: read failed after 0 of 4096 at 0: Input/output error
inactive '/dev/VG_XenStorage-6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e/MGT' [4.00 MB] inherit
inactive '/dev/VG_XenStorage-6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e/VHD-ba5341f8-f801-4d7a-a19f-9eda2a865468' [350.69 GB] inherit

[horizon limit]

the vhd-util command is wrongly executed . please follow the article http://support.citrix.com/article/CTX123461 and provide the output and the there is one lv as per the output.

[DEAN CARRERA]

I'm not sure what is wrong with this command, I have compared the link and it looks the same. Can you be specific, Here is the command I used:

vhd-util scan -f -c -m "VHD-*" -l VG_XenStroage-6b93e0ef-fa87-6048-e6ed-a6bc7a7e093e -p

Thanks

[Shubam Sharma]

Hi,
I have a question, I am using citrix xencenter 6.5.0 and in one compute i accidently deleted .vhd file of one VM and after reboot VM didn't started again by giving error vdi not available. I have checked VDI and VBD, VBD is connected with the VM and VDI is not connected with the VM, I want to know how can i recover vhd files back so that and how can i connect same vdi with the VM so that it will boot and start. Please help me with this. I have an important DB with this VM. How do i recover deleted vhd file?