Saturday, April 26, 2014

Solution - VMware vSphere Datastore is not accessible or not detected after unplanned PDL

I would like to share some KB & solutions related to PDL.
Some explanations on storage connectivity problems - PDL (Permanent Device Loss) and APD (All Path Down) is available in VMware Documentation here
Identifying Device Connectivity Problems
Permanent Device Loss (PDL) and All-Paths-Down (APD) in vSphere 5.x (2004684)

Some of my customers experiences PDL, unplanned PDL, where they can no longer access the datastore but storage team has confirmed that they have presented the LUN to the hosts & clusters.
However, the datastore is not detected or not accessible from vSphere.
Below are some of the scenario and its resolution.


1. After unplanned PDL, vSphere can see the Datastores. However, when we try to browse the datastore the Datastore Browser is just loading ... no response.
Solution: Follow this KB VMware KB: Cannot remount a datastore after an unplanned permanent device loss (PDL) or just simply reboot the ESXi host

2. After unplanned PDL, Datastore is gone from vSphere. Storage team confirmed that they have presented the LUN to the hosts & clusters. However, the datastore is not detected as existing datastore and also not detected as new LUN.
Solution: Check storage devices from the host, see if you found any detached devices

If you found any, check the naa id/LUN# if it is the LUN that you are looking for. Select the device then attach and rescan. The datastore you are looking for will normally appear in the list of datastore



3. After unplanned PDL, Datastore is gone but you can see it when you want to add new Datastore (detected as if it is a new Datastore).
You do not want to reformat the datastore but the options to keep or resignature VMFS only are greyed out. The available option is only to format VMFS.

Solution: Follow this KB VMware KB: Cannot remount a datastore after an unplanned permanent device loss (PDL)  or just simply reboot the ESXi host


4. After unplanned PDL,You want to remove inactive datastore because of PDL, but all options are greyed out
Delete it from vCenter Database, the steps are explained in this KB: VMware KB: Unable to remove a datastore from the vCenter Server 4.x / 5.x inventory
or you can also reboot the affected ESXi host

To avoid these issues follow the proper way/best practices for removing a LUN from ESX host
VMware KB: Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x hosts
Best Practice: How to correctly remove a LUN from an ESX host | VMware vSphere Blog - VMware Blogs

2 comments:

vmware_vs said...

Nice, excellent! Thank you

Arild Skullerud said...

Nice article :)

I do however have another scenario that I can't seem to find a solution for.

Some (not all) LUNs vanish after a reboot. I can remount them with the command: esxcfg-volume -M "Name", but they vanish again after the next reboot.