Jump to content
Welcome to our new Citrix community!
  • 0

Cannot boot after updating 7.1 CU02 clean to latest patch (ECU2040 & ECU2041)


Jordan Kostov1709152909

Question

Dear everybody,


I have a prod cluster running CU02 clean for an year. Last week I do decided to catch up with all latest patches but updating the master host puts him in emergency mode after reboot (tested a couple of times).

Here is the setup:

- 2 x hosts running X7.1 CU02 and boot local disk from SAN.

- Storage is HPE Nimble which has 4 luns:

-> 1 x 50GB OS lun

-> 3 x 3TB shared for VM data.

 

Note: multipath.conf is properly configured as per HPE Nimble guide 

multipath.conf

 

Update procedure is straight forward. Just run the automatic patch utility where:

- Master host is evacuated form all vms

- master host is put in maintenance mode

- all patches are applied

- host is then rebooted

 

During boot time, host manages to boot from SAN but faces the following errors (picture 1 and 2) after which the OS goes into emergency mode and xen never boots.

After some investigation multipath -ll reports all 3 VM data luns but no local lun (even though the host boots from it).

To keep things simple from storage I do disabled access to 3 VM luns so I can get info on the OS lun only. Then did a reboot.

 

Host manages to boot again but multipath -ll reports nothing. (there is just 1 one LUN that cannot be managed by multipathd so nothing to report).


However multipath -v4 -ll reports another story (check files MM_1-4).

 

it seems the multipath sees the links and sdds but I do not understand why the issue appears.


Moreover there is something wrong with the links in /dev/disk/by-label

 

There are broken links to devices called sdd3, sdd5, sdd6  where the names should be dm-*. (by_label.png)


I've also opened a ticket to Citrix support as this have not happened before and it should be a supported upgrade operation.
 

 

I do found some information about this issue on reddit but no solution was found.

 

 

 

 

 

picture2.jpg

picture1.jpg

mm_1.jpg

mm_2.jpg

mm_3.jpg

mm_4.jpg

by_label.jpg

Edited by Jordan Kostov
added multipath.conf file
Link to comment

3 answers to this question

Recommended Posts

  • 0

After a couple of days troubleshooting, we managed to find the root cause which is - XenServer update has issue with Active Passive multipath config.

 

To mitigate the issue:.

 

1. Before upgrade revert multipath.conf to default that comes with the host. This will make next reboot load active/active pathing.

2. Upgrade the host

3. Change multipath.conf with the active/passive one that is required for HPE Nimble storages.

4. systemctl reload multipathd.service to take the new changes.

 

That's it!

 

  • Like 1
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...