Jump to content
Welcome to our new Citrix community!
  • 0

XenServer appears disconnected in XenPool - But VMs running


Keith Woodworth

Question

Hi folks, not sure where to post this but will try this in the CLI Forum.

 

Xenserver 7.2, 3 hosts in a pool.

Host 3 in the pool has become unresponsive/disconnected in XenCenter. Red dot with white line through it.

Currently there are 4 VM's running on that host. All hosts are up and working as normal, except one which I

have managed to put into a state where VDI is not available, but is still running and can SSH to that VM.

 

Not in HA

can ssh to disconnected host 3,

toolstack-restart works on that host 3

xapi is running and have restarted it on host 3

4 VM's running on host 3

xe host-list  on master and slaves all show the 3 Xenserver hosts in the list.

Pool.conf on host 3 shows correct: slave:10.11.32.230 for the master.

 

Xencenter shows ram and cpu usage for disconnected host.

I can list all the VM's running via CLI on host 3.

From /var/xapi the state.db file shows last update on Sept 21 @ 15:28.

All NTP times are in sync, DNS works.

 

I have not had a server in a pool do this before and did a command line shutdown of a test VM

so I could restart it on one of the other hosts.

 

I ran this CLI on Host 3:

xe vm-reset-powerstate uuid=a8b5fc05-07bc-c67e-339e-66e4110c7d32 force=true

 

I can now see this VM in Xencenter now but it is shutdown and is not longer in the CLI

list of VMs running on host 3.

 

But now I cannot start that VM as VDI is not available but it is still running as I can ssh

to that VM and do DNS queries against it. (test DNS server), so the powerstate command

did not shutdown the machine properly.

 

Can I issue a shutdown on the VM CLI? CentOS 7.latest, was fully patched last week.

 

So I have Host 3 disconnected in XenCenter, the VMs on the host are running and accessible

but one VM is now VDI not available.

 

How to reattach the VDI to the VM? Tons of articles, all slightly different and so far have not been able

to reconnect VDI to the VM.

 

I see this article: https://support.citrix.com/article/CTX221207

This link lists steps to reconnect a host into a pool, but it doesn't mention anything about any VMs

that might be running on a host.

 

All VM's in shared NFS mounted Netapp, no data on local SR of Host 3.

 

Some promising links in discussions.citrix.com are no longer working. Anyone have some

links to help reconnect host 3 and reattach the VDI to my VM?

 

Thank you,

Keith

 

 

Link to comment

10 answers to this question

Recommended Posts

  • 0

Hi.

So update to this as I don't like leaving these things hanging.

 

I have managed to move all the VM's that were still running on this host to other

xenserver hosts.

 

I followed instructions on this link:

 

https://www.serveroperations.net/citrix/start-a-xenserver-vm-that-throws-vdi-not-available-error-when-you-attempt-to-start

 

and was able to restart each VM on a new Xen host.

 

The Xen host is still not in the pool and the only option is to destroy the host from Xencenter.

 

Thing is, everything else on this Xen host seems ok, xsconsole from CLI works, I can see all network

interfaces, ssh into the Xen host, restart xentoolstack, restart xapi, but it is dropped from pool.

 

I'd like to try this:

 

https://support.citrix.com/article/CTX221207

 

But it is from an older version of Xen and not sure if it will work as the network stack seems to be

fully working.

 

Anything else I can try to get this host to rejoin the pool? One last, I have not actually restarted this

Xen host yet. Is this something that should be ok to do?

 

Thank you.

  • Like 1
Link to comment
  • 0

Make sure all your hosts are properly synchronized to NTP and the offsets re not too great. You can also try a "xe pool-sync-databasse" to see if that helps.

As long as the VMs are running, it should still be OK though you may run into problems if you try to Xenmotion a VM to a different host.

Finally, if you use any external databases, make sure they are also synchronized to the same NTP services.

-=Tobias

Link to comment
  • 0

I did try on a test VM I have running on that host.

 

First I did:

 

xe vm-list resident-on=9b86ecc6-7a97-4c71-b507-9785cd72a1c2 (UUID being Host 3)

 

To get the list of VM UUID's.

 

Then did on Host 3:

xe vm-reset-powerstate uuid=a8b5fc05-07bc-c67e-339e-66e4110c7d32 force=true


Should this have been done from the master?

 

It shutdown the VM and now shows up as a powered off VM, but in reality the VM is still

running but it has disassociated the VDI and get a VDI is not available when trying to start

the VM in Xencenter on another host.

 

Is there a different xe cli to do a forced reset associate the VDI back to the VM?

Thank you.

Keith

Link to comment
  • 0

Hello.

Thank you for answering my questions.

 

Final update.

 

I found a topic thread very similar to what I was seeing:

 

https://discussions.citrix.com/topic/389164-xenserver-host-disabled-after-pool-master-migration/page/2/

 

Main diff, I didn't do anything with pool-master before this happened.

 

One of the people in that thread just restarted their server. That seemed my easiest route rather than changing and

renaming files on the server.

 

So that's what I did, just hit the console and did a reboot. Took a while (forgot how long it can take to restart

one of these) but it eventually came back up and rejoined the pool automatically.

 

Though I'm still without network stats on the console directly, but xsconsole shows all the network and

management info.

 

Thank you.

Keith

 

 

 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...