High number of user connection failures of type Other and Connection Timeout and also VDAs unregistering and re-registering randomly at all hours.

[Jason Edelen1709160707]

We have been seeing high numbers of user connection failures in Director since XenApp 7.15 LTSR.  We've upgraded to CVAD 1912 CU6 and are still seeing this.  We've completely rebuilt new machines on Windows Server 2019 on the control plane.  All that is still on Windows Server 2012 R2 are the VDAs and we're in the process of upgrading / rebuilding those.  I've checked the settings on the load balancer, storefront, DDCs, and VDAs and can't find anything.  I do see a lot of errors and warnings in the Storefront Citrix Delivery Services logs. I have had multiple cases open with Citrix on this and we've tried to collect CDF traces for them.  I now have two more cases open now because our connection error count has been randomly going through the roof lately (over 1200 per hour one day this past week).  I have session reliability enabled and I don't know what the users will experience if I turn it off, but Citrix documentation definitely says to leave it on.  Storefront LBVIP is using SOURCEIP with the same timeout set in Storefront for session timeout.  We captured CDF traces on the Storefront servers that are being reviewed for the connection error issue and from the DDCs for the VDA unregistering issue as well in the other case.  Another issue we seem to keep having related to this is random VDAs will start having very poor performance even with low session counts at the time.  On the Storefront servers, there are warnings and errors all over the place but there doesn't seem to be much information on them I can find.  

 

Our users don't report having to launch multiple times as they've become accustomed to just doing that.  When we call to ask, they don't remember by then what happened or even if they noticed anything at the time Director indicated.  Is there anything the community might be able to help with this as I'm waiting on Citrix?  What information can I provide if so?

[Jeff Riechers]

I have seen a lot of the Other and Connection Timeout messages in Director that when digging through correspond to user reconnects.  Basically the system starts brokering the user a new session, but then sees they have an active session and reconnects them to it.  The initial connection it started brokering then gets abandoned and the system logs it as an error.  Even though it isn't.

 

Are users getting any errors or disconnections?  

 

[Jason Edelen1709160707]

Some do but most don't report anything anymore.  My problem is that is just started reporting many more errors than normal in the last week.  Normal for us is 150 to 200 per hour and it's jumped up to almost 2000 per hour recently.  Not seeing network or storage issues or issues with the hosts.  It just seems to ramp up and then level off to about 200 - 400 per hour.  

 

[Jeff Riechers]

Do you have pre-launch enabled on machines?  If so try turning that off.

 

Also check to see if you have ctrl-alt-del disabled on the machines via GPO.  That also can cause a huge amount of those connection messages.

 

[Jason Edelen1709160707]

pre-launch is not enabled.  for the ctr-alt-delete, I don't think it is but where would I go in GPO to see?

 

[Jeff Riechers]

Sorry, not Ctrl-Alt-Del, but DisableLockWorkstation.

 

That would be under HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\System

 

 

[Jason Edelen1709160707]

Sorry for the delay.  No, it is not enabled either.  

 

[Simon Beresford]

Hi Jason, just wondering if you were able to resolve this issue as we have something similar happening.

Thanks for any help/advice.

Simon

[Jason Edelen1709160707]

I still haven't figured this out yet.