Disconnected sessions not logging off

[Matt Sherman]

I've been struggling getting disconnected sessions to log off by themselves. Citrix support can't seem to find a solution either. I was able to recreate the issue.

It appears that if a user logs off the workspace portal without closing the applications first, the sessions hang. In a company of about 700 users, this happens A LOT. 

 

We are running VA/VD 7 1912 from the cloud

 

The situation is exactly like the one in this previous post.......... https://discussions.citrix.com/topic/397967-xenapp-sessions-not-terminating/

We found 2 process that were causing the issue and added them to the registry on all VDAs 

 

After that was done, the applications themselves (Outlook, Chrome, Excel, etc) started causing the sessions to hang. After ending the application, the sessions do successfully log off.

 

Citrix recommended the following changes to local policy, Computer Config/Windows components/remote desktop services/remote desktop session host/session time limits

 

These changes were unsuccessful.

 

If anyone has any suggestions other than the one listed that have been successful, PLEASE let me know. 

 

Manually killing up to 80 sessions every morning is started to take a toll on me :)  

 

[Jozef Jonas1709152693]

The same issue here after updating from CU3 to CU4. Some users missing user name in Citrix Studio. When I check the particular server, user is there and seems to be active, but when I try to logoff the user directly from the affected server via Task Manager or by logoff command, it does nothing. Also no log in Citrix Profile Mgmt about logoff attempt. Only thing that helps is killing LogonUI.exe via Task Manager.

 

This issue seriously affecting also regular reboot of the VMs as they time out during reboot process as the reboot takes due to this logoff issue very long time and then they keep turned off.

[Arthur Chan1709152361]

Here you go, everyone: https://support.citrix.com/article/CTX338807

[tony roberts1709152250]

I was excited to see the fix above but unfortunately it hasn't resolved our issue.  We have had this issue since going to CU4 and its driving us mad. We  get users complaining that their session starts to freeze up and then becomes unresponsive. There session cannot then be disconnected once this happens and there are the usual processes loginui.exe crss.exe and any sometimes apps they were using Outlook.exe Excel.exe etc. I'm completely hacked of with Citrix its just not good enough and they are not helping. Were not the only ones with this issue and its annoying because its only happened since we have upgraded.  Can anyone 100 % confirm  rolling back to CU3 fixes this ?

 

Thanks in advance.

[Jens Strohschnitter1709152492]

For us replacing StatUI.dll which were delivered from Citrix for specific editions (2109, 2112 and 19.12 CU4) fixed the issue of stucking logons and hanging disconnected sessions.

Before we've got the fix from Citrix, rollback of the VDAs to CU3 fixed the issue. Our farm was still on CU4!

 

The only issue that still exists, are the randomly disconnections. After disable "HDX Adaptive Transport" the disconnections are less than with enabled "HDX Adaptive Transport", but still there.

 

We've opened for that a Citrix Case #80987505 - they are still investigating the logs.

[Jens Strohschnitter1709152492]

We also have trouble with disconnets and not logged off sessions since upgrading from 1912 CU3 to CU4 and upgrading WEM from 2106 to 2109.

After removing VUEMUIAgent.exe from 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Citrix\wfshell\TWI
LogoffCheckSysModules

 

The disconnects and logoff issues seem to be fixed.

[Mats Wallin1709151976]

We have this after uppgrading to CU4 ..... any solution ?

 

Matt ? do you still have this issue ?

[LK-CitrixAdmin LK-CitrixAdmin]

We have exactly the same problem after updating to CU4. Many disconnected sessions can no longer be logged off.

[Matt Sherman]

Update- Last night Citrix support had me add the executable for the applications to that same registry string (outlook.exe, winword,exe,eplorer.exe, etc) and that failed miserably. Doing that generated so many connection issues this morning, I had to remove that from from all the VDAs 

[Jan Herman1709161051]

We have the same problem, is Citrix looking into this issue?

[Jens Strohschnitter1709152492]

I take back my last posting. The disconnects still exists. The only issue that is fixed by removing VUEMUIAgent.exe from 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Citrix\wfshell\TWI
LogoffCheckSysModules

 

is the stucking on logoffs. We still have same issues as described in this article:

 

https://discussions.citrix.com/topic/408019-citrix-virtual-apps-and-desktops-1912-random-disconnects/page/3/

[TAGHLEEF INDUSTRIES SpA]

Same problem here after updating from 1912 CU3 to CU4 on Windows Server 2019 (1809)

Reverting VMs with VDA service onboard to snapshot taken previously the update, problem disappears so I'm pretty sure the issue depends on something triggered by CU4. I suspect there is something wrong interfacing to RDP services. Still investigating.

 

This is just to recap and to try to pitch in somehow:

• What I can see are disconnected user sessions that get disconnected just a few seconds later the logon. Users that face the problem, indeed, since they don't see any window app (we use Virtual Apps)  appearing on screen, try again untill the requested app regularly starts on a different VDA server.
• These session do not respect the session timeout policies and stay on until manually killed ot server restarts.
• Several users report the error: "The task you are trying to do can't be completed because remote desktop services is currently busy. Please try again in a few minutes. Other users should still be able to log on."
• The orphaned sessions run exclusively system processes such as winlogon.exe, csrss.exe, LogonUI.exe, dwm.exe, fontdrvhost.exe,ctxgfx.exe that basically are processes running under the same user's istance ID but executed by other service accounts (SYSTEM,DWM-sessionID, UMFD-sessionID). To force session to close, I've to kill the associated LogonUI.exe.
• In Events (Application and Services Logs / RemoteDesktopServices-xxx) , I find the following errors related to RDP's RemoteFX module. This errors no longer appears after reverting back to CU3. I'd even exclude that it can depend on the MS updates level. We're running CU3 and CU4 with the same Windows Server 2019 Cumulative Update (2021-12 Cumulative Update for Windows Server 2019 fo x64-based Systems - KB5008218).

  Event ID		Task Category		Description
  101			RemoteFX module		The network characteristics detection function has been disabled because of Reason Code: 2(Server Configuration)..
  142			RemoteFX module		TCP socket READ operation failed, error 64
  143			RemoteFX module		TCP socket WRITE operation failed, error 64
  226			RemoteFX module		RDP_TCP: An error was encountered when transitioning from StateUnknown in response to Event_Disconnect (error code 0x80070040).
  227			RemoteFX module		'Failed GetConnectionProperty' in CUMRDPConnection::QueryProperty at 2884 err=[0x80004001]
  228			RemoteFX module		Disconnect trace:CUMRDPConnection Disconnect trace:'calling spGfxPlugin->PreDisconnect()' in CUMRDPConnection::PreDisconnect at 4595 err=[0xc], Error code:0xC

   

• Analizing these sessions with PS (Get-BrokerSession cmdlet), I noticed that all of them have the LogoffInProgress attribute set as True:

  IdleSince			09/01/2022 08:42:17
  ImageOutOfDate			False
  InMaintenanceMode		False
  IsAnonymousUser			False
  IsPhysical			True
  LaunchedViaHostName		ctxdc01.mydomain.com
  LaunchedViaIP			10.1.1.209
  LaunchedViaPublishedName	My Computer
  LogoffInProgress		True	<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  LogonInProgress			False
  MachineName			MYDOMAIN\CTXVDA03
  MachineSummaryState		InUse
  MachineUid			4
  MetadataMap			System.Collections.Generic.Dictionary`2[System.String,System.String]
  OSType				Windows 2019
  PersistUserChanges		OnLocal
  PowerState			Unmanaged
  PreferredZoneName	
  PreferredZoneUid	
  Protocol			HDX
  ProvisioningType		Manual
  ReceiverIPAddress		10.21.3.4
  ReceiverName			MYCLIENT1
  SecureIcaActive			False
  SessionId			-1
  SessionKey			8b0b191a-2c85-42d4-b929-bec3cc2ca4e8
  SessionReconnection		Always
  SessionState			Active
  SessionStateChangeTime		05/01/2022 08:42:11
  SessionSupport			MultiSession
  SessionType			Application
  SmartAccessTags			System.String[]
  StartTime			05/01/2022 08:41:27
  TenantId	
  Uid				71454
  UntrustedUserName		MYDOMAIN\jdoe
  UserFullName	
  UserName	
  UserSID	
  UserUPN	
  ZoneName			Primary
  ZoneUid				559932d5-c8f8-4657-b360-79d0de0ce600

   

[Arthur Chan1709152361]

I just upgraded a customer from 1903 to 1912 CU4 and started seeing this issue. Initially it was reported as inability to logoff a user session. Fails to logoff from Director, Studio, Task Manager or logoff.exe.

 

I found this article, followed the process of identifying the LogonUI.exe PID and killing with Process Explorer. Once I did that the session logged off.  I also have a number of disconnected sessions with no username (presumably unable to logoff) across multiple VDAs.

 

https://community.spiceworks.com/topic/2058276-rds-2016-4-disconnected-cant-log-it-off?page=1#entry-7832112

 

I will open a Citrix support call tomorrow on this. By any chance care to share any existing case# you may have on this so they can cross-reference?

[Roman Sobota]

we have same problem with version VDA 2112 too

Fails to logoff from Director, Studio, Task Manager or logoff.exe.

only kill logonui.exe help with end disconnect session

[Gunther Oellibrandt]

Also seeing it with VDA 2109 version. Rolled back to VDA 2106 and all seems fine again. We are putting the update to VDA 2112 on hold until this gets fixed by Citrix.

[Brian Black]

We are experiencing this issue as well and unfortunately cannot pin it on a specific VDA upgrade. We are currenly running 2112, though, and we've tried adding the LogonUI.exe and winlogon.exe entries to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\Sysprocs registry key.

 

I assume adding those entries to the (analogous?) Citrix LogOffCheckSysModules entry would be similarly fruitless, but I'll likely give it a shot soon nonetheless.

 

EDIT: Also, we have yet to find any rhyme or reason with who is affected by the issue, though, interestingly, there are plenty of users who have never experienced the problem.

Edited January 12, 2022 by Brian Black

[Arthur Chan1709152361]

Thanks for the feedback. I did open a Citrix case and they pointed me to the standard LogOffCheckSysModules value https://support.citrix.com/article/CTX891671

 

I have switched our environment over to 1912CU3 to see if the behaviour persists. I am starting to wonder if there is some interaction with recent Windows updates. Our systems are Server 2019 currently patched to November 2021 level.

 

Also to note - in my environment it is 100% published desktops and no published apps.

Edited January 12, 2022 by Arthur Chan

[Gunther Oellibrandt]

After rolling back to VDA2106 we have not seen the issue.

 

I did install out of band KB5010196 (https://support.microsoft.com/en-us/topic/january-4-2022-kb5010196-os-build-17763-2369-out-of-band-1a7a9a37-b154-4e73-92dc-1a2f65a4c0d1) This fixes some issues with RDP.

 

I do remember that some servers running VDA2109 and did not have this KB had the described issue of the server no longer responding (and RDP not working anymore). Could be related, could be coincidence.

[Arthur Chan1709152361]

Going back from 1912CU4 to 1912CU3 has resolved our issue. Ended up archiving the Citrix support case as I was no longer in a position to troubleshoot the issue. 

[Reinhard Teischl1709152679]

Same issue here. Citrix, are you listening? Why opening up a support case when the reply is go for "LogOffCheckSysModules"? ?

[Steve Istas]

I'm seeing the exact same issue. Server 2019 with VDA 1912 CU4. Didn't have any issues with CU2. I could log off disconnected sessions without issue. I called Citrix Support to find they don't have a solution and weren't familiar with the problems.

 

I was on CU2 but going to go ahead and move backwards to CU3.

 

Citrix doesn't have a fix nor do they seem interested in figuring out what the VDA is doing.

[Cary Barron]

Do any of you utilize Imprivata by chance? What OS are you guys seeing this on? If you take one session stuck in Disconnected state, kill one process at a time, without stopping logonUI or any others which ones are left? If just native OS, that you may want to look in Process explorer to get more info if possible. 

[Arthur Chan1709152361]

Thanks for popping in. No, my affected client is straightforward CVAD/S to on-prem VDA with just AD authentication. VDA is Server 2019 patched to Nov 2021. The only interesting system-level software is Crowdstrike for A/V. I didn't get a chance to try, but if left long enough the session goes into a state where there is no username attached (it appears in Studio as a disco session with no username). The session doesn't exist in quser but does in qwinsta. If we try and shut down the OS it doesn't succeed. Doing ctrl-alt-del at VM console results in no logon prompt so the OS is pretty borked at that point and we have to power off.

[Brian Black]

We are also running a pretty vanilla Server 2019 VDAs hosting just virtual apps and patched to December 2021 level. I have not yet tried adding the registry keys to LogOffCheckSysModules.

 

We also see the servers fairly often refuse to finish shutting down, hung at Windows logon screen, seemingly unresponsive to ctrl-alt-del. I would like to try to correlate the act of manually killing all "ghost" disconnected sessions at the end of the day (by killing LogonUI.exe) to see if that allows the server to always shutdown gracefully.

 

I would also like to try disabling our A/V to see if that has any bearing.

[Brian Black]

We received the replacement StatUI.dll and have now gone 2+ days with no ghost sessions or stuck logon issues.

This is on Server 2019 VDA 2112 environment.

[Alexander Hentschel]

We had the same issue and was also replacing the StatUI.dll but totally different results:

 

VDA2109 Windows Server 2019 as OS, it fixed the issue, everything was fine

VDA2109 Windows Server 2016 as OS, login not possible anymore. Got only Client connection error in Director and the session was not build. EventViewer show, that the application login.exe crashed by every try to login.
After reverting the the MC and usinf the "original" StatUI.dll the login is fine again.
So, be careful by applying that patch. On my test member server everything was okay, after MC update over the night for 60 member server, i getting a lot problems......