XenApp slow logon times, user get black screen for 20 seconds.

[Andy White1709154166]

Hello,

 

We use XenApp 7.6 with Windows 2012R2 on VMware 5.5.

 

Logon times have become slow recently for users, they get a black screen just before their XenApp desktop shows up for about 20 seconds.  How do we find out the cause of this?  Most profiles are under 100mb.

 

If I turn all the GPOs off apart from the UPM GPO it's still slow, if I turn it off logons are fast.

 

The first user logon on the VDAs everyday is always slow (we reboot every night) even with the Citrix Profile Manger service off and all GPOs.

 

Thanks

[Andy White1709154166]

So do I add this key or edit?

 

How long does the UberAgent trial last for and does it just sit on a server with It's?

[Andy White1709154166]

Eric,

 

Where did you exclude HKCU\Software\Microsoft\Windows\CurrentVersion\UFH\SHC?

 

Was it a compute or user Citrix policy?

 

I've gone to Computer > Policies > Admin Templates > Citrix > Profile Management > File system > and the the is files or folders to exclude.

 

Thanks

[Trond Eirik Haringvarstein]

Hi Andy,

 

You need to add the Serialize key, it doesn't exist.

 

UFH/SHC is a folder and therefore goes into Excluded Folders - Computer > Policies > Admin Templates > Citrix > Profile Management > File system

 

uberAgent - For production use a license file is required. Without a license file uberAgent is fully functional but displays a splash screen during logon.

 

You should also look into ControlUp and their Script Based Actions (SBA). Best tools out there for troubleshooting.

[Kevin Scott1709153591]

I find it frustrating and unacceptable that the first login of the day takes almost a minute (30 seconds for policies and 30 seconds of a black screen), the fact that it only occurs via ICA make me think Citrix has to some up with some sort of resolution. I still have my case open with them and I'll continue to see what they recommend.....

[Andy White1709154166]

Citrix are looking again Monday and gathering more logs/traces. Will keep all posted too...

[Andy White1709154166]

@Eric

 

What version of Splunk do you use?

[Trond Eirik Haringvarstein]

Latest Splunk Enterprise Indexer - https://www.splunk.com/en_us/products/splunk-enterprise.html

 

uberAgent is installed on all endpoints/Master Images that you want to monitor.

[Liza Nolan]

What type of user profiles do you use? Roaming by any chance?

[Scott Busse1709153852]

Any update on the issue?   I'm experiencing the same issue.

[Per-Ole Fanuelsen]

I´m having this problem too on Windows 10 and Xendesktop 7.11

[Andy White1709154166]

I wish I had an update, but don't, Citrix are not sure.

 

If I delete the users profile form the UPM server and create a new profile it is faster.  Do I want to do this for 100s of users, not really.

[Mark DePalma1709157107]

Stumbled across this thread because I was searching for a relation between the 'Software\Microsoft\Windows\CurrentVersion\UFH\SHC' registry key and logon slowness. I noticed the same activity in a trace. I have added SHC as a UPM registry exclusion and added the 'StartupDelayInMSec'=0 registry value. So far it looks good, but I'd like to do more testing on multiple servers from a fresh reboot.

[Mark DePalma1709157107]

Stumbled across this thread because I was searching for a relation between the 'Software\Microsoft\Windows\CurrentVersion\UFH\SHC' registry key and logon slowness. I noticed the same activity in a trace. I have added SHC as a UPM registry exclusion and added the 'StartupDelayInMSec'=0 registry value. So far it looks good, but I'd like to do more testing on multiple servers from a fresh reboot.

 

This didn't seem to help much. My issue seems to be specifically with the first published desktop logon on a Win2k2012R2 server. Published apps don't have this issue. There is a delay between UPM completing the profile load and explorer showing the desktop/start menu. A logon from any other user after this on the same server has a lower load time. This delay always shows in the "interactive" duration time in Director.

[Mark DePalma1709157107]

I have been looking at this more over the past two days. It looks like the black screen delay is aligning with a 'AppResolver' task that runs. This can be viewed in the 'Microsoft-Windows-Shell-Core/Operational' event log. Still haven't found out how to manipulate this.

 

UPDATE:

I have found that this task is running under Explorer and is reading/writing to %LOCALAPPDATA%\Microsoft\Windows\Caches. I was already excluding this directory from UPM. I tried adding it back, but this didn't seem to change the logon time. Still looking into this.

[Mark DePalma1709157107]

UPDATE #2:

 

I spent a few more hours tracing this 'AppResolver Scan' process out and troubleshooting this issue. I found that the process is indeed directly tied to '%LOCALAPPDATA%\Microsoft\Windows\Caches'. Whether I include or exclude this directory in UPM I still have the same logon 'black screen' delay. The delay ALWAYS aligns with the 'AppResolver Scan' tasks in the 'Microsoft-Windows-Shell-Core/Operational' event log. My next step was to find out what is initiating this time consuming scan (which takes anywhere from 6-60 seconds for me) and how to stop it. I couldn't find any documentation on this process, so it was very difficult to troubleshoot. I kept looking through different things and eventually found something very promising in the UPM log...

 

2017-01-12;16:55:28.705;INFORMATION;SOP;citrixtestuser;2;460;Forcing Win8 Tile refresh by overwriting ResetCache value. Old value: 0

 

My thought here was that UPM might have been forcing the refresh by changing this value to something other than 0. To test this I created a little script that set that value to 0 in a loop for my test account on my test server and then I logged on. My hope was to set the value back to zero AFTER UPM changed it and BEFORE Explorer read it. The logon was fast with almost ZERO delay between ActiveSetup and Explorer presenting the start menu and desktop. I tested it a few more times with different accounts after fresh reboots and achieved the same result. I did notice that an 'AppResolver Scan' did still run, but AFTER the desktop and start menu had already loaded which I'm totally fine with. Everything else looked fine, pinning was still working, and shortcuts worked properly. I also tested excluding '%LOCALAPPDATA%\Microsoft\Windows\Caches' from UPM and the results were the same. This could possibly affect the native Windows 2012 R2/Windows 8 start menus, but I am using Classic Shell for my published desktops and it is not affected. In my opinion the Windows 2012 R2/Windows 8 start menu is unacceptable for users coming from a Windows 7 or Windows 10 world anyway.

 

The next step was to create a proper fix. I needed a way to set this value back to zero AFTER UPM, but before Explorer loads. Using the 'Run' registry value might not spawn the fix before Explorer and GPO scripts are in the same boat. Active Setup is the perfect candidate for this. I created a little Active Setup registry key that runs this command "REG ADD HKCU\Software\Microsoft\Windows\CurrentVersion\ImmersiveShell\StateStore /v ResetCache /t REG_DWORD /d 0 /f". I also excluded this Active Setup key from UPM so that it will run every time. This has made a tremendous difference for my published desktop logon times.

 

Instructions:

 

1.) Add Active Setup registry key/values (save the text below into a 'DisableUPMResetCache.reg' file):

 

Windows Registry Editor Version 5.00

 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components\DisableUPMResetCache]

@="DisableUPMResetCache"

"Version"="1,1,1,1"

"StubPath"="REG ADD HKCU\Software\Microsoft\Windows\CurrentVersion\ImmersiveShell\StateStore /v ResetCache /t REG_DWORD /d 0 /f"

"Locale"="*"

 

2.) Add UPM exclusions:

 

Directory - '!ctx_localappdata!\Microsoft\Windows\Caches'

Registry - 'SOFTWARE\Microsoft\Active Setup\Installed Components\DisableUPMResetCache'

 

3.) Didn't seem necessary, but remove the '%LOCALAPPDATA%\Microsoft\Windows\Caches' directories for affected user profiles.

 

If you block CMD/REG via GPO you may have to loosen that up, so this can run.

[Andy White1709154166]

I'm going to try this today!  Well done, hope this works too.

 

Where did you add DisableUPMResetCache.reg just on a new GPO?

[Mark DePalma1709157107]

I'm going to try this today!  Well done, hope this works too.

 

Where did you add DisableUPMResetCache.reg just on a new GPO?

 

I did it through GPP for now and may just move it it to my MCS image, but you can apply the .reg directly to your worker server(s). Also, you can look at the event log I mentioned on the server to see if this same issue is happening to you.

[Andy White1709154166]

Did this reg file go under the user or computer area of the GPO, I guess computer?

[Joe Robinson]

Did this reg file go under the user or computer area of the GPO, I guess computer?

 

HKLM settings are for the Computer, HKCU settings are for the user.

[Andy White1709154166]

Where do I exclude:

 

SOFTWARE\Microsoft\Active Setup\Installed Components\DisableUPMResetCache

 

In the user profile group policy I only see Exclusion lists for files and directories?

[Mark DePalma1709157107]

Where do I exclude:

 

SOFTWARE\Microsoft\Active Setup\Installed Components\DisableUPMResetCache

 

In the user profile group policy I only see Exclusion lists for files and directories?

 

Computer Configuration > Administrative Templates > Citrix Components > Profile Management > Registry > Exclusion list

 

The folder exclusion would be there under: Computer Configuration > Administrative Templates > Citrix Components > Profile Management > File system > Exclusion list - directories

[Sascha Matter1709156668]

Thanks for that hint.

 

When I run this reg.key the login is really a bit faster. I added your regkey on the master image, but I'm not sure if the key was added correctly.

 

The "StubPath" key is missing on the "DisableUPMResetCache" folder inside the registry as visible in the attached pic.

 

Is that correct or do I have to add the stubpath?

 

 

[Mark DePalma1709157107]

Thanks for that hint.

 

When I run this reg.key the login is really a bit faster. I added your regkey on the master image, but I'm not sure if the key was added correctly.

 

The "StubPath" key is missing on the "DisableUPMResetCache" folder inside the registry as visible in the attached pic.

 

Is that correct or do I have to add the stubpath?

StubPath needs to be there. That is the string that actually defines what needs to be run. It should be called "StubPath" with a type of REG_SZ. The value should be: "REG ADD HKCU\Software\Microsoft\Windows\CurrentVersion\ImmersiveShell\StateStore /v ResetCache /t REG_DWORD /d 0 /f".

 

[Sascha Matter1709156668]

Just changed it to this, but the result isnt that good, specially it's not very constant. Sometimes it's really fast, after a reboot it takes again between 30 und 300s. Only when I log in with a temporary profile, then it's super fast.

 

Are your W2K12 VM's also PVS based?

[Mark DePalma1709157107]

Just changed it to this, but the result isnt that good, specially it's not very constant. Sometimes it's really fast, after a reboot it takes again between 30 und 300s. Only when I log in with a temporary profile, then it's super fast.

 

Are your W2K12 VM's also PVS based?

 

I am using MCS. Like I said in earlier posts, I would look at the 'Microsoft-Windows-Shell-Core/Operational' event log. This gives a lot of information on timing during shell load. It is very clear in this log if this specific issue is affecting you. Also, do you get the same delay when launching a seamless application? If you do, then it is not shell related. My issue was specifically with published desktops (seamless applications were OK), so that alone pointed me in the direction of looking at the shell. Another thing to verify is that silent regedit changes are allowed in your environment. Active Setup could be running, but the required registry change might not actually be happening.