Jump to content
Welcome to our new Citrix community!
  • 0

Another lot of registry bloats on Server 2019


Ken Z

Question

Hi everyone

 

has anyone come across the SYSTEM registry getting bloated when using audio/Microphones on a Citrix 1912 Farm?

 

I've got a Citrix farm running Server 2019 and 1912 CU4 as the Citrix version.

we're using headphones when connecting to the Citrix farm and using the Speech Recognition component of Server 2019 for an application.

We've noticed that the SYSTEM registry hive has grown to 400MB and is still growing (taking up 800MB in C:\windows\system32\config but sysinternals "ru" utility reporting 400MB used

we've traced the registry  bloating to the following keys

 

HKLM\SYSTEM\CurrentControlSet\control\Class\c166523c-fe0c-4a94-a586-f1a80cfbbf3e                           - Audio

HKLM\SYSTEM\CurrentControlSet\control\DeviceClasses\2EEF81BE-33FA-4800-9670-1CD474972C3F   - Microphone device

HKLM\SYSTEM\CurrentControlSet\control\DeviceClasses\e6327cad-dcec-4949-ae8a-991e976a79d2      - Audio Device

HKLM\SYSTEM\CurrentControlSet\Enum\SWD\MMDEVAPI                                                                                             - Audio Endpoints

 

(also the corresponding ControlSet001 and ControlSet002 keys)

 

Has anyone else noticed this behaviour?

 

If anyone else has this issue it would be useful to compare notes

 

Regards

 

Ken Z

Link to comment

4 answers to this question

Recommended Posts

  • 0

Did you ever make any progress on resolving this?

We have a group of servers doing the same thing. 2 have hit the 1.5G limit of the SYSTEM hive file and are effectively useless at this time.

We are on 1912 CU1.

We have opened a case with Citrix support and are pending assistance with it.

 

We've run these commands to count the entries on those bloated keys. Results are up to about 16,000 entries per key.

(Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}').count

(Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}').count


(Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}').count

(Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Enum\SWD\MMDEVAPI').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Enum\SWD\MMDEVAPI').count
(Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Enum\SWD\MMDEVAPI').count

 

We've run these to delete those items.

Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}' | Where {$_.Name -ne "Properties"} | Remove-Item
Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}' | Where {$_.Name -ne "Properties"} | Remove-Item
Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}' | Where {$_.Name -ne "Properties"} | Remove-Item


Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}'  | Where {$_.Name -like "*##?#SWD#MMDEVAPI#*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}'  | Where {$_.Name -like "*##?#SWD#MMDEVAPI#*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}'  | Where {$_.Name -like "*##?#SWD#MMDEVAPI#*"} | Remove-Item -Recurse


Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}' | Where {$_.Name -like "*##?#SWD#MMDEVAPI*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}' | Where {$_.Name -like "*##?#SWD#MMDEVAPI*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}' | Where {$_.Name -like "*##?#SWD#MMDEVAPI*"} | Remove-Item -Recurse

Get-ChildItem 'HKLM:\SYSTEM\CurrentControlSet\Enum\SWD\MMDEVAPI' | Where {$_.Name -like "*{3.0.*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet001\Enum\SWD\MMDEVAPI' | Where {$_.Name -like "*{3.0.*"} | Remove-Item -Recurse
Get-ChildItem 'HKLM:\SYSTEM\ControlSet002\Enum\SWD\MMDEVAPI' | Where {$_.Name -like "*{3.0.*"} | Remove-Item -Recurse

 

After deleting them the server is kind of unstable until we do a hive compression process and create a new hive file. This only saves 100-200MB, so there's got to be bloat happening elsewhere as well.

Link to comment
  • 0

Hi Dan

 

Are you using Audio redirection? 

 

ignore the ControlSet01 and ControlSet02. These are just backups of CurrentControlSet.

 

I'd initially logged this with Citrix but they closed the call saying it's a Microsoft issue.

I'd then logged it with Microsoft who are still investigating this, and it's been a few weeks, but they don't seem to have a clue.

 

Also, can you check another registry key for me please?

 

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\PnpLockdownFiles

 

use you PowerShell commands to count the size

 

Regards

 

Ken Z

 

 

Link to comment
  • 0

Dan

 

a good way to find which keys are causing the bloating is to use the sysinternals ru64.exe utility

Copy it to a system that has a bloated registry and run is as follows

 

ru64.exe HKLM\SYSTEM -l 1

or 

ru64.exe HKLM\SOFTWARE -l 1

 

this lists all the first level keys under SYSTEM or SOFTWARE and tells you the total size of all the keys under each one.

Once you find a key/set of keys that huge, you can drill down further, eg.

 

ru64.exe HKLM\SYSTEM\CurrentControlSet\Enum -l 1

 

to list all the keys under Enum and see each top level key there

 

Regards

 

Ken Z

 

Link to comment
  • 0

Ken,

Sorry for the delayed response.

I also opened a case with Citrix and they said it's a MS issue as well.

Using RU we were able to identify several different registry locations that had bloat.

Then we devised a powershell script to purge them all, with lots of testing and validation along the way.

We have 9 machines we all brought from 1G+ SYSTEM hive files down to 30MB. 

It's been 2 months and they are all working 100% fine, actually better than before.

Our overall process to fix our machines looks like this:

  1. Drain and reboot server.
  2. Run powershell script
  3. Reboot to a recovery console
  4. Run regedit in the recovery console and load the bloated (but now cleaned) SYSTEM hive file.
  5. Export SYSTEM hive that to a new hive file. This creates a new file minus all the bloated stuff we purged.
  6. Disconnect the attached SYSTEM hive file.
  7. Rename the bloated SYSTEM hive file to SYSTEM.old
  8. Rename the newly created unbloated hive file to SYSTEM.
  9. Reboot the server normally.
  10. Done.

Our on going maintenance for these is to have that powershell script execute as a scheduled task that kicks off at server boot.
It will delete the bloat and leave it as whitespace inside the hive file.

Currently all our servers are still sitting at under 40MB for the SYSTEM hive file.

 

These were the locations we saw major bloat in. The bulk of the space was for printers. They were all remnants of printers, monitors, audio devices that redirect on connection but do not clear out on logoff or reboot.

  • HKLM:\SYSTEM\CurrentControlSet\Enum\SWD\PRINTENUM
  • HKLM:\SYSTEM\CurrentControlSet\Enum\SWD\MMDEVAPI
  • HKLM:\system\CurrentControlSet\control\DeviceClasses\{0ecef634-6ef0-472a-8085-5ad023ecbccd}
  • HKLM:\system\CurrentControlSet\control\DeviceContainers
  • HKLM:\system\CurrentControlSet\control\Class\{1ed2bbf9-11f0-4084-b21f-ad83a8e6dcdc}
  • HKLM:\SYSTEM\CurrentControlSet\Control\DeviceClasses\{2eef81be-33fa-4800-9670-1cd474972c3f}
  • HKLM:\SYSTEM\CurrentControlSet\Control\Class\{c166523c-fe0c-4a94-a586-f1a80cfbbf3e}
  • HKLM:\SYSTEM\CurrentControlSet\control\DeviceClasses\{e6327cad-dcec-4949-ae8a-991e976a79d2}
  • HKLM:\system\CurrentControlSet\Control\DeviceContainers\{00000000-0000-0000-FFFF-FFFFFFFFFFFF}\BaseContainers\{00000000-0000-0000-FFFF-FFFFFFFFFFFF}

In regards to the ControlSet01, ControlSet02, and CurrentControlSet. My conclusion is as you stated. We were purging them all, only to realize it's triple the work for no gain. Just going CurrentControlSet and getting in one good reboot got them all in sync. I believe CurrentControlSet is just a pointer to either 01 or 02, with 02 being what we consider "LastKnownGood" when booting to safe mode.

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...