Testing App Layering 2304, after upgrading and publishing image template, my VDAs are missing ICA Session performance counters....

[Sergio Masone1709161115]

Upgraded ELM to 2304, published image template, i noticed in our monitoring console and monitor cannot retrieve ica latency metrics, further investigation, the ICA SEssion performance counters are completely missing. dont hav ethis issue with the previous version of ELM. anyone else have this behavior?
Thanks

[Vincent Bernard]

@Rob Zylowski1709158051Rebuild of the Plafform Layer didn't resolve the issue.

 

Executing powershell command 'Get-Counter -ListSet * | Sort-Object -Property CounterSetName | Format-Table CounterSetName, CounterSetType -AutoSize | Out-File C:\Pe
rfCounters.txt' and comparing with a vdisk leveraging 2211 shows the following perfCounters are missing:

 

BITS Net Utilization, Browser, Cache, Citrix GDT VC, ICA Session, ICMP, ICMPv6, IPv4, IPv6, Job Object, Job Object Details, LogicalDisk, Memory, MSDTC Bridge 3.0.0.0,                                   
MSDTC Bridge 4.0.0.0, NBT Connection, Network Adapter, Network Interface, NUMA Node Memory, Objects, Paging File, PhysicalDisk, Print Queue, Process,                                                 
Processed Logoff Files - Under 1KB, Processor, RAS Port, RAS Total, Redirector, Search Gatherer, Search Gatherer Projects, Search Indexer, Security Per-Process Statistics,                         
Security System-Wide Statistics, Server, Server Work Queues, ServiceModelEndpoint 3.0.0.0, ServiceModelOperations 3.0.0.0, ServiceModelService 3.0.0.0,                            
SMSvcHost 3.0.0.0, SMSvcHost 4.0.0.0, System, TCPv4, TCPv6, Telephony, Terminal Services Session, Thread, UDPv4, UDPv6, USB, WF (System.Workflow) 4.0.0.0,                           
Windows Workflow Foundation, WMI Objects.

 

Command "Get-BrokerMachine -AdminAddress <deliveryController> -MaxRecordCount 1000 | where {$_.DesktopGroupName -like "*<pattern>*"} | select DNSName,LoadIndex,LoadIndexes,SessionCount | sort DNSName"  shows LoadIndexes are not being correctly calculated.

                                       

Restore to version 2211 resolved our issue.

[Rob Zylowski]

Try creating a new platform layer. 

[Vincent Bernard]

@Sergio Masone1709161115Yes, we had issue with the performance counters too.   @Rob Zylowski1709158051I give that a try and feedback the outcome

[Mike Kelly1709153237]

We have the same exact issue.  No issues with performance counters on 2211 and now we see the issue on 2304.  Rebuilt platform layer does not resolve.

 

Has anyone on this thread opened a ticket on this?  Seems like Citrix could EASILY reproduce this.

[Sergio Masone1709161115]

 

3 hours ago, Mike Kelly1709153237 said:

We have the same exact issue.  No issues with performance counters on 2211 and now we see the issue on 2304.  Rebuilt platform layer does not resolve.

 

Has anyone on this thread opened a ticket on this?  Seems like Citrix could EASILY reproduce this.

I will open a ticket first thing tomorrow morning and update this as i get news. Thank you all for confirming the issue

[Darryl Sakach]

We have been experiencing this issue since we upgraded from VDA 1912 to VDA 2203. It was found to be an issue with App Layering. There have been two reported fixes to App Layering so far to fix this, the 2304 update being the latest. We have not tested 2304 yet so we do not know if it resolves the issue, but it sounds like we should not be hopeful.

[Chip Gonsalves]

One question that we have for anyone seeing this issue, can you please confirm that you have the directory c:\windows\setup\scripts\kmsdir?  Looking at the code, the only reason that the new code could be running into an issue is only if that directory doesn't exist.  And for our testing, we don't know how you can have a scenario in which that directory doesn't exist, or else there will be other problems on the deployed images.  So please could I get some of you to reply yes the directory exists or no the directory does not exist?

[Sergio Masone1709161115]

19 minutes ago, Chip Gonsalves said:

One question that we have for anyone seeing this issue, can you please confirm that you have the directory c:\windows\setup\scripts\kmsdir?  Looking at the code, the only reason that the new code could be running into an issue is only if that directory doesn't exist.  And for our testing, we don't know how you can have a scenario in which that directory doesn't exist, or else there will be other problems on the deployed images.  So please could I get some of you to reply yes the directory exists or no the directory does not exist?

@Chip Gonsalves
I confirm this directory and its contents exist in my vda.

[Mike Kelly1709153237]

8 minutes ago, Sergio Masone1709161115 said:

@Chip Gonsalves
I confirm this directory and its contents exist in my vda.

Citrix support issued us a fix...  We had to run the "citrix_app_layering_os_machine_tools_23.4.0.exe" install on our OS layers and then rebuild the performance counters in the same version of the OS layer  C:\windows\system32\lodctr /R and C:\windows\syswow64\lodctr \R - reboot and finalized both of our OS layers and then redeployed our images using this version of the OS layer and the performance counters are working again.  Unsure what is so dramatically different in this version of app layering.  As I understand it, its not required to run those tools with each app layering upgrade.   We had files in there back from March of 2018 and everything has been working fine in this regard all the way up to 2211.

[Chip Gonsalves]

For those that answer "yes" the directory does exist, can you please look for a file called c:\windows\setup\scripts\kmsdir\PerfGenRecompile.cmd in the image that was deployed?  And also please let us know if you have updated the gold image tools to the 2304 version since there was a change to the kmssetup.cmd that will execute the PerfGenRecompile.cmd to allow the perf counters to be registered.  

[Sergio Masone1709161115]

1 minute ago, Chip Gonsalves said:

For those that answer "yes" the directory does exist, can you please look for a file called c:\windows\setup\scripts\kmsdir\PerfGenRecompile.cmd in the image that was deployed?  And also please let us know if you have updated the gold image tools to the 2304 version since there was a change to the kmssetup.cmd that will execute the PerfGenRecompile.cmd to allow the perf counters to be registered.  

i do see this file, i was just looking at it when you asked the question concerning this. on my part i didnt update the os tools in my OS layer yet.
the script your referring has this in contents, and naturally that log file doesnt exist since it prob didnt run due to tools out of date:

 

@echo off
Setlocal EnableDelayedExpansion

cd \windows\setup\scripts\kmsdir < is this missing a C:?

echo !date!-!time!-PerfGenRecompile.cmd: Begin Register Of Percounters from Ini files >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/Configuration/module.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/Configuration/module.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/Configuration/module_Wince.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/Configuration/module_Wince.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/module.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/module.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Citrix/HDX/bin/IcaPerf.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files/Citrix/HDX/bin/IcaPerf.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Citrix/User Profile Manager/upmPerf.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files/Citrix/User Profile Manager/upmPerf.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Microsoft Office/root/Office16/1033/OUTLPERF.INI  >>PerfGenRecompile.log
lodctr "C:/Program Files/Microsoft Office/root/Office16/1033/OUTLPERF.INI" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Rebuild counters >>PerfGenRecompile.log
lodctr /r >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Recompile Complete >>PerfGenRecompile.log

do we simply need to install the OS layer tools, then republish and it should recompile the perf counters afterwards on publish?

[Chip Gonsalves]

3 minutes ago, Mike Kelly1709153237 said:

Citrix support issued us a fix...  We had to run the "citrix_app_layering_os_machine_tools_23.4.0.exe" install on our OS layers and then rebuild the performance counters in the same version of the OS layer  C:\windows\system32\lodctr /R and C:\windows\syswow64\lodctr \R - reboot and finalized both of our OS layers and then redeployed our images using this version of the OS layer and the performance counters are working again.  Unsure what is so dramatically different in this version of app layering.  As I understand it, its not required to run those tools with each app layering upgrade.   We had files in there back from March of 2018 and everything has been working fine in this regard all the way up to 2211.

In my latest reply you will note that we made a change to the kmssetup.cmd in the latest tools to ensure that all performance counters found in all layers would be re-registered.  The procedure that support said of having to rebuild the performance counters is not correct.  You simply only need to run the gold image tools (which is just an unzip) in your OS revision to get the updates.  What changed was we needed to ensure that perf counters from every layer (all application layers, platform layers, and for those using full user layers) would properly be updated on each boot. 

[Chip Gonsalves]

Just now, Sergio Masone1709161115 said:

i do see this file, i was just looking at it when you asked the question concerning this. on my part i didnt update the os tools in my OS layer yet.
the script your referring has this in contents, and naturally that log file doesnt exist since it prob didnt run due to tools out of date:

 

@echo off
Setlocal EnableDelayedExpansion

cd \windows\setup\scripts\kmsdir < is this missing a C:?

echo !date!-!time!-PerfGenRecompile.cmd: Begin Register Of Percounters from Ini files >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/Configuration/module.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/Configuration/module.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/Configuration/module_Wince.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/Configuration/module_Wince.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files (x86)/Citrix/ICA Client/module.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files (x86)/Citrix/ICA Client/module.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Citrix/HDX/bin/IcaPerf.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files/Citrix/HDX/bin/IcaPerf.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Citrix/User Profile Manager/upmPerf.ini  >>PerfGenRecompile.log
lodctr "C:/Program Files/Citrix/User Profile Manager/upmPerf.ini" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Register C:/Program Files/Microsoft Office/root/Office16/1033/OUTLPERF.INI  >>PerfGenRecompile.log
lodctr "C:/Program Files/Microsoft Office/root/Office16/1033/OUTLPERF.INI" >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Rebuild counters >>PerfGenRecompile.log
lodctr /r >>PerfGenRecompile.log 2>&1
echo. >>PerfGenRecompile.log
echo !date!-!time!-PerfGenRecompile.cmd: Recompile Complete >>PerfGenRecompile.log

do we simply need to install the OS layer tools, then republish and it should recompile the perf counters afterwards on publish?

Yes.  Just update the tools in the OS layer.  Again that is just really doing an unzip.  There is never a need to run the setup64 ever again.  Also, no need to rebuild the perf counters as was reported before.  Just update with the tools, finalize, and republish.

 

[Sergio Masone1709161115]

Just now, Chip Gonsalves said:

Yes.  Just update the tools in the OS layer.  Again that is just really doing an unzip.  There is never a need to run the setup64 ever again.  Also, no need to rebuild the perf counters as was reported before.  Just update with the tools, finalize, and republish.

 

I will give this a try and repost.

[Mike Kelly1709153237]

4 minutes ago, Chip Gonsalves said:

In my latest reply you will note that we made a change to the kmssetup.cmd in the latest tools to ensure that all performance counters found in all layers would be re-registered.  The procedure that support said of having to rebuild the performance counters is not correct.  You simply only need to run the gold image tools (which is just an unzip) in your OS revision to get the updates.  What changed was we needed to ensure that perf counters from every layer (all application layers, platform layers, and for those using full user layers) would properly be updated on each boot. 

Well, I initially deployed the images with ONLY the updated tools in the OS layer and there were still some performance counters that were broken so I versioned the OS layer again and rebuilt the performance counters and redeployed and that fixed the issue for us.  I was able to reproduce this on both of our Server 2016 and 2019 images / OS layers.

 

I wanted to state this clearly to save people time and not waste time like I did trying to figure this out.

[Mike Kelly1709153237]

20 minutes ago, Chip Gonsalves said:

For those that answer "yes" the directory does exist, can you please look for a file called c:\windows\setup\scripts\kmsdir\PerfGenRecompile.cmd in the image that was deployed?  And also please let us know if you have updated the gold image tools to the 2304 version since there was a change to the kmssetup.cmd that will execute the PerfGenRecompile.cmd to allow the perf counters to be registered.  

 @Chip Gonsalves    - Where is this outlined in the Release notes / fixed issues? Seems that this dramatic change in the scripts should be clearly outlined on the customer facing docs page so it can be clearly seen.  I read that information with every release and I wouldn't even be here talking to you right now if it would have been posted there.

[Sergio Masone1709161115]

10 minutes ago, Mike Kelly1709153237 said:

Well, I initially deployed the images with ONLY the updated tools in the OS layer and there were still some performance counters that were broken so I versioned the OS layer again and rebuilt the performance counters and redeployed and that fixed the issue for us.  I was able to reproduce this on both of our Server 2016 and 2019 images / OS layers.

 

I wanted to state this clearly to save people time and not waste time like I did trying to figure this out.

Hi Mike,

Thanks for sharing that
ill export the reg settings for the perfcounters on my current image using loctr, then compare it to the new publish to see if i have some missing.

[Mike Kelly1709153237]

1 minute ago, Sergio Masone1709161115 said:

Hi Mike,

Thanks for sharing that
ill export the reg settings for the perfcounters on my current image using loctr, then compare it to the new publish to see if i have some missing.

Sounds good Sergio.  I'd be curious what you see as we had no issues with performance counters in any of our images prior to 2304 (was running 2211 prior).

[Chip Gonsalves]

1 hour ago, Mike Kelly1709153237 said:

 @Chip Gonsalves    - Where is this outlined in the Release notes / fixed issues? Seems that this dramatic change in the scripts should be clearly outlined on the customer facing docs page so it can be clearly seen.  I read that information with every release and I wouldn't even be here talking to you right now if it would have been posted there.

We had called out that the external documentation needed to be updated to explicitly indicate that the gold image tools needed to be updated. Somewhere in the deployment process either that was not properly reported to the documentation team or else there was a disconnect on their part with regards to whether they should be included.  Once I am done with the review of the breakdown, I will ask the documentation team to update the "What's new" to be explicit.  And I will be asking them to include that a rebuild of the counters is needed at that time as well.  And just to be clear, in all of our testing during development, nightly automation testing, our internal dogfood, and 100% of the customers that were reporting the original issues of performance counters being inconsistent, absolutely none of them required a rebuild.  That is the number one reason we didn't think it was needed, because in all of our testing and at every customer site that we had them deploy the fix, we first confirmed that it fixed their issue, and we also found that the rebuild was not needed.  Thanks to your input, we will be adding it to the documentation, and in the next version of the product, we will do it automatically on every OS revision (it is a quick operation and should never cause any harm).  And now that this forum has told us it is sometimes needed, we will do it always.

[Sergio Masone1709161115]

2 hours ago, Mike Kelly1709153237 said:

Sounds good Sergio.  I'd be curious what you see as we had no issues with performance counters in any of our images prior to 2304 (was running 2211 prior).

Good Catch Mike! you are absolutely right, I'm missing a lot of counters compared to 2211, I'll have to reopen OS layer and re-register the counters in it.

Before upgrade (2211):

[Perflib]
Base Index=1847
Last Counter=14258
Last Help=14259

[PERF_.NET CLR Data]
First Counter=6840
First Help=6841
Last Counter=6852
Last Help=6853

[PERF_.NET CLR Networking]
First Counter=6828
First Help=6829
Last Counter=6838
Last Help=6839

[PERF_.NET CLR Networking 4.0.0.0]
First Counter=6900
First Help=6901
Last Counter=6926
Last Help=6927

[PERF_.NET Data Provider for Oracle]
First Counter=8916
First Help=8917
Last Counter=8944
Last Help=8945

[PERF_.NET Data Provider for SqlServer]
First Counter=6854
First Help=6855
Last Counter=6882
Last Help=6883

[PERF_.NET Memory Cache 4.0]
First Counter=6884
First Help=6885
Last Counter=6898
Last Help=6899

[PERF_.NETFramework]
First Counter=8740
First Help=8741
Last Counter=8914
Last Help=8915

[PERF_BITS]
First Counter=8966
First Help=8967
Last Counter=8982
Last Help=8983

[PERF_Citrix User Profile Manager]
First Counter=13778
First Help=13779
Last Counter=13826
Last Help=13827

[PERF_CitrixBrokerAgent]
First Counter=11064
First Help=11065
Last Counter=11110
Last Help=11111

[PERF_CitrixICA]
First Counter=13404
First Help=13405
Last Counter=13574
Last Help=13575

[PERF_ESENT]
First Counter=6928
First Help=6929
Last Counter=8708
Last Help=8709

[PERF_LSM]
First Counter=9042
First Help=9043
Last Counter=9048
Last Help=9049

[PERF_Microsoft Dynamics AX Client 6.3]
First Counter=10292
First Help=10293
Last Counter=10314
Last Help=10315

[PERF_MSDTC]
First Counter=6776
First Help=6777
Last Counter=6802
Last Help=6803

[PERF_MSDTC Bridge 3.0.0.0]
First Counter=10024
First Help=10025
Last Counter=10046
Last Help=10047

[PERF_MSDTC Bridge 4.0.0.0]
First Counter=6804
First Help=6805
Last Counter=6826
Last Help=6827

[PERF_rdyboost]
First Counter=8984
First Help=8985
Last Counter=9004
Last Help=9005

[PERF_RemoteAccess]
First Counter=9050
First Help=9051
Last Counter=9088
Last Help=9089

[PERF_ServiceModelEndpoint 3.0.0.0]
First Counter=9916
First Help=9917
Last Counter=9954
Last Help=9955

[PERF_ServiceModelOperation 3.0.0.0]
First Counter=10048
First Help=10049
Last Counter=10078
Last Help=10079

[PERF_ServiceModelService 3.0.0.0]
First Counter=9956
First Help=9957
Last Counter=10022
Last Help=10023

[PERF_SMSvcHost 3.0.0.0]
First Counter=9886
First Help=9887
Last Counter=9914
Last Help=9915

[PERF_SMSvcHost 4.0.0.0]
First Counter=8710
First Help=8711
Last Counter=8738
Last Help=8739

[PERF_TapiSrv]
First Counter=8946
First Help=8947
Last Counter=8964
Last Help=8965

[PERF_TermService]
First Counter=6774
First Help=6775
Last Counter=6774
Last Help=6775

[PERF_UGatherer]
First Counter=9134
First Help=9135
Last Counter=9238
Last Help=9239

[PERF_UGTHRSVC]
First Counter=9240
First Help=9241
Last Counter=9310
Last Help=9311

[PERF_usbhub]
First Counter=9006
First Help=9007
Last Counter=9040
Last Help=9041

[PERF_Windows Workflow Foundation 3.0.0.0]
First Counter=10080
First Help=10081
Last Counter=10122
Last Help=10123

[PERF_Windows Workflow Foundation 4.0.0.0]
First Counter=9090
First Help=9091
Last Counter=9132
Last Help=9133

[PERF_WmiApRpl]
First Counter=14092
First Help=14093
Last Counter=14258
Last Help=14259

[PERF_WSearchIdxPi]
First Counter=9312
First Help=9313
Last Counter=9438
Last Help=9439

After (2304):
 

[Perflib]
Base Index=1847
Last Counter=7644
Last Help=7645

[PERF_Citrix User Profile Manager]
First Counter=2188
First Help=2189
Last Counter=2236
Last Help=2237

[PERF_CitrixICA]
First Counter=2016
First Help=2017
Last Counter=2186
Last Help=2187

[PERF_WmiApRpl]
First Counter=7478
First Help=7479
Last Counter=7644
Last Help=7645

[Mike Kelly1709153237]

5 minutes ago, Sergio Masone1709161115 said:

Good Catch Mike! you are absolutely right, I'm missing a lot of counters compared to 2211, I'll have to reopen OS layer and re-register the counters in it.

Before upgrade (2211):

[Perflib]
Base Index=1847
Last Counter=14258
Last Help=14259

[PERF_.NET CLR Data]
First Counter=6840
First Help=6841
Last Counter=6852
Last Help=6853

[PERF_.NET CLR Networking]
First Counter=6828
First Help=6829
Last Counter=6838
Last Help=6839

[PERF_.NET CLR Networking 4.0.0.0]
First Counter=6900
First Help=6901
Last Counter=6926
Last Help=6927

[PERF_.NET Data Provider for Oracle]
First Counter=8916
First Help=8917
Last Counter=8944
Last Help=8945

[PERF_.NET Data Provider for SqlServer]
First Counter=6854
First Help=6855
Last Counter=6882
Last Help=6883

[PERF_.NET Memory Cache 4.0]
First Counter=6884
First Help=6885
Last Counter=6898
Last Help=6899

[PERF_.NETFramework]
First Counter=8740
First Help=8741
Last Counter=8914
Last Help=8915

[PERF_BITS]
First Counter=8966
First Help=8967
Last Counter=8982
Last Help=8983

[PERF_Citrix User Profile Manager]
First Counter=13778
First Help=13779
Last Counter=13826
Last Help=13827

[PERF_CitrixBrokerAgent]
First Counter=11064
First Help=11065
Last Counter=11110
Last Help=11111

[PERF_CitrixICA]
First Counter=13404
First Help=13405
Last Counter=13574
Last Help=13575

[PERF_ESENT]
First Counter=6928
First Help=6929
Last Counter=8708
Last Help=8709

[PERF_LSM]
First Counter=9042
First Help=9043
Last Counter=9048
Last Help=9049

[PERF_Microsoft Dynamics AX Client 6.3]
First Counter=10292
First Help=10293
Last Counter=10314
Last Help=10315

[PERF_MSDTC]
First Counter=6776
First Help=6777
Last Counter=6802
Last Help=6803

[PERF_MSDTC Bridge 3.0.0.0]
First Counter=10024
First Help=10025
Last Counter=10046
Last Help=10047

[PERF_MSDTC Bridge 4.0.0.0]
First Counter=6804
First Help=6805
Last Counter=6826
Last Help=6827

[PERF_rdyboost]
First Counter=8984
First Help=8985
Last Counter=9004
Last Help=9005

[PERF_RemoteAccess]
First Counter=9050
First Help=9051
Last Counter=9088
Last Help=9089

[PERF_ServiceModelEndpoint 3.0.0.0]
First Counter=9916
First Help=9917
Last Counter=9954
Last Help=9955

[PERF_ServiceModelOperation 3.0.0.0]
First Counter=10048
First Help=10049
Last Counter=10078
Last Help=10079

[PERF_ServiceModelService 3.0.0.0]
First Counter=9956
First Help=9957
Last Counter=10022
Last Help=10023

[PERF_SMSvcHost 3.0.0.0]
First Counter=9886
First Help=9887
Last Counter=9914
Last Help=9915

[PERF_SMSvcHost 4.0.0.0]
First Counter=8710
First Help=8711
Last Counter=8738
Last Help=8739

[PERF_TapiSrv]
First Counter=8946
First Help=8947
Last Counter=8964
Last Help=8965

[PERF_TermService]
First Counter=6774
First Help=6775
Last Counter=6774
Last Help=6775

[PERF_UGatherer]
First Counter=9134
First Help=9135
Last Counter=9238
Last Help=9239

[PERF_UGTHRSVC]
First Counter=9240
First Help=9241
Last Counter=9310
Last Help=9311

[PERF_usbhub]
First Counter=9006
First Help=9007
Last Counter=9040
Last Help=9041

[PERF_Windows Workflow Foundation 3.0.0.0]
First Counter=10080
First Help=10081
Last Counter=10122
Last Help=10123

[PERF_Windows Workflow Foundation 4.0.0.0]
First Counter=9090
First Help=9091
Last Counter=9132
Last Help=9133

[PERF_WmiApRpl]
First Counter=14092
First Help=14093
Last Counter=14258
Last Help=14259

[PERF_WSearchIdxPi]
First Counter=9312
First Help=9313
Last Counter=9438
Last Help=9439

After (2304):
 

[Perflib]
Base Index=1847
Last Counter=7644
Last Help=7645

[PERF_Citrix User Profile Manager]
First Counter=2188
First Help=2189
Last Counter=2236
Last Help=2237

[PERF_CitrixICA]
First Counter=2016
First Help=2017
Last Counter=2186
Last Help=2187

[PERF_WmiApRpl]
First Counter=7478
First Help=7479
Last Counter=7644
Last Help=7645

 

I'm glad you could confirm!  

[Mike Kelly1709153237]

23 hours ago, Chip Gonsalves said:

We had called out that the external documentation needed to be updated to explicitly indicate that the gold image tools needed to be updated. Somewhere in the deployment process either that was not properly reported to the documentation team or else there was a disconnect on their part with regards to whether they should be included.  Once I am done with the review of the breakdown, I will ask the documentation team to update the "What's new" to be explicit.  And I will be asking them to include that a rebuild of the counters is needed at that time as well.  And just to be clear, in all of our testing during development, nightly automation testing, our internal dogfood, and 100% of the customers that were reporting the original issues of performance counters being inconsistent, absolutely none of them required a rebuild.  That is the number one reason we didn't think it was needed, because in all of our testing and at every customer site that we had them deploy the fix, we first confirmed that it fixed their issue, and we also found that the rebuild was not needed.  Thanks to your input, we will be adding it to the documentation, and in the next version of the product, we will do it automatically on every OS revision (it is a quick operation and should never cause any harm).  And now that this forum has told us it is sometimes needed, we will do it always.

Thanks Chip.

 

I see you guys updated the App layering documentation but there is an error.

 

it should be lodctr /R not LOADCTR /r    - FYI ?

 

Performance counters now work from any layer, including app layers and full user layers. The gold image tools must be updated in the OS revision to ensure the performance counters are correctly rebulit when booting an image. The OS revison needs at least one rebuild of the performance counters. Open an administrator command window in the same OS revision where you’re updating the gold image and run the commands c:\windows\system32\loadctr /r and c:\windows\syswow64\loadctr \r.

[Chip Gonsalves]

16 hours ago, Mike Kelly1709153237 said:

Thanks Chip.

 

I see you guys updated the App layering documentation but there is an error.

 

it should be lodctr /R not LOADCTR /r    - FYI ?

 

Performance counters now work from any layer, including app layers and full user layers. The gold image tools must be updated in the OS revision to ensure the performance counters are correctly rebulit when booting an image. The OS revison needs at least one rebuild of the performance counters. Open an administrator command window in the same OS revision where you’re updating the gold image and run the commands c:\windows\system32\loadctr /r and c:\windows\syswow64\loadctr \r.

That is 100% my fault and not the doc teams.  Every time I type the word lodctr, my fingers refuse to not type loadctr.  The doc team put in exactly what I asked them to add, and they would have no way of knowing that it was wrong.  I have asked them to update it (and of course apologized for my blunder).  

 

Thanks, Mike for pointing it out.  And thanks to all of you for helping us to understand that the rebuild needed to be done in your OS layers.  That scan will be done automatically in the next release of the product.  Heads up, we will have to do the scan in the cmd file that is updated with the tools, so this will be the second time that the tools will need to be updated.  We promise it will not be a trend going forward.

 

[Mike Kelly1709153237]

On 5/9/2023 at 3:50 PM, Mike Kelly1709153237 said:

 

I'm glad you could confirm!  

Sergio, can you let me know if the CPU performance counter on your image fails sometime after the image boots?  We're finding that after we initially boot the image the CPU counter looks fine and then after 30 minutes or so it fails and requires the performance counter to be rebuilt to resolve.

[Sergio Masone1709161115]

2 hours ago, Mike Kelly1709153237 said:

Sergio, can you let me know if the CPU performance counter on your image fails sometime after the image boots?  We're finding that after we initially boot the image the CPU counter looks fine and then after 30 minutes or so it fails and requires the performance counter to be rebuilt to resolve.

sure Mike, we are testing republishing today on two images, with the rebuild done in new OS version layer + machine tools. will let you know If i observe the same behavior with those counters.