Upgrade Storefront 1912 LTSR to 1912 CU1 failing (msi 1603 and "The directory is not empty" indication in eventvwr)

[BartP]

Hi all,

 

Testing the 1912 to 1912CU1 upgrade in the test environment.

Nu issues upgrading the DDC's and License server.

Storefront servers are hosted on Windows Server 2016.

 

When upgrading the Storefront servers the installer often terminates with the 1603 error code leaving the server in a useless state (services and IIS stores deleted).
Did several trials reverting the OS via VM snapshot and sometimes the install succeeds and sometimes it throws the 1603 error.
 

Logfile doesn't give a clue:
[18/11/2020 10:14:26][7508][Information] === CitrixStoreFront-x64.msi Installation exited. ===
[18/11/2020 10:14:26][7508][Error] Installation of '..\CitrixStoreFront-x64.msi' failed with error code 1603. Fatal error during installation
[18/11/2020 10:14:26][7508][Verbose] DeliveryServicesPkg : State = Failed
[18/11/2020 10:14:26][7508][Verbose] Broadcasting Event: package_failed
    Event Name: package_failed
    Event Arguments: 
        Id : DeliveryServicesPkg
        Message : Installation of '..\CitrixStoreFront-x64.msi' failed with error code 1603. Fatal error during installation
        Item : DeliveryServicesPkg (MsiPackage)

[18/11/2020 10:14:26][7508][Verbose] DeliveryServices : State = Failed
[18/11/2020 10:14:26][7508][Verbose] Broadcasting Event: package_failed
    Event Name: package_failed
    Event Arguments: 
        Id : DeliveryServices
        Message : An error occurred while updating one of Citrix StoreFront 1912.0.1000.17's subcomponents.
        Item : DeliveryServices (Feature)

[18/11/2020 10:14:26][7508][Verbose] DeliveryServicesRole : State = Failed
[18/11/2020 10:14:26][7508][Verbose] Broadcasting Event: package_failed
    Event Name: package_failed
    Event Arguments: 
        Id : DeliveryServicesRole
        Message : An error occurred while updating one of StoreFront's subcomponents.
        Item : DeliveryServicesRole (Role)

[18/11/2020 10:14:26][7508][Information] Requested Exit Code = 1603. Reason = Installation of DeliveryServicesRole failed.

 

 

 

When the installation fails I always find this indication in eventviewer (often pointing to a different subfolder of C:\Program Files\Citrix\Receiver StoreFront\Features):

 

An error occurred while running a PowerShell command.

Citrix.DeliveryServices.PowerShell.Command.RunnerInterfaces.Exceptions.PowerShellExecutionException, Citrix.DeliveryServices.PowerShell.Command.RunnerInterfaces, Version=3.22.0.0, Culture=neutral, PublicKeyToken=e8b77d454fa2a856
An error occurred while running a PowerShell command.

System.Exception, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
The directory is not empty.
: C:\Program Files\Citrix\Receiver StoreFront\Features\ExplicitFormsCommon
 

Also already tried stopping all Citrix services, IIS service and MS monitoring agent before starting the upgrade process.

 

Would like to find the root cause of this before proceeding to the production environment.

Any clues are welcome.

 

 

[Robert Seegers]

Hey,

 

check this post, it's a bit older but similar to your error message...

 

https://discussions.citrix.com/topic/378442-stf-36-upgrade-from-stf-35-to-36/page/3/

 

1. Test the Citrix StoreFront Protocol Transition Service. Start the Service. It will then shortly start and stop directly.
2. If this happens, open the following file in Editor: C:\Program Files\Citrix\Receiver StoreFront\Services\ProtocolTransitionService\Citrix.DeliveryServices.ProtocolTransition.ServiceHost.exe.config and change the Version from 3.0.0.0 to 3.5.0.0 (also when StoreFront shows up that Version 3.5.0.23 is installed in the Console) in each line except in line 4! Leave line 4 like it is and just change the value for Version in each other line below. Save the file.
3. Go back to Services and try to start the Citrix StoreFront Protocol Transition Service. When it starts ( I hope it will) then you are on the right way. Stop the Service.
4. Stop all other StoreFront Services and the WWW Publishing Service
5. Run the Upgrade

 

Versions Overview: https://support.citrix.com/article/CTX200356

 

hope that will help you...

 

Regards

 

Rob

[BartP]

26 minutes ago, Robert Seegers said:

Hey,

 

check this post, it's a bit older but similar to your error message...

 

https://discussions.citrix.com/topic/378442-stf-36-upgrade-from-stf-35-to-36/page/3/

 

1. Test the Citrix StoreFront Protocol Transition Service. Start the Service. It will then shortly start and stop directly.
2. If this happens, open the following file in Editor: C:\Program Files\Citrix\Receiver StoreFront\Services\ProtocolTransitionService\Citrix.DeliveryServices.ProtocolTransition.ServiceHost.exe.config and change the Version from 3.0.0.0 to 3.5.0.0 (also when StoreFront shows up that Version 3.5.0.23 is installed in the Console) in each line except in line 4! Leave line 4 like it is and just change the value for Version in each other line below. Save the file.
3. Go back to Services and try to start the Citrix StoreFront Protocol Transition Service. When it starts ( I hope it will) then you are on the right way. Stop the Service.
4. Stop all other StoreFront Services and the WWW Publishing Service
5. Run the Upgrade

 

Versions Overview: https://support.citrix.com/article/CTX200356

 

hope that will help you...

 

Regards

 

Rob

 

No ProtocoltransitionService in our environment.

[Robert Seegers]

Hello,

do you have any news for us, is everything working?

 

Regards

[BartP]

On 11/30/2020 at 9:44 AM, Robert Seegers said:

Hello,

do you have any news for us, is everything working?

 

Regards

No, didn't find a solution to make the installer stable for upgrade.

Decided to go for a scenario where I uninstall/reinstall storefront node per node.

 

[DOUGLAS CARNEY]

Happened to me today.

First failure, was due to Sophos AV considering the installer to be a crytoVirus.  Updated the sophos policy.  Reran and again got the 1603 error, this time not because of the AV software.  Digging through the logs, saw about the temporary:  C:\ProgramData\Citrix\Storefront Install.  I renamed that  with .bak, and update finally went.

 

 

Invalid version will be ignored: 'C:\ProgramData\Citrix\Storefront Install\3.20.0.34' Exception: System.Security.Cryptography.CryptographicException: The data is invalid.

   at System.Security.Cryptography.ProtectedData.Unprotect(Byte encryptedData, Byte optionalEntropy, DataProtectionScope scope)
   at Citrix.DeliveryServices.InstallController.Utilities.DPAPIUtils.DecryptData(Byte encyptedData)
   at Citrix.DeliveryServices.InstallController.Utilities.DPAPIUtils.DecryptFile(String filename)
   at Citrix.DeliveryServices.InstallController.Configuration.VersionData.<>c.<LoadInstanceBackups>b__51_0(FileInfo fil)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at Citrix.DeliveryServices.InstallController.Configuration.VersionData.LoadInstanceBackups()
   at Citrix.DeliveryServices.InstallController.Configuration.VersionData.Load(String backupPath, Version version)
   at Citrix.DeliveryServices.InstallController.ConfigurationController.LoadConfiguration()
 No snapshot exists for version '3.20.0.34', create one now.
 An error occurred creating the snapshot: 'System.IO.IOException: The file 'C:\ProgramData\Citrix\Storefront Install\3.20.0.34\Framework\Framework.xml' already exists.

[BartP]

And still experiencing exact same issues upgrading CU2 to CU3 now.

 

[Balaji]

I had faced the same issue while upgrading from 1912 SF CU1 to CU5. Got the attached error, From the MSI logs found the same logs as mentioned in the post.

 

Finally I was able to get to the issue and able to fix it and then upgraded SF to CU5

 

Issue was with the CitrixCOnfigurationReplication service account in CitrixStorefrontAdministrators Group showing as UNKOWN SID.

After removing the SID

rebooted SF

Then started the upgrade again by launching the installer

It completed successfully

 

I had the same issue in almost 4 Servers and did the same (OS - win 2012R2)