SSL bindings disappear after Citrix ADC Release (Feature Phase) 14.1 Build 8.50 upgrade

[Warren Simondson]

After the big security alert (https://support.citrix.com/article/CTX579459/netscaler-adc-and-netscaler-gateway-security-bulletin-for-cve20234966-and-cve20234967) naturally we upgraded our Citrix ADC Release (Feature Phase) 14.1 Build 4.42 to Citrix ADC Release (Feature Phase) 14.1 Build 8.50 upgrade. After several attempts at upgrading via the GUI, which got stuck part way through the install, we had to install the update via CLI. We restored the original VM appliance from just before we began the 8.50 build, to ensure the stuck install did not affect the CLI install. On success of the 8.50 install, and being presented with a new Netscaler logon page, SSL certificates had disappeared and bindings were broken on Services/ Virtual servers. Essentially we had to re-add each and every SSL cert/ keypair. VERY frustrating.  Sadly I have done this on 10 different customer sites of different NS/ ADC appliances and all exposed the same issue.

 

And of course the license goes back to a Freemium, which was another fun experience.

[CarlStalhood]

If you look at the contents of the license file in /nsconfig/license, is the date still in the future?

 

Once the appliance is licensed correctly, you can run the following to get the config back.

 

batch -filename /nsconfig/ns.conf

[Warren Simondson]

So ultimately the license file is the issue, even though it is valid. To avoid this issue, the best approach I found was to:

1. Download the license file from mycitrix for the existing Netscaler appliance on Firmware 14.1 Build 4.42 . No need to reallocate the license just download the license file again fresh from the site.

2. Backup the full config and backup the virtual appliance.

3. Logon to the Netscaler, choose configuration-->system-->licenses-->manage licenses - delete the existing license file, but DO NOT reboot and stay on the existing screen.

4. Add new license file -->upload license file--> browse to the new license file you downloaded earlier. Once you see the license file accepted --> reboot.

5. After reboot check the appliance license is correct and not saying freemium. It should be good.

6. Now start the upgrade process to 14.1 Build 8.50. I still found the CLI way went without a hitch - See details at https://support.citrix.com/article/CTX549397/how-to-upgrade-citrix-adc-stepbystep

7. Reboot once complete and the Netscaler will upgrade and working as expected.

9. Take another backup

Every other direction took so many BAD paths, especially in a HA pair upgrade.

It still stuns me that a rushed security update had more focus on the rebranding of Netscaler, than ensuring the upgrade worked in a normal upgrade path.

Do better Citrix and think about those that have to follow change control.

[FUNDY MUTUAL]

Warren - that is the method I found too. But I also discovered at some point along the way, my licensing host ID has updated from hostname to mac address on some of the older installations (I have something like 20 Netscalers).  The only difference between your steps and mine is that I generally shut down the Netscalers and take a powered off snapshot (they are all VMs) before I start, and I don't bother with the backups as per se.  And instead of using the GUI for any of this, I do it all via scripted CLI (and upload the new license file via SFTP to /nsconfig/license).

 

dcc

[Ricardo Olazabal]

I have the same Issue, but I can't get the NS to take my licenses. I followed @Warren's steps but Fremium still appears.
Anything else that can be done?

Best regards

 

[Warren Simondson]

Have you tried to used WinSCP or SFTP to remove old license files and upload the new license file to /nsconfig/license