Joost Sannen Posted January 17, 2022 Share Posted January 17, 2022 (edited) Hi, After upgrading from 13.0 83.29 to 84.10 or 84.11 all our Access Gateways are broken. After a successfull logon and before going to Storefront I see Dutch for "Please select one of the following:" without any visible options. The URL is /logon/LogonPoint/index.html . The same happens with a default theme. Using Firefox it appears that cookies are refused because they are not valid or expired. We use LDAP and RADIUS without nFactor and a reqrite to hide the second password field. Very simple. Might be related to some 'fixes' with SameSite cookie attribute? Is anyone else experiencing the same problem? UPDATE: Problem is also present in all current versions of 13.0 and 13.1 Edited June 8, 2022 by Joost Sannen typo Link to comment Share on other sites More sharing options...
Julian Jakob Posted January 18, 2022 Share Posted January 18, 2022 Are you able to share your bound session policy expressions? Link to comment Share on other sites More sharing options...
Joost Sannen Posted January 19, 2022 Author Share Posted January 19, 2022 Thank you for responding. I'm still using classic policies. nonmobile expression = REQ.HTTP.HEADER User-Agent NOTCONTAINS CitrixReceiver mobile expression = REQ.HTTP.HEADER User-Agent CONTAINS CitrixReceiver Link to comment Share on other sites More sharing options...
Patrick Bahne Posted January 20, 2022 Share Posted January 20, 2022 Hell everybody. i've the same error in our enviroment. Link to comment Share on other sites More sharing options...
Joost Sannen Posted January 20, 2022 Author Share Posted January 20, 2022 Hmm, time to raise a ticket. I'll let you guys know when I've significant information. Link to comment Share on other sites More sharing options...
Sjoerd Dijsselbloem Posted January 20, 2022 Share Posted January 20, 2022 Hello everybody. i've the same error in our enviroment. Any info / updates: please share Link to comment Share on other sites More sharing options...
Joost Sannen Posted January 24, 2022 Author Share Posted January 24, 2022 Suggestions Citrix did was to use advanced session policies https://docs.citrix.com/en-us/citrix-gateway/current-release/vpn-user-config/configure-gateway-session-policies-for-storefront.html#create-a-session-policy-for-web-browser-based-access and to Citrix Gateway -> Global Settings -> Under Authentication settings -> Change AAA Authentication settings disable static caching. Both produce the same result. Waiting for a GTM now. Link to comment Share on other sites More sharing options...
Ben Brayshaw Posted January 28, 2022 Share Posted January 28, 2022 In our case we had to change the Theme from RfWebUI to Default as a workaround. Link to comment Share on other sites More sharing options...
Joost Sannen Posted January 28, 2022 Author Share Posted January 28, 2022 Thanks for letting know Ben. Citrix is stepping away from all themes except RfWebUI. That's one of the reasons we changed all our customized Access Gateways last year to use RfWebUI. I hope there will be a solution soon. Link to comment Share on other sites More sharing options...
Ben Brayshaw Posted January 31, 2022 Share Posted January 31, 2022 We did the same. Support mentioned that this will be fixed with the new firmware release, no ETA but I reckon sometime in Feb. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 4, 2022 Author Share Posted February 4, 2022 I rebuild our test setup on a HA pair of VPX on 13.1 12.51 but the problem remains. A remote session on Monday may shed some light on this. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 4, 2022 Author Share Posted February 4, 2022 (edited) Found in debug logging on the ADC with the Access Gateway -ServerPort 443 - ProtocolVersion TLSv1.1 - CipherSuite "NA" - Session New - Reason "Handshake failure-Internal Error" Event: SSL_HANDSHAKE_FAILURE [EDIT] In packet tracing you see ADC to Storefront 'Client Hello' Storefront to ADC 'Fatal, Handshake Failure (40)' Storefront connection reset (RST, ACK). ADC to Storefront connection reset (RST, ACK). Storefront is load balanced on another HA pair VPX. Strange is the setup has been working for ages and now Frontend ADC 13.0 83.29 - Backend ADC 13.0.84.10 = OK Frontend ADC 13.0 84.10 - Backend ADC 13.0.84.10 = ERROR Frontend ADC 13.0 84.11 - Backend ADC 13.0.84.10 = ERROR Frontend ADC 13.1 12.51 - Backend ADC 13.0.84.10 = ERROR Diving into ciphers now. Edited February 5, 2022 by Joost Sannen more info Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 7, 2022 Author Share Posted February 7, 2022 Ok, never mind the ciphers. It does work when I change the Web Interface Address in the Session profile from https to http. Changing it from http to https was one of steps to resolve the problem in an earlier stage. So I wanted to reproduce the original problem of "Please select one of the following:" without any visible options again. I restored the config of our test ADC to the time of the initial problem and ... it works! Only change we are aware of is that for another problem with Citrix ADM we upgraded the ADC with the same build 13.0 84.11 again. Maybe one or more files only got overwritten at the second attempt. Link to comment Share on other sites More sharing options...
Sjoerd Dijsselbloem Posted February 7, 2022 Share Posted February 7, 2022 5 hours ago, Joost Sannen said: Ok, never mind the ciphers. It does work when I change the Web Interface Address in the Session profile from https to http. Changing it from http to https was one of steps to resolve the problem in an earlier stage. So I wanted to reproduce the original problem of "Please select one of the following:" without any visible options again. I restored the config of our test ADC to the time of the initial problem and ... it works! Only change we are aware of is that for another problem with Citrix ADM we upgraded the ADC with the same build 13.0 84.11 again. Maybe one or more files only got overwritten at the second attempt. Hi Joost, just to make sure we understand. In the first upgrade you ran into this problem. Then you installed the same version again, and the second time you dont have this issue? So just install it a second time and all is good? Did anyone have the same experience? I dont have a lab to test this without messing with our production MPX. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 7, 2022 Author Share Posted February 7, 2022 1 minute ago, Sjoerd Dijsselbloem said: Hi Joost, just to make sure we understand. In the first upgrade you ran into this problem. Then you installed the same version again, and the second time you dont have this issue? So just install it a second time and all is good? Did anyone have the same experience? I dont have a lab to test this without messing with our production MPX. Hi Sjoerd, yes that's the only explanation I have. Would be great when someone can test this. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 9, 2022 Author Share Posted February 9, 2022 After consulting Citrix I did another attempt and tested both ADC's of the HA pair afterwards. I can confirm the Access Gateways are working this time. Running version 13.0 84.11 now. Due to lack of any other changes in the chain from internet to Storefront this must be some file(s) on the ADC didn't got overwritten at my first attempt. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 10, 2022 Author Share Posted February 10, 2022 9 hours ago, Joost Sannen said: After consulting Citrix I did another attempt and tested both ADC's of the HA pair afterwards. I can confirm the Access Gateways are working this time. Running version 13.0 84.11 now. Due to lack of any other changes in the chain from internet to Storefront this must be some file(s) on the ADC didn't got overwritten at my first attempt. Argghh..... Unfortunately the problem is still present for some users. It's not a client problem. Windows is fully patched, browser is up-to-date as is the Workspace app. Went back to 13.0 83.29 to let all users use the published resources. Link to comment Share on other sites More sharing options...
Joost Sannen Posted February 14, 2022 Author Share Posted February 14, 2022 Hej guys who are experiencing the same problem: Can you confirm you're using “set ssl vserver xxxx -HSTS ENABLED -maxage 157680000” too? This time it seems I can't reproduce the issue after disabling this setting and doing the same with a rewrite policy like CTX205221. NStrace files uploaded to Citrix for investigation. Link to comment Share on other sites More sharing options...
Josh Bernstein1709161934 Posted March 7, 2022 Share Posted March 7, 2022 On 2/14/2022 at 4:54 AM, Joost Sannen said: Hej guys who are experiencing the same problem: Can you confirm you're using “set ssl vserver xxxx -HSTS ENABLED -maxage 157680000” too? This time it seems I can't reproduce the issue after disabling this setting and doing the same with a rewrite policy like CTX205221. NStrace files uploaded to Citrix for investigation. I ran into this same issue when using that setting. Link to comment Share on other sites More sharing options...
Ben Brayshaw Posted March 25, 2022 Share Posted March 25, 2022 Has anyone tried to see if the new firmware build for 13.0 85.15 has fixed this issue? It's in the release notes as a fixed issue. Link to comment Share on other sites More sharing options...
Joost Sannen Posted March 25, 2022 Author Share Posted March 25, 2022 In firmware 13.0 85.15 the problem still exists. Still waiting for Citrix. I did set ssl vserver xxxx -HSTS DISABLED add rewrite action REW_ACT_STS insert_http_header Strict-Transport-Security "\"max-age=157680000\"" add rewrite policy REW_POL_STS true REW_ACT_STS bind lb vserver xxxx -policyName REW_POL_STS -priority 100 -gotoPriorityExpression END -type RESPONSE for now and that works. 2 Link to comment Share on other sites More sharing options...
Julian Jakob Posted April 1, 2022 Share Posted April 1, 2022 On 3/25/2022 at 2:35 AM, Ben Brayshaw said: Has anyone tried to see if the new firmware build for 13.0 85.15 has fixed this issue? It's in the release notes as a fixed issue. I am not able to find this as fixed issue in the release notes https://docs.citrix.com/en-us/citrix-adc/downloads/release-notes-13-0-85-15.html Are you able to copy the NSHELP ID? Thanks! Link to comment Share on other sites More sharing options...
Denis Nielsen1709162219 Posted April 1, 2022 Share Posted April 1, 2022 On 3/25/2022 at 2:35 AM, Ben Brayshaw said: Has anyone tried to see if the new firmware build for 13.0 85.15 has fixed this issue? It's in the release notes as a fixed issue. I still have the problem with the 13.0 85.15 But works if HSTS is diabled in ssl profile Link to comment Share on other sites More sharing options...
Tom Swift Posted April 29, 2022 Share Posted April 29, 2022 We've started having the same problem with the newer firmware versions. The temporary workaround has been to disable HSTS. We deploy a lot of Citrix Netscaler (ADC)'s and have had it all nicely scripted for years now so we get the SSL Labs A+ rating. Disabling HSTS drops us down to an A rating, but for the time being it's all we can do. Bummer, built a new Netscaler using 13.1 build 21.50 this week and the issue still exists. > show ver NetScaler NS13.1: Build 21.50.nc, Date: Apr 7 2022, 00:18:35 (64-bit) Done #VirtualServer is name of Virtual Server used for Citrix Gateway set ssl vserver VirtualServer -ssl3 DISABLED -tls1 DISABLED -tls11 DISABLED -tls12 ENABLED -tls13 ENABLED -HSTS DISABLED -maxage 157680000 set ssl parameter -denySSLReneg FRONTEND_CLIENT add ssl cipher SecureCiphers bind ssl cipher SecureCiphers -cipherName TLS1.3-AES256-GCM-SHA384 -cipherPriority 1 bind ssl cipher SecureCiphers -cipherName TLS1.3-AES128-GCM-SHA256 -cipherPriority 2 bind ssl cipher SecureCiphers -cipherName TLS1.3-CHACHA20-POLY1305-SHA256 -cipherPriority 3 bind ssl cipher SecureCiphers -cipherName TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256 -cipherPriority 4 bind ssl cipher SecureCiphers -cipherName TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384 -cipherPriority 5 bind ssl cipher SecureCiphers -cipherName TLS1.2-ECDHE-RSA-AES128-GCM-SHA256 -cipherPriority 6 bind ssl cipher SecureCiphers -cipherName TLS1.2-ECDHE-RSA-AES256-GCM-SHA384 -cipherPriority 7 bind ssl vserver VirtualServer -certkeyName SSL Glad I found others with the same problem. Like I said, we've been doing this for years and have had it nicely scripted and it only broke recently. Too bad a mature product is running in to unexpected bugs. Previously used: set ssl vserver VirtualServer -ssl3 DISABLED -tls1 DISABLED -tls11 DISABLED -tls12 ENABLED -tls13 ENABLED -HSTS ENABLED -maxage 157680000 Link to comment Share on other sites More sharing options...
Kyle Peterson Posted May 18, 2022 Share Posted May 18, 2022 Same problem here, Just tried 21.50 as well and same issue. Thanks for the info! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now