Steven Wright

Overview Citrix Application Delivery Management (ADM) is a centralised management solution that enables you to operate, upgrade, report on, and troubleshoot all your NetScaler's across a global estate. One of Citrix ADM's best features is StyleBooks - declarative descriptions of a desired state written in YAML and consisting of input parameters and components driven by those inputs. A StyleBook causes Citrix ADM to read your current NetScaler configuration, compute the differences between your current and target state, and implement the required settings. While you can write StyleBooks yourself, Citrix ADM comes with more than 35 default StyleBooks, covering everything from building a simple HTTP load balancer to all of the content switching, load balancing, and authentication settings needed to support Exchange 2016. Learn about using default StyleBooks and creating custom StyleBooks. In this blog post, I focus on simple HTTP load balancers but provide enough detail so you can apply the logic to any StyleBook. Specifically, I'm going to focus on a StyleBook called "lb" with the namespace "com.citrix.adc.stylebooks"which you can find by selecting Applications, Configuration, StyleBooks within Citrix ADM. This post is the first in a two-part series that describes how to rapidly create services on your NetScaler's using Citrix ADM StyleBook automation, accelerating your migrations. We'll create 300 load balancers across an estate of several NetScaler's within 15 minutes. This post is aimed at DevOps employees who want to start working with Citrix ADM and need a brief introduction with examples (which you can download here). In my second post, I target sysadmins and focus on using preprepared scripts to achieve our goal of creating the 300 load balancers. Citrix ADM and the NITRO API The second best thing about Citrix ADM is that you can access it using API calls, which is how we'e going to automate our StyleBook. You can see more on that below but, you can also find more information here: Citrix ADM 13.1 NITRO API Reference documentation Using APIs to create configurations from StyleBooks To call your script to log in using the NITRO API, you need an ID and secret. You can create the ID and secret for Citrix ADM service by doing the following steps: Sign in to the Citrix Cloud portal. Select Identity and Access Management from the menu. Select the API Access tab. Name your Secure Client, and click Create Client. NITRO API request for a session ID After you have the ID and secret, you can sign in and collect a session ID using the following Python commands (this isn't my real ID and secret): Python import http.client import json id = "f20e7432-7a83-416d-a171-9347b92761f4" secret = "7-rotB6QMFZTuOgpPajPBg==" conn = http.client.HTTPSConnection("adm.cloud.com") headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': '' } request = { "login": { "ID": id, "Secret": secret } } payload = json.dumps(request) conn.request("POST", "/nitro/v2/config/login", payload, headers) response = conn.getresponse() data = response.read() if response.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) jsondata = json.loads(data) sessionid = jsondata["login"][0]['sessionid'] print (sessionid) Output ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM NITRO API request using the session ID to retrieve a list of NetScaler identifiers Now that you have a session ID token, you can request a list of NetScaler's and find the unique identifier of the instance on which you would like to operate. The session ID is taken from the preceding script. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" conn = http.client.HTTPSConnection("adm.cloud.com") headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn.request("GET", "/nitro/v2/config/managed_device", None, headers) response = conn.getresponse() data = response.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) jsondata = json.loads(data) #print('\n\nHere is the full JSON output from ADM.\n') #print (jsondata["managed_device"]) print('\n\nHere is a list of ADCs.\n') for device in jsondata["managed_device"]: name = device['hostname'] ip = device['ip_address'] id = device['id'] type = device['type'] version = device['version'] status = device['status'] if status == "Success": print (name + " " + ip + " " + id + " " + type) Output Here is a list of ADCs. myhostname 192.168.1.240 a1fe50be-e187-44bb-871b-d60f112b0aff nsvpx Knowing the parameters to submit to a StyleBook Now that we have a session ID and the identifier of an NetScaler to target, we can run a StyleBook. First, however, we need to understand the various parameters that it expects us to submit before doing so. You can gather the parameters by programmatically requesting the StyleBook's schema, which describes all valid parameters and which are optional, or by reviewing the source YAML for the StyleBook and its dependencies. Here are the Python commands to request a StyleBook's schema. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" base_types = ["string", "ipaddress", "number", "boolean", "tcp-port", "password", "file", "certfile", "keyfile", "certkey", "ipnetwork"] def get_config_parameters(parameters): configpack_params = {} for parameter in parameters: parameter_name = parameter["name"] parameter_type = parameter["type"] parameter_required = parameter["required"] if parameter_type in base_types: if parameter_required: parameter_type += "*" if parameter_type.endswith("[]"): configpack_params[parameter_name] = [parameter_type] else: configpack_params[parameter_name] = parameter_type else: if "parameters" in parameter: subparameters = parameter["parameters"] datatype = get_config_parameters(subparameters) else: datatype = parameter_type if parameter_type.endswith("[]"): configpack_params[parameter_name] = [datatype] else: configpack_params[parameter_name] = datatype return configpack_params conn = http.client.HTTPSConnection("adm.cloud.com") stylebook = "com.citrix.adc.stylebooks/1.1/lb" headers = { 'Cookie': 'NITRO_AUTH_TOKEN= ' + sessionid } conn.request("GET", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/schema", None, headers) res = conn.getresponse() data = res.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) schema = json.loads(data, "utf-8") parameters = schema["schema"]["parameters"] config_parameters = { "parameters": get_config_parameters(parameters) } print(json.dumps(config_parameters)) The output from this schema request is the format of the parameters section of a config pack. From this output, you can remove parameters you don't intend to use. As a clue, the required parameters are marked with a "*" at the end of their type. Another (and possibly easier) way to achieve the same thing is to use the Citrix ADM UI to create a template that shows only the parameters I want to use. To create the template, I open Citrix ADM's web interface and run my selected StyleBook with dummy values against a non-production NetScaler in a lab. In doing so, I create a config pack with each of the values I would need for a production service. Having created the config pack, I now select Applications > Configuration > Config Packs from within the Citrix ADM web interface and export the resultant configuration. After downloading and opening the exported ZIP file, I find that it contains a single JSON file with all the parameters that I need to submit to the API. NITRO API request using the session ID to automate a StyleBook and target an NetScaler identifier Finally, having retrieved a session ID, the unique identifier of the NetScaler to operate on, and clarity into the parameters needed to submit to the StyleBook, we are ready to script execution of the StyleBook itself. Note: Delete the template configuration pack from Citrix ADM if you want to use the same input parameters within your Python script. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" target_adc = "a1fe50be-e187-44bb-871b-d60f112b0aff" stylebook = "com.citrix.adc.stylebooks/1.1/lb" conn = http.client.HTTPSConnection("adm.cloud.com") payload = json.dumps({ "configpack": { "targets": [ { "id": target_adc } ], "parameters": { "lb-appname": "ApplicationName", "lb-virtual-ip": "192.168.50.10", "lb-virtual-port": "80", "lb-service-type": "HTTP", "svc-service-type": "HTTP", "svc-servers": [ { "ip": "192.168.5.20", "port": 80, "add-server": True }, { "ip": "192.168.5.21", "port": 80, "add-server": True } ], } } }) headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn.request("POST", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/configpacks", payload, headers) res = conn.getresponse() data = res.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) payload = json.loads(data, "utf-8") jobid = payload["configpack"]["job_id"] print("Configuration Job " + jobid + " has started.") Output Configuration Job 204770189 has started. Viewing a job_id At this point you are able to see the config pack that your script created within the ADM GUI and the resultant configuration on your NetScaler. If that didn't happen, you can view the details of the "job_id" and understand its status. We can retrieve the outcome of the job by sending a GET request. To illustrate, I purposely ran another StyleBook targeted at a non-existent NetScaler ID. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" jobid = "2476087615" headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn = http.client.HTTPSConnection("adm.cloud.com") job_status = None while job_status != "completed" and job_status != "failed": conn.request("GET", "/stylebook/nitro/v1/config/jobs/" + jobid, None, headers) response = conn.getresponse() data = response.read() if response.status != 202: print("Error:") print(" Status code: " + str(response.status)) print(" Error payload: " + data) exit(1) payload = json.loads(data, "utf-8") job_status = payload["job"]["status"] config_id = payload["job"]["progress_info"][0]["id"] if job_status == "completed": print("Success: configuration " + config_id + " successully created") else: print("Error: Failed to create configuration.") progress_info = payload["job"]["progress_info"] for step in progress_info: error_details = step["message"] print(" " + json.dumps(error_details)) exit(1) Output (failure) Error: Failed to create configuration. "Validating the parameters" "Configuration pack process failed. Instance with id g4702445-2286-4b6d-8fb1-c231a0ee17b1 not found on the system." Here, we can observe a clear explanation of the issue. We can observe success messages on jobs that run without issue. Output (success) Success: configuration 204770189 successfully created Next steps We have now seen how we can programmatically authenticate to Citrix ADM, retrieve a list of identifiers for the NetScaler's that it manages, prepare and run StyleBooks against a chosen NetScaler, and retrieve the result. We have also observed how you can find the namespace, version, and name of alternative StyleBooks and retrieve the various properties they require. Using this knowledge, your DevOps team is able to integrate Citrix ADM StyleBooks and their automated NetScaler configuration into your existing workflows. In the second of this two-part series, using the example of creating 300 load balancers across an estate of several NetScaler's within 15 minutes, I share prebuilt scripts that allow those without a DevOps team to create business services rapidly during a platform migration.

In this two-part blog series, I describe how to rapidly create services on your NetScaler's using Citrix Application Delivery Management (ADM) StyleBook automation to accelerate migrations. In the example, we create 300 load balancers across an estate of several NetScaler's within 15 minutes. My first post was aimed at DevOps. In this post, I target Sysadmin employees and focus on a pre-prepared script that I wrote. The script helps you to achieve our goal of rapidly creating load balancing virtual servers. You can download the script and its two supporting files here. Note: The script requires Python 3 and jinja2, that you can install with pip using the command pip install jinja2 Prepare the StyleBook template("template.txt") The first of the script's two supporting files is named template.txt. The template.txt file is already preprepared for the simple HTTP load balancers that we create in this post, and you don't need to modify it. However, I do want to explain how you can make those modifications, in case you want to target other StyleBooks or introduce extra parameters. On the second line of the template.txt, you can find the namespace, version, and the name of the targeted StyleBook. StyleBook NAMESPACE/VERSION/STYLEBOOK com.citrix.adc.stylebooks/1.1/lb If you want to change the template file's target, you can view the namespace, version, and the name of another StyleBook within Citrix ADM's web interface. On lines five to 12 of the template.txt, you can find placeholder names that correspond to the columns in a CSV file called lbvserver.csv, which is also included with the script. CSV columns targetadc name lbvirtualip lbvirtualport lbservicetype svcservicetype svcservers: list svcserverport As you can see from the placeholder named svcservers: list, if the columns in the CSV file contain a list of items, such as a list of back-end servers for a load balancing virtual servers, we note this by appending ": list" to the placeholder's name. You can find a template with each value that our script must submit to the StyleBook at line 14. Each item in the template matches one of the placeholder values you created earlier, and the template includes a loop where it encounters a list. Template { "configpack": { "targets": [{"id": "<<targetadc>>"}], "parameters": { "lb-appname": "<<name>>", "lb-virtual-ip": "<<lbvirtualip>>", "lb-virtual-port": "<<lbvirtualport>>", "lb-service-type": "<<lbservicetype>>", "svc-service-type": "<<svcservicetype>>", "svc-servers": [ <% for server in svcservers %> { "port": << svcserverport >>, "ip": "<< server >>", "add-server": true }<<,>> <% endfor %> ] } } } Again, you need to edit the template file only you're modifying the script to target an alternative StyleBook or you're introducing extra parameters. Preparing the script As we saw in Part 1, you need an ID and secret to allow the script to log in into your Citrix ADM environment. If you're using the Citrix ADM service, you can create the ID and secret by doing the following steps: Sign in to the Citrix Cloud portal. Select Identity and Access Management from the menu. Select the API Access tab. Name your Secure Client, and click Create Client. After you have the ID and secret, replace the placeholder example values within the create2.py python script. # Set my login variables for ADM # You can create an ID and secret by selecting the Identity and Access Management tab in your Cloud portal Id = "1707d2ff-e200-46be-848a-93faf2076d6b" Secret = "DIH1CzF_GSZG2zNnEVShLA=="` Preparing the Excel CSV template Now that you've set the ID and secret values, you can run the script with the "-a" parameter to display a list of NetScaler's and their identifiers. Command python create2.py -a Output The following NetScaler instances IDs can be referenced within your CSV file. Name - IP address - ID for use in CSV. 1\. bob - 192.168.1.240 - a0de40bb-e186-44bb-871b-d60f112b0a3a Here, we can observe that I have an NetScaler with the host name "bob" on IP address "192.168.1.240" with the unique identifier "a0de40bb-e186-44bb-871b-d60f112b0a3a." You must note each identifier because we need them later. Prepare a list of load balancing virtual server ("lbvserver.csv") The second of the script's two supporting files is named lbvserver.csv. Each row in the lbvserver.csv file includes the properties for a single StyleBook, in this case, the properties for a single HTTP Load Balancing virtual servers. Each column in the lbvserver.csv file is the source for the placeholders within the template.txt file. The columns and placeholders are in the same order, with the first column being the first placeholder and so on. I have expanded the lbvserver.csv file to contain 300 rows (300 HTTP Load Balancing virtual servers) for this post. If my customer had a similar requirement, I would likely populate an initial CSV with all the expected values. Then, they would forward the CSV internally to ensure that the names, naming conventions, and IP addresses were all as desired. As you can observe by comparing the columns to the template.txt file, column A specifies the unique identifier of the NetScaler that we target, and columns B to H are the input parameters for the StyleBook. You can include lists in the parameters with ":" used as a separator. After modifying the lbvserver.csv file to reflect the virtual servers that you want to create, you must save in "CSV" rather than "CSV (UTF8)" format. Running the script (python create2.py) Having populated the script with an ID and secret that allow access to your Citrix ADM environment and updated the columns to reflect which NetScaler you want to build on and the properties, we are now ready to create our load balancing virtual servers. We can now run the script and create a Citrix ADM config pack along with the resultant NetScaler configuration. Command python create2.py Output (shortened) Creating ADM Config Pack based on CSV row: 1 {"configpack": {"job_id": "1926702141"}} Creating ADM Config Pack based on CSV row: 2 {"configpack": {"job_id": "2520889571"}} Creating ADM Config Pack based on CSV row: 3 {"configpack": {"job_id": "2581540139"}} Creating ADM Config Pack based on CSV row: 4 {"configpack": {"job_id": "1440135035"}} Creating ADM Config Pack based on CSV row: 5 {"configpack": {"job_id": "3264746514"}} The script takes around two to three seconds per row and completes the 300 rows in just under 15 minutes. After completion, the resultant Config Packs will be visible within ADM. Ongoing maintenance In addition to creating the virtual servers rapidly from a pre-agreed Excel CSV file, this script-based approach also allows management of the resultant virtual servers using ADM and future changes without a need to understand the underlying commands on their NetScaler's. You can view each of the resultant Config Packs by selecting Applications, Configuration, Config Packs from within the Citrix ADM web interface. By selecting Edit, you can then manage and update the Config Pack as your needs change. Next steps We have now created 300 load balancers across our estate using an adaptable template that allows you not only to create load balancers but any other StyleBook based configuration. If you want to understand how the script works, you can read through the first article in this series .

NetScaler migration Have you bought a new NetScaler with more throughput or is it time to refresh? Are you migrating from old NetScalers to new ones? Sounds difficult? Then you've stumbled across the right article. Installing new NetScalers into production and migrating services may seem like a complicated process, but once you know how to do it, it's much easier than you might think. This blog post will walk you through the steps needed. In this article, we will assume that you have an existing source NetScaler that's running in production and a new target NetScaler that you want to prepare in a non-production environment and migrate to under a change control process. Prepare for the migration (existing NetScaler) ADM Service's migration feature requires NetScaler firmware version 13.1.17 or above. To prepare for the upcoming migration, upgrade your existing NetScalers to 13.1.17 firmware so that both the source and target NetScaler have the same firmware version and support for ADM Service's migration feature. NetScaler recommends upgrading to the same firmware on the source and target NetScalers to minimize the number of changes during the migration and, in the unlikely event of any issues, to identify the cause more easily. You can upgrade your existing NetScaler to 13.1 firmware using the instructions here. Prepare for the migration (target NetScaler) First, install your new target NetScaler into a non-production environment without a connection to the production network. Ensuring the non-production and production networks are isolated from each other will prevent unintended interactions such as IP address conflicts. Once you have installed the new NetScaler, connect it to ADM Service. To do this, you need to add an ADM Agent into the non-production environment. You can install a new ADM Agent using the instructions here and add you can connect the new NetScaler to ADM Service using these instructions. Next, install a license onto your new NetScaler. You can apply a pooled license to your new NetScaler using the instructions here. Replicate the configuration After ensuring that the new NetScaler is not connected to any production networks, follow these steps to replicate the configuration. 1. Select Infrastructure, Instances, NetScaler, and NetScaler form factor In the screenshot, we have selected the "VPX" form factor and can see two NetScalers, "192.168.6.30" and "192.168.1.230". In this article, we will use "192.168.6.30" as the source NetScaler and will replicate its configuration onto the target NetScaler on "192.168.1.230". 2. Tick the source NetScaler, click "Select Action", and then choose "Replicate Configuration" 3. Move the target NetScaler instance from the "Available" to "Configured" window using the arrow button and click "Create" ADM Service will now replicate the configuration from your source to target NetScaler. Note: Any existing configuration on the target instance will be erased and replaced with the configuration of the source NetScaler. The source and target NetScalers should be on separate networks to prevent IP conflicts. Validate the network configuration on the target NetScaler Although the source and target NetScalers will now have the same configuration, the management NSIP and network interface layout may differ. If you installed your new NetScaler into the non-production environment using a network interface layout that you don't want to use in production, you should change they layout now. To change the network configuration, log into the target NetScaler and ensure: That all unused interfaces are disabled That you have created LACP channels as you require That a SNIP is bound to each VLAN That each VLAN is bound to an interface or channel using your intended network layout You can find instructions detailing how to disable unused interfaces, create LACP channels, and how to bind SNIPs and VLANs here. Replace the NSIP on the target NetScaler If you installed your new NetScaler into the non-production environment using an NSIP that differs from production and would prefer it was the same at the point of migration, you should change the NSIP now. You can change the NSIP using the following steps. Note: If you are migrating an HA pair, you must perform the commands below on both nodes at the same time and shutdown both nodes at the same time. Select the target NetScaler within the "Infrastructure, Instances, NetScaler" menu of ADM, choose "SNMP" from "Select Action", then disable SNMP. Note: We are disabling these services to ensure they are not configured to communicate with the pre-production ADM Agent, we will re-enable them after the migration is complete. Using the same process as "step 1" (above), disable Syslog and Analytics on the target NetScaler. Select the source NetScaler and repeat steps one and two (above) to disable SNMP, Syslog, and Analytics. Log into the target NetScaler's command line interface using SSH. You can find details of how to use SSH here. Change the NSIP of the target NetScaler using the commands shown in red. shell cd /nsconfig cp ns.conf ns.old sed -i -e 's/192.168.1.230/192.168.6.30/g' ns.conf Note: 192.168.6.30 is the source NetScaler in our example and 192.168.1.230 is the target NetScaler. You must replace the IP addresses within the command with those of your source and target NetScaler. Change references to the NSIP of the other HA node if applicable sed -i -e 's/192.168.1.231/192.168.6.31/g' ns.conf Note: You will run the sed command twice on both HA nodes. Firstly, to replace the NSIP of the current node. Secondly, to replace all references to that changed NSIP on its HA partner. Shutdown the NetScaler shutdown now Note: If you shutdown the NetScaler using an alternative command then you should reply "no" when ask if you wish to save the running configuration. The output will look like the text below. Note: In this example, we have included an example HA node with an NSIP of 192.168.1.231 that needs to be replaced with 192.168.6.31 - the replacement needs to be executed on both HA nodes. ############################################################################### # # # WARNING: Access to this system is for authorized users only # # Disconnect IMMEDIATELY if you are not an authorized user! # # # ############################################################################### Password: Last login: Mon Dec 6 14:26:05 2021 from 192.168.6.20 Done nsroot@-Primary> shell Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. root@ns# cd /nsconfig root@ns# cp ns.conf ns.old root@ns# sed -i -e 's/192.168.1.230/192.168.6.30/g' ns.conf root@ns# sed -i -e 's/192.168.1.231/192.168.6.31/g' ns.conf root@ns# shutdown now Shutdown NOW! shutdown: [pid 8140] root@ns# *** FINAL System shutdown message from root@ns *** System going down IMMEDIATELY System shutdown time has arrived Connection to 192.168.6.30 closed by remote host. Replace the NetScaler under change control With the existing source and target NetScalers now having identical configurations, you can replace them in compliance with your change control procedures. Many customers achieve that change using a two-stage replacement process. In the first stage, they work with their network teams to connect the new NetScalers to network switch ports that are administratively disabled or offline. In the second stage, during a change control window, the network team places the switch ports to the old NetScalers into a disabled or offline state, brings the interfaces connecting the new NetScalers online, and flushes the ARP cache on their router. The effect is that all hardware installation and readiness activities can be completed in-advance and the change control window to move from the old to new appliances is completed in minutes. This approach also provides an excellent rollback strategy as the network could disable the interfaces connecting the new NetScaler and reenable those for the old, which are still powered on and running without modification. Re-establish connectivity to ADM Service You have now replaced the source NetScaler with the target. ADM will show the source as online (as the target is using the source NetScaler's NSIP) and the target's old NSIP as offline. You should now complete the following steps to ensure the migrated NetScaler is correctly communicating with ADM Service. To ensure ADM is communicating with the new NetScaler that has taken over the NSIPs of the source, select the source NetScaler within "Infrastructure, Instances, NetScaler" menu of ADM and, choose "Rediscover" from "Select Action". After the rediscovery completes, re-enable the SNMP, Analytics, or Syslog capabilities that you disabled at step one of "Replace the NSIP on the target NetScaler". To enable each, select the NSIP of the source NetScaler and the capability under "Select Action". If you are using pooled licensing on the new NetScaler, you must also ensure it is using the production ADM Agent as its license server. To do this, log into the new NetScaler using SSH and enter the following commands: sh nslicenseserver If the new NetScaler is not using the production ADM Agent as its license server, you can correct the license server to production ADM using the following commands: add nslicenseserver <prod_agent_ip> -port 27000 -forceUpdateIP Remove the target NetScaler's old NSIP from ADM by ticking the old NSIP within the "Infrastructure, Instances, NetScaler" menu of ADM and pressing "Remove". Next steps You have now finished migrating from your old to new NetScalers and can begin using them in production.

NetScalers are used by some of the world's largest service providers to handle inbound application traffic. In this article, I will show how you can deploy an NetScaler environment that will scale to millions of HTTP requests and SSL handshakes per second. In the next, I will show you how to automate that environment with a basic introduction to Terraform. The suggested architecture will allow for multiple datacenters. In addition, it will provide load balancing, SSL offload, and the capability to secure and optimize your web application traffic. The architecture will also be scalable and fully automatable. We will begin by reviewing technologies to get traffic to your datacenter, how to distribute that traffic across NetScalers once it arrives, and how you can configure your NetScalers with code using in-house deployment scripts. Assuming you have sized the rest of your environment appropriately, this design should allow you to process around 100 million HTTP requests per second at each datacenter. However, the layout is scalable and can be used for smaller implementations without significant changes. Directing users to their most appropriate datacenter Historically, there have been two approaches to directing users to their closest datacenter: dynamic routing (anycast) and GSLB. Dynamic routing (anycast) With the dynamic routing / anycast approach, the routers at each datacenter announce the same IP ranges and rely on the Internet's routing protocol (BGP) to take the shortest path. The approach operates on the assumption that a British user will have a shorter path to your British datacenter and your American user to an American datacenter. The method also assumes that a short path will be the best route. The concept is illustrated in simplistic terms in the diagram below. Users, sending traffic to their ISP, will have that traffic forwarded to the datacenter that is the smallest number of hops away. A standardized repeatable design at each datacenter With traffic now arriving at the most appropriate datacenter, we can focus on handling the resultant traffic. We will design each datacenter to operate with a standardized design so that it's simple and easy to update and maintain. We will use physical hardware NetScalers known as "MPXs" for their high performance. However, our design is equally suitable for virtual NetScalers "VPX" if you favor a complete infrastructure-as-code approach. In our design, traffic (directed by CADS) will enter the network at the router/firewall. The router will then distribute these inbound connections to our NetScalers. Next, the NetScalers will terminate the connections, process the requests, and deliver them to backend servers. Finally, the servers will service the requests, potentially using local data or a storage solution replicated between your datacenters. All of the configuration will be automated using Terraform (covered in the next article). However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as: Ansible Citrix ADM Service Nitro REST API to allow for custom scripts. We assume you have appropriately sized your routers, firewalls, and storage. And you have ensured your storage is replicated between datacenters if your servers so require. This article shows a single router as a simplification. In our dynamic routing configuration, each NetScalers "announces" the IP addresses of the virtual servers (vServers) they host to the router. As each NetScaler will host vServers with the same IP addresses, the router will distribute the inbound connections between the NetScalers. On a Cisco router, the distribution of traffic between OSPF neighbors will likely occur in hardware using a technology known as CEF, Cisco Express Forwarding, to reduce the impact on CPU. You must take care to ensure your router has sufficient resources to handle the distribution. Additionally, to ensure all packets from a user are directed to the same NetScaler, you should consult your routers manual to ensure it is using per-destination load-sharing, which is the default on a Cisco router. To implement our dynamic routing configuration, we first need to configure our router to have an OSFP neighbor relationship with the NetScalers. On a Cisco router, a very simplistic OSPF configuration would look as follows: enable router# config t router(config)# router ospf 100 router(config-router)# network 192.168.1.0 0.0.0.255 area 0 router(config-router)# maximum-paths 16 router(config-router)# end After we enter the "network 192.168.1.0 0.0.0.255 area 0" command, our router will enable OSPF on all of its interface addresses within that network range. Next, on our NetScalers, we need to enable the OSPF routing facility with the command "enable ns feature ospf".After enabling OSPF, we can configure the NetScaler's included routing platform known as "ZebOS", by entering the command "vtysh". The ZebOS configuration on each NetScaler would look as follows: NetScaler# config terminal NetScaler(config)# router ospf NetScaler(config-router)# network 192.168.1.0/24 area 0 NetScaler(config-router)# redistribute kernel With the OSPF configuration created, we can tell each NetScaler to advertise the IP address of a vServer (such Load Balancing vServer) using the following command: set ns ip 10.0.0.10 255.255.255.255 -hostroute ENABLED -vserverRHILevel ALL_VSERVERS This command will tell the NetScaler to announce the IP address 10.0.0.10 to the router. With each of our NetScalers now announcing itself to our router as a path to the Load Balancing vServer, the router will distribute inbound traffic to each of the NetScalers. After enabling OSPF, you can validate communication between the NetScalers and router using the following commands on the router. Here, we can see three NetScalers on "192.168.1.10", "1.11", and "1.12" advertising the LB vServer on "10.0.0.10". R1# sh ip ospf | s 10.0.0.10 > 10.0.0.10/32, Intra, cost 1, area 0 via 192.168.1.10, GigabitEthernet1.1 via 192.168.1.11, GigabitEthernet1.2 via 192.168.1.12, GigabitEthernet1.3 R1# sh ip cef 10.0.0.1 detail 10.0.0.1/32, epoch 0, per-destination sharing nexthop 192.168.1.10 GigabitEthernet1.1 nexthop 192.168.1.11 GigabitEthernet1.2 nexthop 192.168.1.12 GigabitEthernet1.3 We have used a relatively simplistic configuration with a single router and each NetScaler acting as an OSPF path. As routers are generally limited to 16 OSPF paths, this design will likely be limited to 16 NetScalers. More advanced implementations will be able to expand on this simple design. 16 low-end appliances (such as the MPX9130) will equate to around 33 million L7 HTTP requests per second. Using high-end appliances (such as the MPX26200-50S), this design should service over 100 million L7 requests/sec at each datacenter. Note: You will have multiple routers in a production environment to meet redundancy and throughput requirements and may need to work with your network team to adjust the proposed configuration. Testing To test, our configuration, we will configure each NetScaler to return a different colored web page. The colors will make it immediately apparent which NetScaler is servicing our traffic.

Tech Paper: Networking SSL / TLS Best Practices February 21, 2023 Author: Steven Wright Overview This Tech Paper aims to convey what someone skilled in ADC would configure as a generic implementation to receive an A+ grade at Qualys SSL Labs. Qualys SSL Labs performs a robust series of tests and provides a scorecard that you can use to improve your configuration. The scan is free and only takes about a minute to complete. Qualys actively develops SSL Labs. Tests are likely to change as new protocols are created and vulnerabilities found. It is good practice to test sites regularly to make sure that any new vulnerabilities are not exposed. NOTE: Talk to your security team about your deployment. Security experts say SSL Labs A+ is a good general target, but it may not fit the needs of your organization. Configuration Items that Need to be Validated Certificates - Is the full chain provided and trusted? Is the signature algorithm secure? Protocols, Keys and Cipher Support - Which SSL and TLS protocol versions are supported? Which cipher suites are preferred and in what order? Do the provided cipher suites support forward secrecy? TLS Handshake Simulation - Determines which protocol and cipher are negotiated by several different clients and browsers Protocol Details - Is Secure Renegotiation supported? Is strict transport security (HSTS) supported? Known Vulnerabilities - Is the server vulnerable to attacks such as POODLE, BEAST, or TLS downgrade? Once SSL Labs completes testing, it presents a letter grade along with a point scale for each of 4 categories: 1 Certificate 2 Protocol Support 3 Key Exchange 4 Cipher Strength Each of the categories receives a numerical score that SSL Labs then averages into a total. Some special cases and configurations that SSL Labs recommends against, such as having SSLv3 enabled, can limit your final grade. You can find complete documentation on how SSL Labs grades servers here. Implementation Concerns SSL Profiles This article uses SSL Profiles. When first enabled, SSL Profiles sets all SSL virtual servers to use the default profile. SSL profiles take precedence over global and per virtual server SSL parameters. Client support Some of the configuration steps in this article can cause connectivity issues with old clients and browsers. For example, Internet Explorer 11 on Windows 7 and 8.1 only supports older cipher suites and, other older browsers may lack support for TLS1.2 and ECC ciphers entirely. In cases where support is missing, the client can experience error messages and an inability to display the site. SSL Labs have a “Test your browser” button on its front page to help determine your needs. Citrix Receiver/Workspace app Cipher Support for Gateway deployments Review the following articles regarding client cipher support when deploying a gateway virtual server for virtual apps and desktops: CTX234227 and CTX232266 for Citrix Receiver CTX250104 for Workspace app Note: See the Firmware Notes section for required builds and other notes regarding specific ADC firmware Basic Steps - GUI Take the following steps to ensure a high score on the SSL Labs test. Ensure that the ADC is running a recent firmware release - 13.0 build 71 or later is recommended to take advantage of TLS1.3 hardware acceleration Ensure that the certificate chain is complete and trusted Root CAs do not always directly sign certificates. Instead, a root CA often uses an intermediary to sign a certificate Install the intermediate certificate on the ADC. Link it to the server certificate you bound to the virtual server Intermediate certificates are provided by the vendor that provides the server certificate, often in a ‘certificate bundle’. They can usually be found on the vendor’s public site You may need to install and link multiple intermediate certificates. For the server certificate to function, the ADC must send a complete chain. A complete chain ends with a certificate signed by one of the client’s trusted root CAs As the client already has the trusted root CA, you don’t need to install and link it on the ADC To install an intermediate certificate, go to: Traffic Management > SSL > Certificates > CA Certificates and choose Install (Note: earlier builds of Citrix ADC do not have the ‘CA Certificates’ option in the GUI) Link an intermediate by selecting the certificate and choosing link from the action menu If the correct intermediate certificate is installed, it is automatically populated in the linking menu Create a custom cipher group that provides Forward Secrecy (FS) Go to Traffic Management > SSL > Cipher Groups and choose Add Name the cipher group “SSL_Labs_Cipher_Group_Q4_2021” Click Add then expand the ALL section - select the following cipher suites: TLS1.3-AES256-GCM-SHA384 TLS1.3-AES128-GCM-SHA256 TLS1.3-CHACHA20-POLY1305-SHA256 TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384 TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256 TLS1.2-ECDHE-ECDSA-AES256-SHA384 TLS1.2-ECDHE-RSA-AES256-GCM-SHA384 Click the > right arrow to move the ciphers from the Available column to the Configured column Click Create Enable SSL Profiles Navigate to Traffic Management > SSL > Change advanced SSL settings, scroll down, and select Enable Default Profile. SSL Profiles sets all SSL virtual servers to use the default profile when first enabled. As existing per virtual server SSL settings are removed, the ADC will prompt you to confirm. Create an SSL Profile Navigate to System > Profiles > SSL Profile, and select Add Name the profile “SSL_Labs_Profile_Q4_2021” Scroll to Deny SSL Renegotiation and select NONSECURE to allow only clients that support RFC 5746 to renegotiate Scroll to HSTS, tick HSTS, and specify a Max Age of 157680000 seconds Scroll to Protocol and select only TLSv12 and TLSv13 Scroll to the end of the form and select OK Scroll to SSL Ciphers, select the pencil icon to edit, then click Remove All Click Add and add the cipher group we created earlier Scroll to the end of the form and select Done Bind the SSL Profile to the SSL virtual server On the selected virtual server, select the pencil icon to edit the bound SSL Profile Select the SSL Profile we created from the drop-down list Click OK Basic Steps - CLI Take the following steps to ensure a high score on the SSL Labs test. In the CLI examples below, the name of the SSL virtual server is listed as Ex-vServer - it can be replaced with the name of the SSL virtual server in your environment. Create a custom cipher group that prefers ECDHE and ECDSA cipher suites add ssl cipher SSL_Labs_Cipher_Group_Q4_2021bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-AES256-GCM-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-AES128-GCM-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-CHACHA20-POLY1305-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES256-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-RSA-AES256-GCM-SHA384 Enable SSL Profiles set ssl parameter -defaultProfile ENABLED Create an SSL Profile add ssl profile SSL_Labs_Profile_Q4_2021 -tls1 DISABLED -tls11 DISABLED -tls12 ENABLED -tls13 ENABLED -denySSLReneg NONSECURE -HSTS ENABLE -maxage 157680000 Unbind the default cipher group from the SSL Profile and bind the custom group unbind ssl profile SSL_Labs_Profile_Q4_2021 -cipherName DEFAULTbind ssl profile SSL_Labs_Profile_Q4_2021 -cipherName SSL_Labs_Cipher_Group_Q4_2021 Bind the SSL Profile to the SSL virtual server set ssl vserver "Ex-vServer" -sslProfile SSL_Labs_Profile_Q4_2021 More Settings SHA1 Certificates Certificates that are signed with SHA1 are considered weak, and prevents a high grade in the SSL Labs test. If any certificates are SHA1 signed, they should be replaced with an SHA256 certificate and installed on the ADC. DNS CAA DNS Certification Authority Authorization (CAA) allows CAs to validate if they are authorized to issue certificates for a domain and provide a contact if something goes wrong. DNS CAA is not required for an A+ at SSL Labs and is an optional step. You can find out more about DNS CAA here. Legacy client support The ECDHE ciphers we use in this guide replace the older slower DHE ciphers. If you have legacy clients that you cannot upgrade, you may have no choice but to enable DHE. Enable DHE cipher suites in the GUI Go to Traffic Management > SSL and select Create Diffie-Hellman (DH) key Name the Diffie-Hellman key “DH_Key_Name_Here.key” Enter the parameter size (Bits). Must be between 512 and 2048 Choose the Diffie-Hellman generator (2 or 5) Select Create Depending on the key size selected, this could take quite some time to complete NOTE: Do not navigate past this screen until it completes Navigate to System > Profiles > SSL Profile Select the SSL_Labs_Profile_Q4_2021 profile and choose Edit Select the pencil icon Scroll down to the Enable DH Param check box and select it Choose the DH key you just created Scroll down and select Okay Bind a DHE cipher suite to the cipher group that we created earlier Navigate to Traffic Management > SSL > Cipher Groups Select the TLS1.2-DHE-RSA-AES256-GCM-SHA384 cipher Click the arrow to add it to the Cipher Group Enable DHE ciper suites in the CLI Create and bind a DH key to the SSL Profile (CLI) create ssl dhparam DH_Key_Name_Here.key 2048 -gen 2set ssl profile SSL_Labs_Profile_Q4_2021 -dh ENABLED -dhFile DH_Key_Name_Here.key Bind a DHE cipher suite to the cipher group that we created earlier bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-DHE-RSA-AES256-GCM-SHA384 Firmware Notes Citrix enabled TLS1.2 as a default in firmware version 10.5 build 57 The ROBOT vulnerability was addressed in builds 12.0 build 53, 11.1 build 56, 11.0 build 71 and 10.5 build 67 - more details are available here The HSTS (Strict Transport Security) flag became available in 12.0 build 35 - prior builds required a rewrite policy to insert the HSTS header. You cannot use both as the ADC to insert 2 headers which is not allowed. Support for TLS1.2 was added to the VPX appliances in 10.5 build 57. It was available in earlier builds for appliances with dedicated SSL hardware Support for TLS1.3 was added in 12.1 build 49.23 - it must be enabled in the vServer SSL Parameters or added to an enhanced SSL Profile, and TLS1.3 ciphers (listed) must be bound. TLS1.3 is not supported on legacy SSL profiles ECC certificate support was added to the VPX appliances in 12.0 build 57. It was available in earlier builds for appliances with dedicated SSL hardware The Zombie POODLE vulnerability was addressed in builds 12.1 build 50.31, 12.0 build 60.9, 11.1 build 60.14, 11.0 build 72.17, and 10.5 build 69.5. This vulnerability only affects MPX\SDX appliances with Nitrox SSL hardware. MPX\SDX appliances with Coleto Creek are not vulnerable. Disabling CBC-based cipher suites also mitigates this vulnerability. See CTX article for more information The cipher list has been modified to address CBC weaknesses, thus removing 0xc028 and 0x39 ciphers Citrix added TLS1.3 hardware acceleration in 13.0 build 71

NetScaler Gateway is a powerful tool that provides secure remote access to applications and data. However, the process of setting up and deploying NetScaler Gateway can be complex and time-consuming. To help simplify this process, we are excited to share reference designs covering two of the three most common deployment scenarios. The designs provide a complete high-level and low-level reference for setting up a standard NetScaler Gateway implementation with LDAP+RADIUS or SAML authentication. You can use these designs as a template for your own deployments, to validate your existing configurations, or simply benefit from having access to every CLI command required in a copy/paste format found at the end of each design document. The first reference design covers the setup of NetScaler Gateway with LDAP+RADIUS authentication. It includes every NetScaler object and the CLI command required to deploy a production environment in under 30 minutes. This design is perfect for those who need to secure their applications and data using multi-factor authentication. Our second reference design covers the setup of NetScaler Gateway with SAML authentication using an existing identity provider. We used OKTA when writing the design but, the configuration should be compatible with any SAML IdP. As with our first reference design, every CLI command required is provided in a copy/paste format, ensuring an accurate, production-quality environment can be quickly deployed. These reference designs includes many best practices shared by Professional Services and Engineering. Whether you're a seasoned NetScaler professional or just getting started, we hope you'll find them packed with valuable insights and tips that can help you optimize your deployment. Download our reference articles here: Gateway with LDAP+RADIUS authentication Gateway with SAML Gateway as an OAuth IdP for integration with Citrix Cloud

Tech Paper: Best practices for NetScaler ADC Deployments Published on: October 20, 2021 Overview This Tech Paper aims to convey what someone skilled in ADC would configure as a generic implementation. Note: It is unlikely that there is a single configuration that suits everyone. A consultant or administrator with a better understanding of your needs may deviate from these defaults and document the reasons for such changes. Power and lights out management settings If you have purchased a NetScaler ADC appliance, you should ensure: The NetScaler ADCs are deployed in locations separate enough to meet your high availability requirements. The redundant Power Supply Units on each ADC (if you have purchased such) are connected to separate electrical supplies. The lights out management card (if you have purchased appliances with such a card) are configured. You can find more information on configuring lights out management cards here. Physical network cabling, VLANs, and connectivity 1. All physical interfaces connecting the NetScaler ADC to your networks are redundant To ensure data flow during a cable, switch, or interface failure, connect your ADC to each network with redundant cables. To combine the interfaces connecting each network into a single link (known as a channel), you must configure link aggregation on your ADC. When possible, it is preferable to use the Link Aggregation Control Protocol (LACP). However, manual aggregated links are also possible if your network switches do not support LACP. Instructions for configuring Link Aggregation on your NetScaler ADC can be found here In a virtualized or cloud environment, your provider handles interface redundancy, and this step is not required. 2. Unused physical interfaces are disabled Disable any physical interfaces that you are not using. Disabling unused interfaces prevents them from being connected to other networks or devices accidentally or maliciously. You can disable a physical interface by selecting System, Network, Interfaces. Then, by ticking the box next to the interface, clicking “Select Action”, followed by “Disable”. 3. Set all redundant physical interfaces to be unmonitored for HA within System, Network, Interfaces As each network has redundant connections, it is not desirable that the ADC initiates an HA failover when a single link fails. Instead, the ADC should rely on the remaining link and only trigger a failover if all links fail. To prevent an HA failover when a single interface in a redundant channel fails, mark the component interfaces as unmonitored. To mark an interface as unmonitored, select System, Network, Interfaces. Then, select each interface that forms a part of each redundant channel and set the “HA Monitoring” radio button to “OFF”. In a virtualized or Cloud environment, you have virtual interfaces and don’t need to perform this step as your provider handles interface redundancy. 4. All channels comprising redundant physical interfaces should be monitored for HA within System, Network, Channels The failure of all aggregated links connecting the ADC to a particular network causes the channel representing those links to enter a failed/DOWN state. Check that monitoring has been enabled for the channel and the ADC will failover if all redundant links fail. To mark a channel as monitored, select System, Network, Channels. Then, select each channel and set the “HA Monitoring” radio button to “ON”. 5. All channels are bound to a separate VLAN and, you have taken care that no untagged channels are accidentally still in VLAN 1 Each redundant channel ordinarily represents the aggregate physical links connecting the ADC to a particular logical network. In a virtualized or Cloud environment, each interface likely represents aggregate physical links with your provider having completed the aggregation work for you. By default, the ADC considers all interfaces, channels, and IP addresses as being VLAN 1 and the same logical network. As such, overlooking the VLAN configuration would cause all IP address assigned to the ADC to be available from every directly connected network. To prevent this behavior, configure VLANs on the ADC to represent your logic networks and appropriately isolate traffic. You can find instructions to create VLANs here. 6. Create an HA pair between your ADCs within System, High Availability NetScaler considers it a best practice to deploy ADCs redundantly. You can achieve redundancy by implementing an HA pair, creating a cluster, or using a technology such as GSLB to split requests between instances. HA pairs comprise two ADC nodes, and clusters can have up to 32-nodes. For a generic implementation, NetScaler recommends the creation of a two-node HA pair. You can find instructions for configuring an HA pair here 7. Create and bind one SNIP to every VLAN, ensuring that each SNIP is in the subnet of the connected network NetScaler ADC initiates communication from a Subnet IP (called a SNIP) with limited exceptions. Create one Subnet IP/SNIP for every directly connected logical network. As you have already isolated each network using VLANs, you must bind each SNIP to its respective VLAN. Take care to ensure that no VLANs are missing a SNIP. The ADC identifies which VLAN each Virtual IP (VIP) needs to be placed in based on the SNIP’s subnets. The VLAN configuration then isolates virtual servers within their intended network. You can find instructions for configuring SNIPs here. Note: SNIPs host management services by default. To create SNIPs without the management service enabled, append the “-mgmtAccess DISABLED” parameter to the “add ns IP” command. 8. Configure the routes that the ADC requires within System, Network, Routes If you have connected multiple logical networks, you likely have routers in each. Therefore, you must now configure all the routes that the ADC requires to reach its clients and back-end servers. You can find instructions for configuring routes here. Note: The ADC has a single routing table that applies to all interfaces. 9. Create any Policy Based Routes required Occasionally, configuring a static route to provide for the behavior you desire is impossible. An ADC with distinct ingress, egress, and dedicated management networks, as well as management clients on the egress network, is the most frequent example. Static rules would not suffice in this case. Instead, a Policy-Based Route would be required (PBR). By using a PBR, you can force traffic from the ADC’s management IP to travel through the management router. Using a PBR will bypass the static routing table, which would otherwise send data to the egress network. You can find instructions for configuring Policy Based Routes here. However, if you have the scenario described in our example, you need the following PBR: add ns pbr Management ALLOW -srcIP = <NSIP_of_first_HA_node>-<NSIP_of_second_HA_node> -destIP "!=" <first_IP_management_subnet>–<last_IP_management_subnet> -nextHop <management_subnet_router> - priority 1apply pbrs 10. Make sure you can ping each SNIP with Mac Based Forwarding (MBF) is disabled or that you understand why you cannot NetScaler ADC has a mode called Mac Based Forwarding (MBF). MBF causes the ADC to ignore the routing table and to send replies to the MAC address from which it received traffic. MBF is of great use where you cannot define your routes. Let’s say the ADC has multiple Internet connections and must reply using the Internet router through which the traffic arrived. Here, MBF would cause the ADC to record the source MAC address of each connection and use this as the destination MAC in its reply. However, having MBF override the routing table can make troubleshooting more complex. With MBF, you cannot understand traffic flows from the ADC’s configuration file alone as MBF overrides the routing table, and traffic may not flow as you intended. The result is that MBF, while crucial to some implementations, can also lead to misconfigurations in the supporting network remaining undetected. Disable MBF to ensure that your routing table and PBRs are correct. After disabling MBF, verify that each SNIP remains reachable or that you understand why it does not. The command to disable MBF is disable ns mode mbf The command to enable MBF is enable ns mode mbf If you do not require MBF, leave it disabled. Note: In Azure Deployments, MAC-based Forwarding (MBF) is enabled by default You can find more information of Mac Based Forwarding here. 11. You have installed a new SSL certificate and key for the management GUI within Traffic Management, SSL, Certificates Web browsers do not trust the NetScaler ADC’s default SSL certificate. The lack of trust causes browsers to display a warning message when accessing the ADC’s management services. Browser warnings for certificates are intended to alert users when the connection is not secure. NetScaler recommends that you replace the default SSL certificate so that management users do not become accustomed to accepting warning messages. You can find details of how to replace the management SSL certificate here. Note: As NetScaler ADC shares the management SSL certificate between HA nodes, your replacement certificate must be trusted for all FQDNs used for management purposes. NetScaler recommends using a SAN certificate to include the FQDN of both HA nodes. Base configuration settings 1. Set the timezone and enable NTP Having accurate and easily understood timestamps in your log files is vital when troubleshooting or handling a security issue. First, set the timezone to something that makes sense for you. For example, if you have devices that log to a central syslog server and need to cross-reference data from each, use the same timezone as your existing servers. The command to set the timezone is: set ns param -timezone CoordinatedUniversalTime Second, add NTP servers and enable time synchronization using the commands: add ntp server pool.ntp.orgenable ntp sync After enabling time synchronization, view the NTP status to verify correct behavior by using the following command: nsroot@StevensADC-Primary> show ntp status remote refid st t when poll reach delay offset jitter==============================================================================*any.time.nl 85.199.214.99 2 u 113 1024 377 21.138 +0.762 0.654 Done nsroot@StevensADC-Primary> You can find details of how to the timezone using the ADC’s GUI here. You can find details of how to add NTP servers here. 2. Create a Key Encryption Key A Key Encryption Key (commonly known as a KEK) is used to encrypt and decrypt credentials that the ADC must store in a reversible form. For example, the ADC must keep its LDAP bind password reversible to retrieve it at authentication time. From NetScaler ADC firmware 13.0–76.31, the ADC automatically creates a KEK. On later firmware, the command returns an error message that you can safely ignore. On older releases, you can create a KEK with the command: create kek <RANDOMSTRING> 3. Set a non-default nsroot password Recent releases of NetScaler ADC firmware prompt you to change the default password for the “nsroot” account when you first log in. Recent firmware builds prompt for the password change as the “-forcePasswordChange” system parameter is enabled. For older releases, you must change the “nsroot” password with the command: set system user nsroot -password <NSROOTPASSWORD> 4. Add an account for ADM with external authentication disabled Ideally, connect all of your NetScaler ADCs to ADM for centralized licensing and management. The connection to ADM requires a user name and password which can be created with the following commands: add system user admuser <ADMPASSWORD> -externalAuth DISABLED -timeout 900bind system user admuser superuser 100set system user admuser -externalAuth DISABLED You can find more details about ADM here. 5. Restrict non-management applications access to the NSIP and only HTTPS access Prevent non-management services from accessing the management IP and set the management IP to require secure communication access (HTTPS rather than HTTP). set ns ip NSIP -restrictAccess enabled -gui SECUREONLY You can find more details on restricting access to the NSIP here. 6. Set a non-default RPC node password Set RPC communication (used for HA and GSLB) to use a non-default password. set rpcNode <NSIP_OF_SECONDARY_NODE> -password <RPC_SECONDARY_PASSWORD> -secure YES set rpcNode <NSIP_OF_PRIMARY_NODE> -password <RPC_PRIMARY_PASSWORD> -secure YES 7. HA failsafe mode is enabled to ensure that the last healthy node continues to provide service If an ADC’s monitored HA interfaces or routes indicate an error, the ADC enters an unhealthy state and trigger an HA failover. If the second HA node enters an unhealthy state, both stop providing service. HA fail-safe mode ensures that the last surviving node of a pair continues attempting to provide business services. set HA node -failSafe ON You can find more details on HA fail-safe mode here. 8. Restrict HA failovers to 3 in 1200 seconds In the unlikely event that HA failovers repeatedly occur, there comes the point where you want them to stop and to attempt to provide service. Here, we define a limit of three HA failovers within a 1200 second (20 minutes) period. set ha node -maxFlips 3set ha node -maxFlipTime 1200 9. Disable SSLv3 for management services The NetScaler ADC management GUI has SSLv3 and TLS1.0 enabled by default. To ensure secure communication, we will disable SSLv3. set ssl service nshttps-::1l-443 -ssl3 disabledset ssl service nshttps-127.0.0.1-443 -ssl3 disabled Depending on your company’s internal security policy, you can optionally disable TLS1.0. set ssl service nshttps-::1l-443 -ssl3 disabled -tls1 disabledset ssl service nshttps-127.0.0.1-443 -ssl3 disabled -tls1 disabled 10. Set generic modes and features NetScaler ADC has Layer 3 mode enabled by default. Layer 3 mode causes the ADC to act as a router and can normally be safely disabled. Edge mode causes the ADC to dynamically learn details about back-end servers when used in a configuration such as link load balancing. disable ns mode l3 edge You can find more details on Layer 3 mode here. The specific modes and features that you need depend on your use case. However, we can select a list of options that would apply to most installations. enable ns feature lb ssl rewrite responder cmp You can find more details about modes and features here. 11. Configure one or more DNS nameserver The NetScaler ADC needs to have access to one or more nameservers for DNS resolution. NetScaler ADC checks if the DNS servers are online using an ICMP monitor. To use DNS based monitoring and distribute load, it is usual to implement a local DNS load balancing virtual server. As DNS can use UDP or TCP, we create one load balancing virtual server for each protocol. Configure nameservers using the following commands: add lb vserver DNS_UDP DNS 0.0.0.0 0 -persistenceType NONE -cltTimeout 120 add serviceGroup DNS_UDP_SVG DNS -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport NO -cltTimeout 120 -svrTimeout 120 -CKA NO -TCPB NO -CMP NObind lb vserver DNS_UDP DNS_UDP_SVG add lb monitor DNS_UDP_monitor DNS -query . -queryType Address -LRTM DISABLED -interval 6 -resptimeout 3 -downTime 20 -destPort 53bind serviceGroup DNS_UDP_SVG -monitorName DNS_UDP_monitor bind serviceGroup DNS_UDP_SVG <DNSSERVERIP1> 53bind serviceGroup DNS_UDP_SVG <DNSSERVERIP2> 53 add lb vserver DNS_TCP DNS_TCP 0.0.0.0 0 -persistenceType NONE -cltTimeout 120 add serviceGroup DNS_TCP_SVG DNS_TCP -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport NO -cltTimeout 120 -svrTimeout 120 -CKA NO -TCPB NO -CMP NObind lb vserver DNS_TCP DNS_TCP_SVG add lb monitor DNS_TCP_monitor DNS-TCP -query . -queryType Address -LRTM DISABLED -interval 6 -resptimeout 3 -downTime 20 -destPort 53bind serviceGroup DNS_TCP_SVG -monitorName DNS_TCP_monitor bind serviceGroup DNS_TCP_SVG <DNSSERVERIP1> 53bind serviceGroup DNS_TCP_SVG <DNSSERVERIP2> 53 add dns nameServer DNS_UDP -type UDPadd dns nameServer DNS_TCP -type TCP 12. Set TCP and HTTP parameters While Window Scaling (WS) and Selective Acknowledgment (SACK) now default to enabled in ADC 13.0 firmware, you must enable these TCP settings in earlier firmware versions. set ns tcpparam -WS ENABLEDset ns tcpparam -SACK ENABLED Nagle causes the NetScaler ADC to combine data to send a smaller number of larger packets and can be enabled with the following command. set ns tcpparam -nagle ENABLED By default, NetScaler ADC forwards HTTP requests that arrive at a load balancer but do not conform to the RFC standard. Configure the ADC to drop invalid requests as a default. To allow exceptions to the default, you can change the HTTP options on an individual virtual server after discussions with your security team. Disable support for the HTTP/0.9 protocol, which has been obsolete for over 20 years. For reference, Mosaic 2.0 on Windows 3.1 includes support for HTTP/1.0. set ns httpparam -dropInvalReqs ENABLED -markHttp09Inval ON Cookie version 0 includes absolute timestamps, whereas version 1 cookies have a relative time. Cookies with an absolute timestamp do not expire at the expected time if the client and ADC clock differ. However, cookies with a relative time of +X minutes until expiry will. Internet Explorer 2 included support for version 1 cookies in 1995, and you are unlikely to experience issues by enabling this option. set ns param -cookieversion 1 13. Restrict SNMP queries to select servers It is good practice that the ADC only answers SNMP queries from hosts supposed to make such queries. Limit the hosts that the ADC allows SNMP queries from using the command: set snmp manager SNMPMANAGERIP You can find more details on configuring SNMP here. 14. Set SNMP alarms and traps It is helpful for the ADC to raise alerts when high CPU or memory usage conditions occur. You can send alerts via trap configuration to your SNMP server. You may also want to trigger alerts when HA failovers occur. You can implement such configuration with the following commands: set snmp alarm CPU-USAGE -state ENABLED -normalValue 35 -thresholdValue 80 -logging ENABLED -severity Informationalset snmp alarm MEMORY -state ENABLED -normalValue 35 -thresholdValue 80 -logging ENABLED -severity Criticalset snmp alarm HA-STATE-CHANGE -severity Critical add snmp trap generic SNMPTRAPDSTIP -communityName public Note: NetScaler recommends that you consider periodically reviewing threshold values to ensure that the ADC alerts on abnormal behavior that you want to investigate. 15. Set a remote syslog server Audit logging should be configured on an ADC, and audit logs should be stored and analyzed on a remote server. You can configure the NetScaler ADC to send audit logs to a remote Syslog server using the following commands: add audit syslogAction RemoteSyslogServerAction SYSLOGSERVERIP -loglevel ALLadd audit syslogpolicy RemoteSyslogServerPolicy true RemoteSyslogServerActionbind audit syslogGlobal -policyName RemoteSyslogServerPolicy -priority 100 [You can find more details about audit logging here] (/en-us/citrix-adc/current-release/system/audit-logging.html) 16. Set a timeout and prompt for management sessions NetScaler ADC 13.0 allows a default of 900 seconds (15 minutes) before disconnecting idle management sessions. On older firmware versions, you must ensure that you have configured an appropriate timeout. set system parameter -timeout 900 An administrator can open SSH sessions to multiple ADCs at the same time. Changing the ADC’s CLI prompt helps clarify the node to which a session is connected. The following commands cause the CLI prompt to display their username, the ADC’s host name, and the HA status of the instance. set system parameter -promptString %u@%h-%s After you run the command, the prompt will be rendered as: nsroot@hostname-Primary> 17. Centralized authentication for management accounts Security teams generally consider it better to control management accounts from a central platform such as Active Directory than to create accounts on each device. Usually, these centralized accounts are then granted permissions based on group memberships. The rationale for centralized authentication and authorization is usually that managing accounts on each device is time-consuming and prone to error. Also, there is the risk that management users not change their passwords frequently and that IT can forget to remove ex-employees’ accounts. Use the following commands to configure centralized authentication, keeping in mind that the LDAP filter controls who are allowed to log in. add authentication ldapAction LDAP_mgmt_auth -serverIP <LDAPMANAGEMENTSERVERIP> -serverPort 636 -ldapBase "<dc=mycoolcompany,dc=local>" -ldapBindDn "<serviceaccount@mycoolcompany.local>" -ldapBindDnPassword <LDAPPASSWORD> -ldapLoginName <sAMAccountName> -searchFilter "&(|(memberOf:1.2.840.113556.1.4.1941:<cn=NetScaler-ADC-FullAccess,ou=groups,dn=mycoolcompany,dc=local>)(memberOf:1.2.840.113556.1.4.1941:<cn=NetScaler-ADC-ReadOnly,ou=groups,dn=mycoolcompany,dc=local>))" -groupAttrName memberOf -subAttributeName cn -secType SSL -passwdChange ENABLED -nestedGroupExtraction ON -maxNestingLevel 5 -groupNameIdentifier samAccountName -groupSearchAttribute memberOf -groupSearchSubAttribute CN add authentication Policy LDAP_mgmt_pol -rule true -action LDAP_mgmt_authbind system global LDAP_mgmt_pol -priority 100 While these commands implement authentication, they do not control authorization and, by default, the authenticated users are not able to perform any actions. To grant the user (or more accurately, the group of which the user is a member) the right to perform actions on the ADC, you should use the following commands: add system group NetScaler-ADC-FullAccess -timeout 900add system group NetScaler-ADC-ReadOnly -timeout 900bind system group NetScaler-ADC-FullAccess -policyName superuser 100bind system group NetScaler-ADC-ReadOnly -policyName read-only 110 You can find more information about centralized authentication and authorization here. You can also find information about the LDAP filter string used above here. Further, from ADC firmware version 12.1.51.16 you can configure multifactor authentication for management users by following the steps here. 18. Disable LDAP authentication for the nsroot user As the NetScaler ADC processes authentication and authorization separately, users can authenticate using LDAP, and the ADC grants permissions based on their group membership. While not a good idea, you could also create user accounts with authorization permissions on the ADC. A password associated with an Active Directory user with the same name could then be used to authenticate these accounts. To prevent an Active Directory administrator from creating an “nsroot” user and being able to authenticate, you must disable external authentication for the “nsroot” user account. set system user nsroot -externalAuth DISABLED 19. TLS/SSL Best practices You can now follow the TLS/SSL Best Practice document to define a secure cipher suite that can be used to protect your virtual servers. You can find the TLS/SSL best practice document here.

Objective Post going thought this article one will be able to understand on how to: Create an ADC LB configuration Template with required set of configuration command. Convert Configuration Template with variables. Push the configuration to one or more Citrix ADC using the said configuration template. Before we start with the main content of this article, let us understand some basics on configuration Job, Citrix ADM Service and its onboarding. What is a Configuration Job? A configuration job is a feature in ADM Service which allows users to create a configuration template that can be run across one or multiple managed Citrix ADC instances. What is Citrix ADM Service? Citrix Application Delivery Management (ADM) Service is a central control and visibility SaaS solution for managing all Citrix deployments that includes Citrix ADC MPX/ VPX/ SDX/ CPX/ BLX and Citrix Gateway appliances that are deployed on-premises or on the cloud. To read and understand more on what all are benefits and features can be leveraged from Citrix ADM service, kindly follow the link below: https://docs.citrix.com/en-us/citrix-application-delivery-management-service/overview.html How to Quickly onboard on ADM service? If you are new to Citrix ADM service, you can easily onboard the Citrix ADM service in three easy steps: Create account in Citrix Cloud. Click ADM service tile on Citrix Cloud to select built-in agent mode. Use the service URL and the activation code to add Citrix ADC on ADM service. Here is the link to a small YouTube video which guide through above mentioned steps: And more details over same can be found here at our Citrix Documentation: https://docs.citrix.com/en-us/citrix-application-delivery-management-service/getting-started/initiate-built-in-agent.html Workflow Overview Let’s create a basic configuration example wherein one wants to load balance and send a secure end to end TLS traffic to its backend server along with Source IP accounting, based on request URL path's content via Citrix ADC. In order to achieve this, we will follow given steps: Add a service group with two services with Client IP forwarding to backend. Disable SSL v3 protocol to deny any SSLV3 request towards backend servers. Bind server to service group. Add a non-addressable load balancing virtual server. Disable SSL v3 protocol to deny any SSLV3 request towards load balancing virtual server. bind the service group to the virtual server. Add a content switch action with target as a load balancing virtual server Add a content switch policy which contains expression rule to direct the request to a load balancing virtual server based on request URL path's content. Add a content switch virtual server. Disable SSL v3 protocol to deny any SSLV3 request for content switch virtual server bind the content switch policy to the content switch virtual server. As stated at start that configuration job is a set of configuration commands that can be run on one or more managed ADC instances. When one runs the same configuration on multiple instances, they might want to use different values for the parameters used in the configuration. One can define variables that enables to assign different values for these parameters or run a job across multiple instances. Variables usually are the IP address, entity name etc in the configuration. Create an ADC configuration Template with required set of configuration command Below is the sample CLI command which we are going to use for configuration of ADC using ADM service Configuration Job feature. add serviceGroup mylb_SVG:443 SSL -maxClient 0 -maxReq 0 -cip ENABLED X-Forwarded-For set ssl serviceGroup mylb_SVG:443 -ssl3 DISABLED bind serviceGroup mylb_SVG:443 10.11.12.13 443 bind serviceGroup mylb_SVG:443 10.14.15.16 443 add lb vserver mylb_VS:443 SSL 0.0.0.0 0 -persistenceType COOKIEINSERT -cltTimeout 180 set ssl vserver mylb_VS:443 -ssl3 DISABLED bind ssl vserver mylb_VS:443 -certkeyName servercert_Mar2022 bind lb vserver mylb_VS:443 mylb_SVG:443 add cs action mylb_cs_action_443 -targetLBVserver mylb_VS:443 add cs policy mylb_cs_policy_443 -rule "HTTP.REQ.URL.PATH_AND_QUERY.STARTSWITH (\"/test\") || HTTP.REQ.URL.PATH_AND_QUERY.STARTSWITH (\"/default\")" -action mylb_cs_action_443 add cb vserver mylb_CSVS:443 SSL 10.17.18.19 443 bind ssl vserver mylb_CSVS:443 -certkeyName servercert_Mar2022 set ssl vserver mylb_CSVS:443 -ssl3 DISABLED bind cs vserver mylb_CSVS:443 -policyName mylb_cs_policy_443 -priority 100 Note: In the above example for command number Seven (7) and (12), I am binding a certificate already present on Citrix ADC. Please follow the “Install SSL certificate on Citrix ADC Instance” section on Citrix ADM documentation covering steps on how to install a Server certificate on a managed Citrix ADC using Citrix ADM service. To create a configuration template: Select the section which the service group name “mylb_SVG:443” to convert it to a variable. Once the dollar sign appears enclosing the variable’s value, bring cursor over green dollar enclosed variable and click on it to edit. On righthand side small window under “Define Variable” section, edit and add in Name, Display Name and Keep “Type” field as “Text Field” and click on Done. Similarly, as per step 6 convert the Service Group name into variable for first 4 command with reference to service group name. Change the backend service IP address into variable with a unique names and the field “Type” to be set as “IP Address Field”. Follow the same steps to define the variable for load balancing virtual sever, Content switch action, content switch policy and content switch virtual server PS: Please find below location to download the config template’s json file created for this configuration job workflow here: https://citrix.sharefile.com/d-sab5397f46b644209a75776f4ec07dc2f This can be used as reference template by importing it on ADM service Configuration template page. In order to upload a template, navigate to Infrastructure > Configuration > Configuration Jobs > Configuration Template and Click on “Import” to upload the configuration json file by navigating to file’s saved location from the local desktop/laptop. Push the configuration to one or more Citrix ADC using the said configuration template Navigate to Infrastructure > Configuration > Configuration Jobs. Click on “Create Job” if this is your first time to this page. and if already have other configuration jobs, then click on “Create Job” as shown below. On the Create Job page, select the custom job parameters such as the name of the job, the instance type, and the configuration type. Within Configuration Editor, the “Configuration Source” will by default be showing “Configuration Templates” which further shows all the custom configuration templates created. Drag and drop the configuration template “CSVserver_Template” created earlier or Click on “+” sign to add it. Click on “Preview Variable” to see all the eight (8) variables that has been defined so far. Click on “Next” button. Under “Select Instances” tab, click on “Add Instance” to add the ADC Instance(s) where the selected configuration needs to be pushed. Click on “Next” button. In the “Specify Variable Values”, one need to add the values for all the variables that are part of configuration template. In our configuration sample as quoted earlier eight variable (step#5) needs to have their values filled in. To perform this step, select the “Upload input file for variable values” and click on “Download Input Key File” to download a csv file which needs to be filled with values and uploaded back. Above step will download the following csv file with all the variable for which values needs to be filed in. Define the values for all variables against the instance(s) and save the file. Note: In case of multiple instances selected for which same configuration need to be pushed one need to input values for respective variable. Go back to ADM service page and on the “Specify Variable Values” tab upload the csv file by navigating to file’s saved location on local desktop/laptop. PS: If the file upload has all the required fields filled up a success message will pop-up on top of ADM service web browser stating: “Input key file “CSVIP_config_job_variable_input_key_file_csv” uploaded successfully.” Click on “Next” button and briefly one can see “Fetching Job Preview” message on screen On the “Job Preview” one can select the individual Instance from “Select an Instance to preview” to view the config that will pushed under this configuration job. Click on “Next” button. Under “Execute” Tab, once can select to execute the “Now” or schedule it for “Later” by selecting under “Execution Mode”. One can also choose what action Citrix ADM service must take if the command fails “On Command Failure” and if one would like to send an Email/Slack notification regarding the success or failure of the job along with other details. Click on “Finish”. PS: In this workflow, job is selected to be executed immediately by ignoring errors. Citrix ADM Service will take back to “Jobs” page and there it will the live progress in percentage. Once the job is executed successfully you will see the “Completed” status under “EXECUTION STATUS” On the “Jobs” page click on “Details” button and then on “Variable Details” to view the details of variables added to the selected Job Details of configuration job executed can be downloaded by clicking on “Download” under “ACTIONS”