Steven Wright

Overview Citrix Application Delivery Management (ADM) is a centralised management solution that enables you to operate, upgrade, report on, and troubleshoot all your NetScaler's across a global estate. One of Citrix ADM's best features is StyleBooks - declarative descriptions of a desired state written in YAML and consisting of input parameters and components driven by those inputs. A StyleBook causes Citrix ADM to read your current NetScaler configuration, compute the differences between your current and target state, and implement the required settings. While you can write StyleBooks yourself, Citrix ADM comes with more than 35 default StyleBooks, covering everything from building a simple HTTP load balancer to all of the content switching, load balancing, and authentication settings needed to support Exchange 2016. Learn about using default StyleBooks and creating custom StyleBooks. In this blog post, I focus on simple HTTP load balancers but provide enough detail so you can apply the logic to any StyleBook. Specifically, I'm going to focus on a StyleBook called "lb" with the namespace "com.citrix.adc.stylebooks"which you can find by selecting Applications, Configuration, StyleBooks within Citrix ADM. This post is the first in a two-part series that describes how to rapidly create services on your NetScaler's using Citrix ADM StyleBook automation, accelerating your migrations. We'll create 300 load balancers across an estate of several NetScaler's within 15 minutes. This post is aimed at DevOps employees who want to start working with Citrix ADM and need a brief introduction with examples (which you can download here). In my second post, I target sysadmins and focus on using preprepared scripts to achieve our goal of creating the 300 load balancers. Citrix ADM and the NITRO API The second best thing about Citrix ADM is that you can access it using API calls, which is how we'e going to automate our StyleBook. You can see more on that below but, you can also find more information here: Citrix ADM 13.1 NITRO API Reference documentation Using APIs to create configurations from StyleBooks To call your script to log in using the NITRO API, you need an ID and secret. You can create the ID and secret for Citrix ADM service by doing the following steps: Sign in to the Citrix Cloud portal. Select Identity and Access Management from the menu. Select the API Access tab. Name your Secure Client, and click Create Client. NITRO API request for a session ID After you have the ID and secret, you can sign in and collect a session ID using the following Python commands (this isn't my real ID and secret): Python import http.client import json id = "f20e7432-7a83-416d-a171-9347b92761f4" secret = "7-rotB6QMFZTuOgpPajPBg==" conn = http.client.HTTPSConnection("adm.cloud.com") headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': '' } request = { "login": { "ID": id, "Secret": secret } } payload = json.dumps(request) conn.request("POST", "/nitro/v2/config/login", payload, headers) response = conn.getresponse() data = response.read() if response.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) jsondata = json.loads(data) sessionid = jsondata["login"][0]['sessionid'] print (sessionid) Output ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM NITRO API request using the session ID to retrieve a list of NetScaler identifiers Now that you have a session ID token, you can request a list of NetScaler's and find the unique identifier of the instance on which you would like to operate. The session ID is taken from the preceding script. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" conn = http.client.HTTPSConnection("adm.cloud.com") headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn.request("GET", "/nitro/v2/config/managed_device", None, headers) response = conn.getresponse() data = response.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) jsondata = json.loads(data) #print('\n\nHere is the full JSON output from ADM.\n') #print (jsondata["managed_device"]) print('\n\nHere is a list of ADCs.\n') for device in jsondata["managed_device"]: name = device['hostname'] ip = device['ip_address'] id = device['id'] type = device['type'] version = device['version'] status = device['status'] if status == "Success": print (name + " " + ip + " " + id + " " + type) Output Here is a list of ADCs. myhostname 192.168.1.240 a1fe50be-e187-44bb-871b-d60f112b0aff nsvpx Knowing the parameters to submit to a StyleBook Now that we have a session ID and the identifier of an NetScaler to target, we can run a StyleBook. First, however, we need to understand the various parameters that it expects us to submit before doing so. You can gather the parameters by programmatically requesting the StyleBook's schema, which describes all valid parameters and which are optional, or by reviewing the source YAML for the StyleBook and its dependencies. Here are the Python commands to request a StyleBook's schema. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" base_types = ["string", "ipaddress", "number", "boolean", "tcp-port", "password", "file", "certfile", "keyfile", "certkey", "ipnetwork"] def get_config_parameters(parameters): configpack_params = {} for parameter in parameters: parameter_name = parameter["name"] parameter_type = parameter["type"] parameter_required = parameter["required"] if parameter_type in base_types: if parameter_required: parameter_type += "*" if parameter_type.endswith("[]"): configpack_params[parameter_name] = [parameter_type] else: configpack_params[parameter_name] = parameter_type else: if "parameters" in parameter: subparameters = parameter["parameters"] datatype = get_config_parameters(subparameters) else: datatype = parameter_type if parameter_type.endswith("[]"): configpack_params[parameter_name] = [datatype] else: configpack_params[parameter_name] = datatype return configpack_params conn = http.client.HTTPSConnection("adm.cloud.com") stylebook = "com.citrix.adc.stylebooks/1.1/lb" headers = { 'Cookie': 'NITRO_AUTH_TOKEN= ' + sessionid } conn.request("GET", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/schema", None, headers) res = conn.getresponse() data = res.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) schema = json.loads(data, "utf-8") parameters = schema["schema"]["parameters"] config_parameters = { "parameters": get_config_parameters(parameters) } print(json.dumps(config_parameters)) The output from this schema request is the format of the parameters section of a config pack. From this output, you can remove parameters you don't intend to use. As a clue, the required parameters are marked with a "*" at the end of their type. Another (and possibly easier) way to achieve the same thing is to use the Citrix ADM UI to create a template that shows only the parameters I want to use. To create the template, I open Citrix ADM's web interface and run my selected StyleBook with dummy values against a non-production NetScaler in a lab. In doing so, I create a config pack with each of the values I would need for a production service. Having created the config pack, I now select Applications > Configuration > Config Packs from within the Citrix ADM web interface and export the resultant configuration. After downloading and opening the exported ZIP file, I find that it contains a single JSON file with all the parameters that I need to submit to the API. NITRO API request using the session ID to automate a StyleBook and target an NetScaler identifier Finally, having retrieved a session ID, the unique identifier of the NetScaler to operate on, and clarity into the parameters needed to submit to the StyleBook, we are ready to script execution of the StyleBook itself. Note: Delete the template configuration pack from Citrix ADM if you want to use the same input parameters within your Python script. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" target_adc = "a1fe50be-e187-44bb-871b-d60f112b0aff" stylebook = "com.citrix.adc.stylebooks/1.1/lb" conn = http.client.HTTPSConnection("adm.cloud.com") payload = json.dumps({ "configpack": { "targets": [ { "id": target_adc } ], "parameters": { "lb-appname": "ApplicationName", "lb-virtual-ip": "192.168.50.10", "lb-virtual-port": "80", "lb-service-type": "HTTP", "svc-service-type": "HTTP", "svc-servers": [ { "ip": "192.168.5.20", "port": 80, "add-server": True }, { "ip": "192.168.5.21", "port": 80, "add-server": True } ], } } }) headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn.request("POST", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/configpacks", payload, headers) res = conn.getresponse() data = res.read() if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1) payload = json.loads(data, "utf-8") jobid = payload["configpack"]["job_id"] print("Configuration Job " + jobid + " has started.") Output Configuration Job 204770189 has started. Viewing a job_id At this point you are able to see the config pack that your script created within the ADM GUI and the resultant configuration on your NetScaler. If that didn't happen, you can view the details of the "job_id" and understand its status. We can retrieve the outcome of the job by sending a GET request. To illustrate, I purposely ran another StyleBook targeted at a non-existent NetScaler ID. Python import http.client import json sessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM" jobid = "2476087615" headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid } conn = http.client.HTTPSConnection("adm.cloud.com") job_status = None while job_status != "completed" and job_status != "failed": conn.request("GET", "/stylebook/nitro/v1/config/jobs/" + jobid, None, headers) response = conn.getresponse() data = response.read() if response.status != 202: print("Error:") print(" Status code: " + str(response.status)) print(" Error payload: " + data) exit(1) payload = json.loads(data, "utf-8") job_status = payload["job"]["status"] config_id = payload["job"]["progress_info"][0]["id"] if job_status == "completed": print("Success: configuration " + config_id + " successully created") else: print("Error: Failed to create configuration.") progress_info = payload["job"]["progress_info"] for step in progress_info: error_details = step["message"] print(" " + json.dumps(error_details)) exit(1) Output (failure) Error: Failed to create configuration. "Validating the parameters" "Configuration pack process failed. Instance with id g4702445-2286-4b6d-8fb1-c231a0ee17b1 not found on the system." Here, we can observe a clear explanation of the issue. We can observe success messages on jobs that run without issue. Output (success) Success: configuration 204770189 successfully created Next steps We have now seen how we can programmatically authenticate to Citrix ADM, retrieve a list of identifiers for the NetScaler's that it manages, prepare and run StyleBooks against a chosen NetScaler, and retrieve the result. We have also observed how you can find the namespace, version, and name of alternative StyleBooks and retrieve the various properties they require. Using this knowledge, your DevOps team is able to integrate Citrix ADM StyleBooks and their automated NetScaler configuration into your existing workflows. In the second of this two-part series, using the example of creating 300 load balancers across an estate of several NetScaler's within 15 minutes, I share prebuilt scripts that allow those without a DevOps team to create business services rapidly during a platform migration.

In this two-part blog series, I describe how to rapidly create services on your NetScaler's using Citrix Application Delivery Management (ADM) StyleBook automation to accelerate migrations. In the example, we create 300 load balancers across an estate of several NetScaler's within 15 minutes. My first post was aimed at DevOps. In this post, I target Sysadmin employees and focus on a pre-prepared script that I wrote. The script helps you to achieve our goal of rapidly creating load balancing virtual servers. You can download the script and its two supporting files here. Note: The script requires Python 3 and jinja2, that you can install with pip using the command pip install jinja2 Prepare the StyleBook template("template.txt") The first of the script's two supporting files is named template.txt. The template.txt file is already preprepared for the simple HTTP load balancers that we create in this post, and you don't need to modify it. However, I do want to explain how you can make those modifications, in case you want to target other StyleBooks or introduce extra parameters. On the second line of the template.txt, you can find the namespace, version, and the name of the targeted StyleBook. StyleBook NAMESPACE/VERSION/STYLEBOOK com.citrix.adc.stylebooks/1.1/lb If you want to change the template file's target, you can view the namespace, version, and the name of another StyleBook within Citrix ADM's web interface. On lines five to 12 of the template.txt, you can find placeholder names that correspond to the columns in a CSV file called lbvserver.csv, which is also included with the script. CSV columns targetadc name lbvirtualip lbvirtualport lbservicetype svcservicetype svcservers: list svcserverport As you can see from the placeholder named svcservers: list, if the columns in the CSV file contain a list of items, such as a list of back-end servers for a load balancing virtual servers, we note this by appending ": list" to the placeholder's name. You can find a template with each value that our script must submit to the StyleBook at line 14. Each item in the template matches one of the placeholder values you created earlier, and the template includes a loop where it encounters a list. Template { "configpack": { "targets": [{"id": "<<targetadc>>"}], "parameters": { "lb-appname": "<<name>>", "lb-virtual-ip": "<<lbvirtualip>>", "lb-virtual-port": "<<lbvirtualport>>", "lb-service-type": "<<lbservicetype>>", "svc-service-type": "<<svcservicetype>>", "svc-servers": [ <% for server in svcservers %> { "port": << svcserverport >>, "ip": "<< server >>", "add-server": true }<<,>> <% endfor %> ] } } } Again, you need to edit the template file only you're modifying the script to target an alternative StyleBook or you're introducing extra parameters. Preparing the script As we saw in Part 1, you need an ID and secret to allow the script to log in into your Citrix ADM environment. If you're using the Citrix ADM service, you can create the ID and secret by doing the following steps: Sign in to the Citrix Cloud portal. Select Identity and Access Management from the menu. Select the API Access tab. Name your Secure Client, and click Create Client. After you have the ID and secret, replace the placeholder example values within the create2.py python script. # Set my login variables for ADM # You can create an ID and secret by selecting the Identity and Access Management tab in your Cloud portal Id = "1707d2ff-e200-46be-848a-93faf2076d6b" Secret = "DIH1CzF_GSZG2zNnEVShLA=="` Preparing the Excel CSV template Now that you've set the ID and secret values, you can run the script with the "-a" parameter to display a list of NetScaler's and their identifiers. Command python create2.py -a Output The following NetScaler instances IDs can be referenced within your CSV file. Name - IP address - ID for use in CSV. 1\. bob - 192.168.1.240 - a0de40bb-e186-44bb-871b-d60f112b0a3a Here, we can observe that I have an NetScaler with the host name "bob" on IP address "192.168.1.240" with the unique identifier "a0de40bb-e186-44bb-871b-d60f112b0a3a." You must note each identifier because we need them later. Prepare a list of load balancing virtual server ("lbvserver.csv") The second of the script's two supporting files is named lbvserver.csv. Each row in the lbvserver.csv file includes the properties for a single StyleBook, in this case, the properties for a single HTTP Load Balancing virtual servers. Each column in the lbvserver.csv file is the source for the placeholders within the template.txt file. The columns and placeholders are in the same order, with the first column being the first placeholder and so on. I have expanded the lbvserver.csv file to contain 300 rows (300 HTTP Load Balancing virtual servers) for this post. If my customer had a similar requirement, I would likely populate an initial CSV with all the expected values. Then, they would forward the CSV internally to ensure that the names, naming conventions, and IP addresses were all as desired. As you can observe by comparing the columns to the template.txt file, column A specifies the unique identifier of the NetScaler that we target, and columns B to H are the input parameters for the StyleBook. You can include lists in the parameters with ":" used as a separator. After modifying the lbvserver.csv file to reflect the virtual servers that you want to create, you must save in "CSV" rather than "CSV (UTF8)" format. Running the script (python create2.py) Having populated the script with an ID and secret that allow access to your Citrix ADM environment and updated the columns to reflect which NetScaler you want to build on and the properties, we are now ready to create our load balancing virtual servers. We can now run the script and create a Citrix ADM config pack along with the resultant NetScaler configuration. Command python create2.py Output (shortened) Creating ADM Config Pack based on CSV row: 1 {"configpack": {"job_id": "1926702141"}} Creating ADM Config Pack based on CSV row: 2 {"configpack": {"job_id": "2520889571"}} Creating ADM Config Pack based on CSV row: 3 {"configpack": {"job_id": "2581540139"}} Creating ADM Config Pack based on CSV row: 4 {"configpack": {"job_id": "1440135035"}} Creating ADM Config Pack based on CSV row: 5 {"configpack": {"job_id": "3264746514"}} The script takes around two to three seconds per row and completes the 300 rows in just under 15 minutes. After completion, the resultant Config Packs will be visible within ADM. Ongoing maintenance In addition to creating the virtual servers rapidly from a pre-agreed Excel CSV file, this script-based approach also allows management of the resultant virtual servers using ADM and future changes without a need to understand the underlying commands on their NetScaler's. You can view each of the resultant Config Packs by selecting Applications, Configuration, Config Packs from within the Citrix ADM web interface. By selecting Edit, you can then manage and update the Config Pack as your needs change. Next steps We have now created 300 load balancers across our estate using an adaptable template that allows you not only to create load balancers but any other StyleBook based configuration. If you want to understand how the script works, you can read through the first article in this series .

NetScaler migration Have you bought a new NetScaler with more throughput or is it time to refresh? Are you migrating from old NetScalers to new ones? Sounds difficult? Then you've stumbled across the right article. Installing new NetScalers into production and migrating services may seem like a complicated process, but once you know how to do it, it's much easier than you might think. This blog post will walk you through the steps needed. In this article, we will assume that you have an existing source NetScaler that's running in production and a new target NetScaler that you want to prepare in a non-production environment and migrate to under a change control process. Prepare for the migration (existing NetScaler) ADM Service's migration feature requires NetScaler firmware version 13.1.17 or above. To prepare for the upcoming migration, upgrade your existing NetScalers to 13.1.17 firmware so that both the source and target NetScaler have the same firmware version and support for ADM Service's migration feature. NetScaler recommends upgrading to the same firmware on the source and target NetScalers to minimize the number of changes during the migration and, in the unlikely event of any issues, to identify the cause more easily. You can upgrade your existing NetScaler to 13.1 firmware using the instructions here. Prepare for the migration (target NetScaler) First, install your new target NetScaler into a non-production environment without a connection to the production network. Ensuring the non-production and production networks are isolated from each other will prevent unintended interactions such as IP address conflicts. Once you have installed the new NetScaler, connect it to ADM Service. To do this, you need to add an ADM Agent into the non-production environment. You can install a new ADM Agent using the instructions here and add you can connect the new NetScaler to ADM Service using these instructions. Next, install a license onto your new NetScaler. You can apply a pooled license to your new NetScaler using the instructions here. Replicate the configuration After ensuring that the new NetScaler is not connected to any production networks, follow these steps to replicate the configuration. 1. Select Infrastructure, Instances, NetScaler, and NetScaler form factor In the screenshot, we have selected the "VPX" form factor and can see two NetScalers, "192.168.6.30" and "192.168.1.230". In this article, we will use "192.168.6.30" as the source NetScaler and will replicate its configuration onto the target NetScaler on "192.168.1.230". 2. Tick the source NetScaler, click "Select Action", and then choose "Replicate Configuration" 3. Move the target NetScaler instance from the "Available" to "Configured" window using the arrow button and click "Create" ADM Service will now replicate the configuration from your source to target NetScaler. Note: Any existing configuration on the target instance will be erased and replaced with the configuration of the source NetScaler. The source and target NetScalers should be on separate networks to prevent IP conflicts. Validate the network configuration on the target NetScaler Although the source and target NetScalers will now have the same configuration, the management NSIP and network interface layout may differ. If you installed your new NetScaler into the non-production environment using a network interface layout that you don't want to use in production, you should change they layout now. To change the network configuration, log into the target NetScaler and ensure: That all unused interfaces are disabled That you have created LACP channels as you require That a SNIP is bound to each VLAN That each VLAN is bound to an interface or channel using your intended network layout You can find instructions detailing how to disable unused interfaces, create LACP channels, and how to bind SNIPs and VLANs here. Replace the NSIP on the target NetScaler If you installed your new NetScaler into the non-production environment using an NSIP that differs from production and would prefer it was the same at the point of migration, you should change the NSIP now. You can change the NSIP using the following steps. Note: If you are migrating an HA pair, you must perform the commands below on both nodes at the same time and shutdown both nodes at the same time. Select the target NetScaler within the "Infrastructure, Instances, NetScaler" menu of ADM, choose "SNMP" from "Select Action", then disable SNMP. Note: We are disabling these services to ensure they are not configured to communicate with the pre-production ADM Agent, we will re-enable them after the migration is complete. Using the same process as "step 1" (above), disable Syslog and Analytics on the target NetScaler. Select the source NetScaler and repeat steps one and two (above) to disable SNMP, Syslog, and Analytics. Log into the target NetScaler's command line interface using SSH. You can find details of how to use SSH here. Change the NSIP of the target NetScaler using the commands shown in red. shell cd /nsconfig cp ns.conf ns.old sed -i -e 's/192.168.1.230/192.168.6.30/g' ns.conf Note: 192.168.6.30 is the source NetScaler in our example and 192.168.1.230 is the target NetScaler. You must replace the IP addresses within the command with those of your source and target NetScaler. Change references to the NSIP of the other HA node if applicable sed -i -e 's/192.168.1.231/192.168.6.31/g' ns.conf Note: You will run the sed command twice on both HA nodes. Firstly, to replace the NSIP of the current node. Secondly, to replace all references to that changed NSIP on its HA partner. Shutdown the NetScaler shutdown now Note: If you shutdown the NetScaler using an alternative command then you should reply "no" when ask if you wish to save the running configuration. The output will look like the text below. Note: In this example, we have included an example HA node with an NSIP of 192.168.1.231 that needs to be replaced with 192.168.6.31 - the replacement needs to be executed on both HA nodes. ############################################################################### # # # WARNING: Access to this system is for authorized users only # # Disconnect IMMEDIATELY if you are not an authorized user! # # # ############################################################################### Password: Last login: Mon Dec 6 14:26:05 2021 from 192.168.6.20 Done nsroot@-Primary> shell Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. root@ns# cd /nsconfig root@ns# cp ns.conf ns.old root@ns# sed -i -e 's/192.168.1.230/192.168.6.30/g' ns.conf root@ns# sed -i -e 's/192.168.1.231/192.168.6.31/g' ns.conf root@ns# shutdown now Shutdown NOW! shutdown: [pid 8140] root@ns# *** FINAL System shutdown message from root@ns *** System going down IMMEDIATELY System shutdown time has arrived Connection to 192.168.6.30 closed by remote host. Replace the NetScaler under change control With the existing source and target NetScalers now having identical configurations, you can replace them in compliance with your change control procedures. Many customers achieve that change using a two-stage replacement process. In the first stage, they work with their network teams to connect the new NetScalers to network switch ports that are administratively disabled or offline. In the second stage, during a change control window, the network team places the switch ports to the old NetScalers into a disabled or offline state, brings the interfaces connecting the new NetScalers online, and flushes the ARP cache on their router. The effect is that all hardware installation and readiness activities can be completed in-advance and the change control window to move from the old to new appliances is completed in minutes. This approach also provides an excellent rollback strategy as the network could disable the interfaces connecting the new NetScaler and reenable those for the old, which are still powered on and running without modification. Re-establish connectivity to ADM Service You have now replaced the source NetScaler with the target. ADM will show the source as online (as the target is using the source NetScaler's NSIP) and the target's old NSIP as offline. You should now complete the following steps to ensure the migrated NetScaler is correctly communicating with ADM Service. To ensure ADM is communicating with the new NetScaler that has taken over the NSIPs of the source, select the source NetScaler within "Infrastructure, Instances, NetScaler" menu of ADM and, choose "Rediscover" from "Select Action". After the rediscovery completes, re-enable the SNMP, Analytics, or Syslog capabilities that you disabled at step one of "Replace the NSIP on the target NetScaler". To enable each, select the NSIP of the source NetScaler and the capability under "Select Action". If you are using pooled licensing on the new NetScaler, you must also ensure it is using the production ADM Agent as its license server. To do this, log into the new NetScaler using SSH and enter the following commands: sh nslicenseserver If the new NetScaler is not using the production ADM Agent as its license server, you can correct the license server to production ADM using the following commands: add nslicenseserver <prod_agent_ip> -port 27000 -forceUpdateIP Remove the target NetScaler's old NSIP from ADM by ticking the old NSIP within the "Infrastructure, Instances, NetScaler" menu of ADM and pressing "Remove". Next steps You have now finished migrating from your old to new NetScalers and can begin using them in production.

NetScalers are used by some of the world's largest service providers to handle inbound application traffic. In this article, I will show how you can deploy an NetScaler environment that will scale to millions of HTTP requests and SSL handshakes per second. In the next, I will show you how to automate that environment with a basic introduction to Terraform. The suggested architecture will allow for multiple datacenters. In addition, it will provide load balancing, SSL offload, and the capability to secure and optimize your web application traffic. The architecture will also be scalable and fully automatable. We will begin by reviewing technologies to get traffic to your datacenter, how to distribute that traffic across NetScalers once it arrives, and how you can configure your NetScalers with code using in-house deployment scripts. Assuming you have sized the rest of your environment appropriately, this design should allow you to process around 100 million HTTP requests per second at each datacenter. However, the layout is scalable and can be used for smaller implementations without significant changes. Directing users to their most appropriate datacenter Historically, there have been two approaches to directing users to their closest datacenter: dynamic routing (anycast) and GSLB. Dynamic routing (anycast) With the dynamic routing / anycast approach, the routers at each datacenter announce the same IP ranges and rely on the Internet's routing protocol (BGP) to take the shortest path. The approach operates on the assumption that a British user will have a shorter path to your British datacenter and your American user to an American datacenter. The method also assumes that a short path will be the best route. The concept is illustrated in simplistic terms in the diagram below. Users, sending traffic to their ISP, will have that traffic forwarded to the datacenter that is the smallest number of hops away. A standardized repeatable design at each datacenter With traffic now arriving at the most appropriate datacenter, we can focus on handling the resultant traffic. We will design each datacenter to operate with a standardized design so that it's simple and easy to update and maintain. We will use physical hardware NetScalers known as "MPXs" for their high performance. However, our design is equally suitable for virtual NetScalers "VPX" if you favor a complete infrastructure-as-code approach. In our design, traffic (directed by CADS) will enter the network at the router/firewall. The router will then distribute these inbound connections to our NetScalers. Next, the NetScalers will terminate the connections, process the requests, and deliver them to backend servers. Finally, the servers will service the requests, potentially using local data or a storage solution replicated between your datacenters. All of the configuration will be automated using Terraform (covered in the next article). However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as: Ansible Citrix ADM Service Nitro REST API to allow for custom scripts. We assume you have appropriately sized your routers, firewalls, and storage. And you have ensured your storage is replicated between datacenters if your servers so require. This article shows a single router as a simplification. In our dynamic routing configuration, each NetScalers "announces" the IP addresses of the virtual servers (vServers) they host to the router. As each NetScaler will host vServers with the same IP addresses, the router will distribute the inbound connections between the NetScalers. On a Cisco router, the distribution of traffic between OSPF neighbors will likely occur in hardware using a technology known as CEF, Cisco Express Forwarding, to reduce the impact on CPU. You must take care to ensure your router has sufficient resources to handle the distribution. Additionally, to ensure all packets from a user are directed to the same NetScaler, you should consult your routers manual to ensure it is using per-destination load-sharing, which is the default on a Cisco router. To implement our dynamic routing configuration, we first need to configure our router to have an OSFP neighbor relationship with the NetScalers. On a Cisco router, a very simplistic OSPF configuration would look as follows: enable router# config t router(config)# router ospf 100 router(config-router)# network 192.168.1.0 0.0.0.255 area 0 router(config-router)# maximum-paths 16 router(config-router)# end After we enter the "network 192.168.1.0 0.0.0.255 area 0" command, our router will enable OSPF on all of its interface addresses within that network range. Next, on our NetScalers, we need to enable the OSPF routing facility with the command "enable ns feature ospf".After enabling OSPF, we can configure the NetScaler's included routing platform known as "ZebOS", by entering the command "vtysh". The ZebOS configuration on each NetScaler would look as follows: NetScaler# config terminal NetScaler(config)# router ospf NetScaler(config-router)# network 192.168.1.0/24 area 0 NetScaler(config-router)# redistribute kernel With the OSPF configuration created, we can tell each NetScaler to advertise the IP address of a vServer (such Load Balancing vServer) using the following command: set ns ip 10.0.0.10 255.255.255.255 -hostroute ENABLED -vserverRHILevel ALL_VSERVERS This command will tell the NetScaler to announce the IP address 10.0.0.10 to the router. With each of our NetScalers now announcing itself to our router as a path to the Load Balancing vServer, the router will distribute inbound traffic to each of the NetScalers. After enabling OSPF, you can validate communication between the NetScalers and router using the following commands on the router. Here, we can see three NetScalers on "192.168.1.10", "1.11", and "1.12" advertising the LB vServer on "10.0.0.10". R1# sh ip ospf | s 10.0.0.10 > 10.0.0.10/32, Intra, cost 1, area 0 via 192.168.1.10, GigabitEthernet1.1 via 192.168.1.11, GigabitEthernet1.2 via 192.168.1.12, GigabitEthernet1.3 R1# sh ip cef 10.0.0.1 detail 10.0.0.1/32, epoch 0, per-destination sharing nexthop 192.168.1.10 GigabitEthernet1.1 nexthop 192.168.1.11 GigabitEthernet1.2 nexthop 192.168.1.12 GigabitEthernet1.3 We have used a relatively simplistic configuration with a single router and each NetScaler acting as an OSPF path. As routers are generally limited to 16 OSPF paths, this design will likely be limited to 16 NetScalers. More advanced implementations will be able to expand on this simple design. 16 low-end appliances (such as the MPX9130) will equate to around 33 million L7 HTTP requests per second. Using high-end appliances (such as the MPX26200-50S), this design should service over 100 million L7 requests/sec at each datacenter. Note: You will have multiple routers in a production environment to meet redundancy and throughput requirements and may need to work with your network team to adjust the proposed configuration. Testing To test, our configuration, we will configure each NetScaler to return a different colored web page. The colors will make it immediately apparent which NetScaler is servicing our traffic.

Tech Paper: Networking SSL / TLS Best Practices February 21, 2023 Author: Steven Wright Overview This Tech Paper aims to convey what someone skilled in ADC would configure as a generic implementation to receive an A+ grade at Qualys SSL Labs. Qualys SSL Labs performs a robust series of tests and provides a scorecard that you can use to improve your configuration. The scan is free and only takes about a minute to complete. Qualys actively develops SSL Labs. Tests are likely to change as new protocols are created and vulnerabilities found. It is good practice to test sites regularly to make sure that any new vulnerabilities are not exposed. NOTE: Talk to your security team about your deployment. Security experts say SSL Labs A+ is a good general target, but it may not fit the needs of your organization. Configuration Items that Need to be Validated Certificates - Is the full chain provided and trusted? Is the signature algorithm secure? Protocols, Keys and Cipher Support - Which SSL and TLS protocol versions are supported? Which cipher suites are preferred and in what order? Do the provided cipher suites support forward secrecy? TLS Handshake Simulation - Determines which protocol and cipher are negotiated by several different clients and browsers Protocol Details - Is Secure Renegotiation supported? Is strict transport security (HSTS) supported? Known Vulnerabilities - Is the server vulnerable to attacks such as POODLE, BEAST, or TLS downgrade? Once SSL Labs completes testing, it presents a letter grade along with a point scale for each of 4 categories: 1 Certificate 2 Protocol Support 3 Key Exchange 4 Cipher Strength Each of the categories receives a numerical score that SSL Labs then averages into a total. Some special cases and configurations that SSL Labs recommends against, such as having SSLv3 enabled, can limit your final grade. You can find complete documentation on how SSL Labs grades servers here. Implementation Concerns SSL Profiles This article uses SSL Profiles. When first enabled, SSL Profiles sets all SSL virtual servers to use the default profile. SSL profiles take precedence over global and per virtual server SSL parameters. Client support Some of the configuration steps in this article can cause connectivity issues with old clients and browsers. For example, Internet Explorer 11 on Windows 7 and 8.1 only supports older cipher suites and, other older browsers may lack support for TLS1.2 and ECC ciphers entirely. In cases where support is missing, the client can experience error messages and an inability to display the site. SSL Labs have a “Test your browser” button on its front page to help determine your needs. Citrix Receiver/Workspace app Cipher Support for Gateway deployments Review the following articles regarding client cipher support when deploying a gateway virtual server for virtual apps and desktops: CTX234227 and CTX232266 for Citrix Receiver CTX250104 for Workspace app Note: See the Firmware Notes section for required builds and other notes regarding specific ADC firmware Basic Steps - GUI Take the following steps to ensure a high score on the SSL Labs test. Ensure that the ADC is running a recent firmware release - 13.0 build 71 or later is recommended to take advantage of TLS1.3 hardware acceleration Ensure that the certificate chain is complete and trusted Root CAs do not always directly sign certificates. Instead, a root CA often uses an intermediary to sign a certificate Install the intermediate certificate on the ADC. Link it to the server certificate you bound to the virtual server Intermediate certificates are provided by the vendor that provides the server certificate, often in a ‘certificate bundle’. They can usually be found on the vendor’s public site You may need to install and link multiple intermediate certificates. For the server certificate to function, the ADC must send a complete chain. A complete chain ends with a certificate signed by one of the client’s trusted root CAs As the client already has the trusted root CA, you don’t need to install and link it on the ADC To install an intermediate certificate, go to: Traffic Management > SSL > Certificates > CA Certificates and choose Install (Note: earlier builds of Citrix ADC do not have the ‘CA Certificates’ option in the GUI) Link an intermediate by selecting the certificate and choosing link from the action menu If the correct intermediate certificate is installed, it is automatically populated in the linking menu Create a custom cipher group that provides Forward Secrecy (FS) Go to Traffic Management > SSL > Cipher Groups and choose Add Name the cipher group “SSL_Labs_Cipher_Group_Q4_2021” Click Add then expand the ALL section - select the following cipher suites: TLS1.3-AES256-GCM-SHA384 TLS1.3-AES128-GCM-SHA256 TLS1.3-CHACHA20-POLY1305-SHA256 TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384 TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256 TLS1.2-ECDHE-ECDSA-AES256-SHA384 TLS1.2-ECDHE-RSA-AES256-GCM-SHA384 Click the > right arrow to move the ciphers from the Available column to the Configured column Click Create Enable SSL Profiles Navigate to Traffic Management > SSL > Change advanced SSL settings, scroll down, and select Enable Default Profile. SSL Profiles sets all SSL virtual servers to use the default profile when first enabled. As existing per virtual server SSL settings are removed, the ADC will prompt you to confirm. Create an SSL Profile Navigate to System > Profiles > SSL Profile, and select Add Name the profile “SSL_Labs_Profile_Q4_2021” Scroll to Deny SSL Renegotiation and select NONSECURE to allow only clients that support RFC 5746 to renegotiate Scroll to HSTS, tick HSTS, and specify a Max Age of 157680000 seconds Scroll to Protocol and select only TLSv12 and TLSv13 Scroll to the end of the form and select OK Scroll to SSL Ciphers, select the pencil icon to edit, then click Remove All Click Add and add the cipher group we created earlier Scroll to the end of the form and select Done Bind the SSL Profile to the SSL virtual server On the selected virtual server, select the pencil icon to edit the bound SSL Profile Select the SSL Profile we created from the drop-down list Click OK Basic Steps - CLI Take the following steps to ensure a high score on the SSL Labs test. In the CLI examples below, the name of the SSL virtual server is listed as Ex-vServer - it can be replaced with the name of the SSL virtual server in your environment. Create a custom cipher group that prefers ECDHE and ECDSA cipher suites add ssl cipher SSL_Labs_Cipher_Group_Q4_2021bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-AES256-GCM-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-AES128-GCM-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.3-CHACHA20-POLY1305-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-ECDSA-AES256-SHA384bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-ECDHE-RSA-AES256-GCM-SHA384 Enable SSL Profiles set ssl parameter -defaultProfile ENABLED Create an SSL Profile add ssl profile SSL_Labs_Profile_Q4_2021 -tls1 DISABLED -tls11 DISABLED -tls12 ENABLED -tls13 ENABLED -denySSLReneg NONSECURE -HSTS ENABLE -maxage 157680000 Unbind the default cipher group from the SSL Profile and bind the custom group unbind ssl profile SSL_Labs_Profile_Q4_2021 -cipherName DEFAULTbind ssl profile SSL_Labs_Profile_Q4_2021 -cipherName SSL_Labs_Cipher_Group_Q4_2021 Bind the SSL Profile to the SSL virtual server set ssl vserver "Ex-vServer" -sslProfile SSL_Labs_Profile_Q4_2021 More Settings SHA1 Certificates Certificates that are signed with SHA1 are considered weak, and prevents a high grade in the SSL Labs test. If any certificates are SHA1 signed, they should be replaced with an SHA256 certificate and installed on the ADC. DNS CAA DNS Certification Authority Authorization (CAA) allows CAs to validate if they are authorized to issue certificates for a domain and provide a contact if something goes wrong. DNS CAA is not required for an A+ at SSL Labs and is an optional step. You can find out more about DNS CAA here. Legacy client support The ECDHE ciphers we use in this guide replace the older slower DHE ciphers. If you have legacy clients that you cannot upgrade, you may have no choice but to enable DHE. Enable DHE cipher suites in the GUI Go to Traffic Management > SSL and select Create Diffie-Hellman (DH) key Name the Diffie-Hellman key “DH_Key_Name_Here.key” Enter the parameter size (Bits). Must be between 512 and 2048 Choose the Diffie-Hellman generator (2 or 5) Select Create Depending on the key size selected, this could take quite some time to complete NOTE: Do not navigate past this screen until it completes Navigate to System > Profiles > SSL Profile Select the SSL_Labs_Profile_Q4_2021 profile and choose Edit Select the pencil icon Scroll down to the Enable DH Param check box and select it Choose the DH key you just created Scroll down and select Okay Bind a DHE cipher suite to the cipher group that we created earlier Navigate to Traffic Management > SSL > Cipher Groups Select the TLS1.2-DHE-RSA-AES256-GCM-SHA384 cipher Click the arrow to add it to the Cipher Group Enable DHE ciper suites in the CLI Create and bind a DH key to the SSL Profile (CLI) create ssl dhparam DH_Key_Name_Here.key 2048 -gen 2set ssl profile SSL_Labs_Profile_Q4_2021 -dh ENABLED -dhFile DH_Key_Name_Here.key Bind a DHE cipher suite to the cipher group that we created earlier bind ssl cipher SSL_Labs_Cipher_Group_Q4_2021 -cipherName TLS1.2-DHE-RSA-AES256-GCM-SHA384 Firmware Notes Citrix enabled TLS1.2 as a default in firmware version 10.5 build 57 The ROBOT vulnerability was addressed in builds 12.0 build 53, 11.1 build 56, 11.0 build 71 and 10.5 build 67 - more details are available here The HSTS (Strict Transport Security) flag became available in 12.0 build 35 - prior builds required a rewrite policy to insert the HSTS header. You cannot use both as the ADC to insert 2 headers which is not allowed. Support for TLS1.2 was added to the VPX appliances in 10.5 build 57. It was available in earlier builds for appliances with dedicated SSL hardware Support for TLS1.3 was added in 12.1 build 49.23 - it must be enabled in the vServer SSL Parameters or added to an enhanced SSL Profile, and TLS1.3 ciphers (listed) must be bound. TLS1.3 is not supported on legacy SSL profiles ECC certificate support was added to the VPX appliances in 12.0 build 57. It was available in earlier builds for appliances with dedicated SSL hardware The Zombie POODLE vulnerability was addressed in builds 12.1 build 50.31, 12.0 build 60.9, 11.1 build 60.14, 11.0 build 72.17, and 10.5 build 69.5. This vulnerability only affects MPX\SDX appliances with Nitrox SSL hardware. MPX\SDX appliances with Coleto Creek are not vulnerable. Disabling CBC-based cipher suites also mitigates this vulnerability. See CTX article for more information The cipher list has been modified to address CBC weaknesses, thus removing 0xc028 and 0x39 ciphers Citrix added TLS1.3 hardware acceleration in 13.0 build 71

NetScaler Gateway is a powerful tool that provides secure remote access to applications and data. However, the process of setting up and deploying NetScaler Gateway can be complex and time-consuming. To help simplify this process, we are excited to share reference designs covering two of the three most common deployment scenarios. The designs provide a complete high-level and low-level reference for setting up a standard NetScaler Gateway implementation with LDAP+RADIUS or SAML authentication. You can use these designs as a template for your own deployments, to validate your existing configurations, or simply benefit from having access to every CLI command required in a copy/paste format found at the end of each design document. The first reference design covers the setup of NetScaler Gateway with LDAP+RADIUS authentication. It includes every NetScaler object and the CLI command required to deploy a production environment in under 30 minutes. This design is perfect for those who need to secure their applications and data using multi-factor authentication. Our second reference design covers the setup of NetScaler Gateway with SAML authentication using an existing identity provider. We used OKTA when writing the design but, the configuration should be compatible with any SAML IdP. As with our first reference design, every CLI command required is provided in a copy/paste format, ensuring an accurate, production-quality environment can be quickly deployed. These reference designs includes many best practices shared by Professional Services and Engineering. Whether you're a seasoned NetScaler professional or just getting started, we hope you'll find them packed with valuable insights and tips that can help you optimize your deployment. Download our reference articles here: Gateway with LDAP+RADIUS authentication Gateway with SAML Gateway as an OAuth IdP for integration with Citrix Cloud

Tech Paper: Best practices for NetScaler ADC Deployments Published on: October 20, 2021 Overview This Tech Paper aims to convey what someone skilled in ADC would configure as a generic implementation. Note: It is unlikely that there is a single configuration that suits everyone. A consultant or administrator with a better understanding of your needs may deviate from these defaults and document the reasons for such changes. Power and lights out management settings If you have purchased a NetScaler ADC appliance, you should ensure: The NetScaler ADCs are deployed in locations separate enough to meet your high availability requirements. The redundant Power Supply Units on each ADC (if you have purchased such) are connected to separate electrical supplies. The lights out management card (if you have purchased appliances with such a card) are configured. You can find more information on configuring lights out management cards here. Physical network cabling, VLANs, and connectivity 1. All physical interfaces connecting the NetScaler ADC to your networks are redundant To ensure data flow during a cable, switch, or interface failure, connect your ADC to each network with redundant cables. To combine the interfaces connecting each network into a single link (known as a channel), you must configure link aggregation on your ADC. When possible, it is preferable to use the Link Aggregation Control Protocol (LACP). However, manual aggregated links are also possible if your network switches do not support LACP. Instructions for configuring Link Aggregation on your NetScaler ADC can be found here In a virtualized or cloud environment, your provider handles interface redundancy, and this step is not required. 2. Unused physical interfaces are disabled Disable any physical interfaces that you are not using. Disabling unused interfaces prevents them from being connected to other networks or devices accidentally or maliciously. You can disable a physical interface by selecting System, Network, Interfaces. Then, by ticking the box next to the interface, clicking “Select Action”, followed by “Disable”. 3. Set all redundant physical interfaces to be unmonitored for HA within System, Network, Interfaces As each network has redundant connections, it is not desirable that the ADC initiates an HA failover when a single link fails. Instead, the ADC should rely on the remaining link and only trigger a failover if all links fail. To prevent an HA failover when a single interface in a redundant channel fails, mark the component interfaces as unmonitored. To mark an interface as unmonitored, select System, Network, Interfaces. Then, select each interface that forms a part of each redundant channel and set the “HA Monitoring” radio button to “OFF”. In a virtualized or Cloud environment, you have virtual interfaces and don’t need to perform this step as your provider handles interface redundancy. 4. All channels comprising redundant physical interfaces should be monitored for HA within System, Network, Channels The failure of all aggregated links connecting the ADC to a particular network causes the channel representing those links to enter a failed/DOWN state. Check that monitoring has been enabled for the channel and the ADC will failover if all redundant links fail. To mark a channel as monitored, select System, Network, Channels. Then, select each channel and set the “HA Monitoring” radio button to “ON”. 5. All channels are bound to a separate VLAN and, you have taken care that no untagged channels are accidentally still in VLAN 1 Each redundant channel ordinarily represents the aggregate physical links connecting the ADC to a particular logical network. In a virtualized or Cloud environment, each interface likely represents aggregate physical links with your provider having completed the aggregation work for you. By default, the ADC considers all interfaces, channels, and IP addresses as being VLAN 1 and the same logical network. As such, overlooking the VLAN configuration would cause all IP address assigned to the ADC to be available from every directly connected network. To prevent this behavior, configure VLANs on the ADC to represent your logic networks and appropriately isolate traffic. You can find instructions to create VLANs here. 6. Create an HA pair between your ADCs within System, High Availability NetScaler considers it a best practice to deploy ADCs redundantly. You can achieve redundancy by implementing an HA pair, creating a cluster, or using a technology such as GSLB to split requests between instances. HA pairs comprise two ADC nodes, and clusters can have up to 32-nodes. For a generic implementation, NetScaler recommends the creation of a two-node HA pair. You can find instructions for configuring an HA pair here 7. Create and bind one SNIP to every VLAN, ensuring that each SNIP is in the subnet of the connected network NetScaler ADC initiates communication from a Subnet IP (called a SNIP) with limited exceptions. Create one Subnet IP/SNIP for every directly connected logical network. As you have already isolated each network using VLANs, you must bind each SNIP to its respective VLAN. Take care to ensure that no VLANs are missing a SNIP. The ADC identifies which VLAN each Virtual IP (VIP) needs to be placed in based on the SNIP’s subnets. The VLAN configuration then isolates virtual servers within their intended network. You can find instructions for configuring SNIPs here. Note: SNIPs host management services by default. To create SNIPs without the management service enabled, append the “-mgmtAccess DISABLED” parameter to the “add ns IP” command. 8. Configure the routes that the ADC requires within System, Network, Routes If you have connected multiple logical networks, you likely have routers in each. Therefore, you must now configure all the routes that the ADC requires to reach its clients and back-end servers. You can find instructions for configuring routes here. Note: The ADC has a single routing table that applies to all interfaces. 9. Create any Policy Based Routes required Occasionally, configuring a static route to provide for the behavior you desire is impossible. An ADC with distinct ingress, egress, and dedicated management networks, as well as management clients on the egress network, is the most frequent example. Static rules would not suffice in this case. Instead, a Policy-Based Route would be required (PBR). By using a PBR, you can force traffic from the ADC’s management IP to travel through the management router. Using a PBR will bypass the static routing table, which would otherwise send data to the egress network. You can find instructions for configuring Policy Based Routes here. However, if you have the scenario described in our example, you need the following PBR: add ns pbr Management ALLOW -srcIP = <NSIP_of_first_HA_node>-<NSIP_of_second_HA_node> -destIP "!=" <first_IP_management_subnet>–<last_IP_management_subnet> -nextHop <management_subnet_router> - priority 1apply pbrs 10. Make sure you can ping each SNIP with Mac Based Forwarding (MBF) is disabled or that you understand why you cannot NetScaler ADC has a mode called Mac Based Forwarding (MBF). MBF causes the ADC to ignore the routing table and to send replies to the MAC address from which it received traffic. MBF is of great use where you cannot define your routes. Let’s say the ADC has multiple Internet connections and must reply using the Internet router through which the traffic arrived. Here, MBF would cause the ADC to record the source MAC address of each connection and use this as the destination MAC in its reply. However, having MBF override the routing table can make troubleshooting more complex. With MBF, you cannot understand traffic flows from the ADC’s configuration file alone as MBF overrides the routing table, and traffic may not flow as you intended. The result is that MBF, while crucial to some implementations, can also lead to misconfigurations in the supporting network remaining undetected. Disable MBF to ensure that your routing table and PBRs are correct. After disabling MBF, verify that each SNIP remains reachable or that you understand why it does not. The command to disable MBF is disable ns mode mbf The command to enable MBF is enable ns mode mbf If you do not require MBF, leave it disabled. Note: In Azure Deployments, MAC-based Forwarding (MBF) is enabled by default You can find more information of Mac Based Forwarding here. 11. You have installed a new SSL certificate and key for the management GUI within Traffic Management, SSL, Certificates Web browsers do not trust the NetScaler ADC’s default SSL certificate. The lack of trust causes browsers to display a warning message when accessing the ADC’s management services. Browser warnings for certificates are intended to alert users when the connection is not secure. NetScaler recommends that you replace the default SSL certificate so that management users do not become accustomed to accepting warning messages. You can find details of how to replace the management SSL certificate here. Note: As NetScaler ADC shares the management SSL certificate between HA nodes, your replacement certificate must be trusted for all FQDNs used for management purposes. NetScaler recommends using a SAN certificate to include the FQDN of both HA nodes. Base configuration settings 1. Set the timezone and enable NTP Having accurate and easily understood timestamps in your log files is vital when troubleshooting or handling a security issue. First, set the timezone to something that makes sense for you. For example, if you have devices that log to a central syslog server and need to cross-reference data from each, use the same timezone as your existing servers. The command to set the timezone is: set ns param -timezone CoordinatedUniversalTime Second, add NTP servers and enable time synchronization using the commands: add ntp server pool.ntp.orgenable ntp sync After enabling time synchronization, view the NTP status to verify correct behavior by using the following command: nsroot@StevensADC-Primary> show ntp status remote refid st t when poll reach delay offset jitter==============================================================================*any.time.nl 85.199.214.99 2 u 113 1024 377 21.138 +0.762 0.654 Done nsroot@StevensADC-Primary> You can find details of how to the timezone using the ADC’s GUI here. You can find details of how to add NTP servers here. 2. Create a Key Encryption Key A Key Encryption Key (commonly known as a KEK) is used to encrypt and decrypt credentials that the ADC must store in a reversible form. For example, the ADC must keep its LDAP bind password reversible to retrieve it at authentication time. From NetScaler ADC firmware 13.0–76.31, the ADC automatically creates a KEK. On later firmware, the command returns an error message that you can safely ignore. On older releases, you can create a KEK with the command: create kek <RANDOMSTRING> 3. Set a non-default nsroot password Recent releases of NetScaler ADC firmware prompt you to change the default password for the “nsroot” account when you first log in. Recent firmware builds prompt for the password change as the “-forcePasswordChange” system parameter is enabled. For older releases, you must change the “nsroot” password with the command: set system user nsroot -password <NSROOTPASSWORD> 4. Add an account for ADM with external authentication disabled Ideally, connect all of your NetScaler ADCs to ADM for centralized licensing and management. The connection to ADM requires a user name and password which can be created with the following commands: add system user admuser <ADMPASSWORD> -externalAuth DISABLED -timeout 900bind system user admuser superuser 100set system user admuser -externalAuth DISABLED You can find more details about ADM here. 5. Restrict non-management applications access to the NSIP and only HTTPS access Prevent non-management services from accessing the management IP and set the management IP to require secure communication access (HTTPS rather than HTTP). set ns ip NSIP -restrictAccess enabled -gui SECUREONLY You can find more details on restricting access to the NSIP here. 6. Set a non-default RPC node password Set RPC communication (used for HA and GSLB) to use a non-default password. set rpcNode <NSIP_OF_SECONDARY_NODE> -password <RPC_SECONDARY_PASSWORD> -secure YES set rpcNode <NSIP_OF_PRIMARY_NODE> -password <RPC_PRIMARY_PASSWORD> -secure YES 7. HA failsafe mode is enabled to ensure that the last healthy node continues to provide service If an ADC’s monitored HA interfaces or routes indicate an error, the ADC enters an unhealthy state and trigger an HA failover. If the second HA node enters an unhealthy state, both stop providing service. HA fail-safe mode ensures that the last surviving node of a pair continues attempting to provide business services. set HA node -failSafe ON You can find more details on HA fail-safe mode here. 8. Restrict HA failovers to 3 in 1200 seconds In the unlikely event that HA failovers repeatedly occur, there comes the point where you want them to stop and to attempt to provide service. Here, we define a limit of three HA failovers within a 1200 second (20 minutes) period. set ha node -maxFlips 3set ha node -maxFlipTime 1200 9. Disable SSLv3 for management services The NetScaler ADC management GUI has SSLv3 and TLS1.0 enabled by default. To ensure secure communication, we will disable SSLv3. set ssl service nshttps-::1l-443 -ssl3 disabledset ssl service nshttps-127.0.0.1-443 -ssl3 disabled Depending on your company’s internal security policy, you can optionally disable TLS1.0. set ssl service nshttps-::1l-443 -ssl3 disabled -tls1 disabledset ssl service nshttps-127.0.0.1-443 -ssl3 disabled -tls1 disabled 10. Set generic modes and features NetScaler ADC has Layer 3 mode enabled by default. Layer 3 mode causes the ADC to act as a router and can normally be safely disabled. Edge mode causes the ADC to dynamically learn details about back-end servers when used in a configuration such as link load balancing. disable ns mode l3 edge You can find more details on Layer 3 mode here. The specific modes and features that you need depend on your use case. However, we can select a list of options that would apply to most installations. enable ns feature lb ssl rewrite responder cmp You can find more details about modes and features here. 11. Configure one or more DNS nameserver The NetScaler ADC needs to have access to one or more nameservers for DNS resolution. NetScaler ADC checks if the DNS servers are online using an ICMP monitor. To use DNS based monitoring and distribute load, it is usual to implement a local DNS load balancing virtual server. As DNS can use UDP or TCP, we create one load balancing virtual server for each protocol. Configure nameservers using the following commands: add lb vserver DNS_UDP DNS 0.0.0.0 0 -persistenceType NONE -cltTimeout 120 add serviceGroup DNS_UDP_SVG DNS -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport NO -cltTimeout 120 -svrTimeout 120 -CKA NO -TCPB NO -CMP NObind lb vserver DNS_UDP DNS_UDP_SVG add lb monitor DNS_UDP_monitor DNS -query . -queryType Address -LRTM DISABLED -interval 6 -resptimeout 3 -downTime 20 -destPort 53bind serviceGroup DNS_UDP_SVG -monitorName DNS_UDP_monitor bind serviceGroup DNS_UDP_SVG <DNSSERVERIP1> 53bind serviceGroup DNS_UDP_SVG <DNSSERVERIP2> 53 add lb vserver DNS_TCP DNS_TCP 0.0.0.0 0 -persistenceType NONE -cltTimeout 120 add serviceGroup DNS_TCP_SVG DNS_TCP -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport NO -cltTimeout 120 -svrTimeout 120 -CKA NO -TCPB NO -CMP NObind lb vserver DNS_TCP DNS_TCP_SVG add lb monitor DNS_TCP_monitor DNS-TCP -query . -queryType Address -LRTM DISABLED -interval 6 -resptimeout 3 -downTime 20 -destPort 53bind serviceGroup DNS_TCP_SVG -monitorName DNS_TCP_monitor bind serviceGroup DNS_TCP_SVG <DNSSERVERIP1> 53bind serviceGroup DNS_TCP_SVG <DNSSERVERIP2> 53 add dns nameServer DNS_UDP -type UDPadd dns nameServer DNS_TCP -type TCP 12. Set TCP and HTTP parameters While Window Scaling (WS) and Selective Acknowledgment (SACK) now default to enabled in ADC 13.0 firmware, you must enable these TCP settings in earlier firmware versions. set ns tcpparam -WS ENABLEDset ns tcpparam -SACK ENABLED Nagle causes the NetScaler ADC to combine data to send a smaller number of larger packets and can be enabled with the following command. set ns tcpparam -nagle ENABLED By default, NetScaler ADC forwards HTTP requests that arrive at a load balancer but do not conform to the RFC standard. Configure the ADC to drop invalid requests as a default. To allow exceptions to the default, you can change the HTTP options on an individual virtual server after discussions with your security team. Disable support for the HTTP/0.9 protocol, which has been obsolete for over 20 years. For reference, Mosaic 2.0 on Windows 3.1 includes support for HTTP/1.0. set ns httpparam -dropInvalReqs ENABLED -markHttp09Inval ON Cookie version 0 includes absolute timestamps, whereas version 1 cookies have a relative time. Cookies with an absolute timestamp do not expire at the expected time if the client and ADC clock differ. However, cookies with a relative time of +X minutes until expiry will. Internet Explorer 2 included support for version 1 cookies in 1995, and you are unlikely to experience issues by enabling this option. set ns param -cookieversion 1 13. Restrict SNMP queries to select servers It is good practice that the ADC only answers SNMP queries from hosts supposed to make such queries. Limit the hosts that the ADC allows SNMP queries from using the command: set snmp manager SNMPMANAGERIP You can find more details on configuring SNMP here. 14. Set SNMP alarms and traps It is helpful for the ADC to raise alerts when high CPU or memory usage conditions occur. You can send alerts via trap configuration to your SNMP server. You may also want to trigger alerts when HA failovers occur. You can implement such configuration with the following commands: set snmp alarm CPU-USAGE -state ENABLED -normalValue 35 -thresholdValue 80 -logging ENABLED -severity Informationalset snmp alarm MEMORY -state ENABLED -normalValue 35 -thresholdValue 80 -logging ENABLED -severity Criticalset snmp alarm HA-STATE-CHANGE -severity Critical add snmp trap generic SNMPTRAPDSTIP -communityName public Note: NetScaler recommends that you consider periodically reviewing threshold values to ensure that the ADC alerts on abnormal behavior that you want to investigate. 15. Set a remote syslog server Audit logging should be configured on an ADC, and audit logs should be stored and analyzed on a remote server. You can configure the NetScaler ADC to send audit logs to a remote Syslog server using the following commands: add audit syslogAction RemoteSyslogServerAction SYSLOGSERVERIP -loglevel ALLadd audit syslogpolicy RemoteSyslogServerPolicy true RemoteSyslogServerActionbind audit syslogGlobal -policyName RemoteSyslogServerPolicy -priority 100 [You can find more details about audit logging here] (/en-us/citrix-adc/current-release/system/audit-logging.html) 16. Set a timeout and prompt for management sessions NetScaler ADC 13.0 allows a default of 900 seconds (15 minutes) before disconnecting idle management sessions. On older firmware versions, you must ensure that you have configured an appropriate timeout. set system parameter -timeout 900 An administrator can open SSH sessions to multiple ADCs at the same time. Changing the ADC’s CLI prompt helps clarify the node to which a session is connected. The following commands cause the CLI prompt to display their username, the ADC’s host name, and the HA status of the instance. set system parameter -promptString %u@%h-%s After you run the command, the prompt will be rendered as: nsroot@hostname-Primary> 17. Centralized authentication for management accounts Security teams generally consider it better to control management accounts from a central platform such as Active Directory than to create accounts on each device. Usually, these centralized accounts are then granted permissions based on group memberships. The rationale for centralized authentication and authorization is usually that managing accounts on each device is time-consuming and prone to error. Also, there is the risk that management users not change their passwords frequently and that IT can forget to remove ex-employees’ accounts. Use the following commands to configure centralized authentication, keeping in mind that the LDAP filter controls who are allowed to log in. add authentication ldapAction LDAP_mgmt_auth -serverIP <LDAPMANAGEMENTSERVERIP> -serverPort 636 -ldapBase "<dc=mycoolcompany,dc=local>" -ldapBindDn "<serviceaccount@mycoolcompany.local>" -ldapBindDnPassword <LDAPPASSWORD> -ldapLoginName <sAMAccountName> -searchFilter "&(|(memberOf:1.2.840.113556.1.4.1941:<cn=NetScaler-ADC-FullAccess,ou=groups,dn=mycoolcompany,dc=local>)(memberOf:1.2.840.113556.1.4.1941:<cn=NetScaler-ADC-ReadOnly,ou=groups,dn=mycoolcompany,dc=local>))" -groupAttrName memberOf -subAttributeName cn -secType SSL -passwdChange ENABLED -nestedGroupExtraction ON -maxNestingLevel 5 -groupNameIdentifier samAccountName -groupSearchAttribute memberOf -groupSearchSubAttribute CN add authentication Policy LDAP_mgmt_pol -rule true -action LDAP_mgmt_authbind system global LDAP_mgmt_pol -priority 100 While these commands implement authentication, they do not control authorization and, by default, the authenticated users are not able to perform any actions. To grant the user (or more accurately, the group of which the user is a member) the right to perform actions on the ADC, you should use the following commands: add system group NetScaler-ADC-FullAccess -timeout 900add system group NetScaler-ADC-ReadOnly -timeout 900bind system group NetScaler-ADC-FullAccess -policyName superuser 100bind system group NetScaler-ADC-ReadOnly -policyName read-only 110 You can find more information about centralized authentication and authorization here. You can also find information about the LDAP filter string used above here. Further, from ADC firmware version 12.1.51.16 you can configure multifactor authentication for management users by following the steps here. 18. Disable LDAP authentication for the nsroot user As the NetScaler ADC processes authentication and authorization separately, users can authenticate using LDAP, and the ADC grants permissions based on their group membership. While not a good idea, you could also create user accounts with authorization permissions on the ADC. A password associated with an Active Directory user with the same name could then be used to authenticate these accounts. To prevent an Active Directory administrator from creating an “nsroot” user and being able to authenticate, you must disable external authentication for the “nsroot” user account. set system user nsroot -externalAuth DISABLED 19. TLS/SSL Best practices You can now follow the TLS/SSL Best Practice document to define a secure cipher suite that can be used to protect your virtual servers. You can find the TLS/SSL best practice document here.

Objective Post going thought this article one will be able to understand on how to: Create an ADC LB configuration Template with required set of configuration command. Convert Configuration Template with variables. Push the configuration to one or more Citrix ADC using the said configuration template. Before we start with the main content of this article, let us understand some basics on configuration Job, Citrix ADM Service and its onboarding. What is a Configuration Job? A configuration job is a feature in ADM Service which allows users to create a configuration template that can be run across one or multiple managed Citrix ADC instances. What is Citrix ADM Service? Citrix Application Delivery Management (ADM) Service is a central control and visibility SaaS solution for managing all Citrix deployments that includes Citrix ADC MPX/ VPX/ SDX/ CPX/ BLX and Citrix Gateway appliances that are deployed on-premises or on the cloud. To read and understand more on what all are benefits and features can be leveraged from Citrix ADM service, kindly follow the link below: https://docs.citrix.com/en-us/citrix-application-delivery-management-service/overview.html How to Quickly onboard on ADM service? If you are new to Citrix ADM service, you can easily onboard the Citrix ADM service in three easy steps: Create account in Citrix Cloud. Click ADM service tile on Citrix Cloud to select built-in agent mode. Use the service URL and the activation code to add Citrix ADC on ADM service. Here is the link to a small YouTube video which guide through above mentioned steps: And more details over same can be found here at our Citrix Documentation: https://docs.citrix.com/en-us/citrix-application-delivery-management-service/getting-started/initiate-built-in-agent.html Workflow Overview Let’s create a basic configuration example wherein one wants to load balance and send a secure end to end TLS traffic to its backend server along with Source IP accounting, based on request URL path's content via Citrix ADC. In order to achieve this, we will follow given steps: Add a service group with two services with Client IP forwarding to backend. Disable SSL v3 protocol to deny any SSLV3 request towards backend servers. Bind server to service group. Add a non-addressable load balancing virtual server. Disable SSL v3 protocol to deny any SSLV3 request towards load balancing virtual server. bind the service group to the virtual server. Add a content switch action with target as a load balancing virtual server Add a content switch policy which contains expression rule to direct the request to a load balancing virtual server based on request URL path's content. Add a content switch virtual server. Disable SSL v3 protocol to deny any SSLV3 request for content switch virtual server bind the content switch policy to the content switch virtual server. As stated at start that configuration job is a set of configuration commands that can be run on one or more managed ADC instances. When one runs the same configuration on multiple instances, they might want to use different values for the parameters used in the configuration. One can define variables that enables to assign different values for these parameters or run a job across multiple instances. Variables usually are the IP address, entity name etc in the configuration. Create an ADC configuration Template with required set of configuration command Below is the sample CLI command which we are going to use for configuration of ADC using ADM service Configuration Job feature. add serviceGroup mylb_SVG:443 SSL -maxClient 0 -maxReq 0 -cip ENABLED X-Forwarded-For set ssl serviceGroup mylb_SVG:443 -ssl3 DISABLED bind serviceGroup mylb_SVG:443 10.11.12.13 443 bind serviceGroup mylb_SVG:443 10.14.15.16 443 add lb vserver mylb_VS:443 SSL 0.0.0.0 0 -persistenceType COOKIEINSERT -cltTimeout 180 set ssl vserver mylb_VS:443 -ssl3 DISABLED bind ssl vserver mylb_VS:443 -certkeyName servercert_Mar2022 bind lb vserver mylb_VS:443 mylb_SVG:443 add cs action mylb_cs_action_443 -targetLBVserver mylb_VS:443 add cs policy mylb_cs_policy_443 -rule "HTTP.REQ.URL.PATH_AND_QUERY.STARTSWITH (\"/test\") || HTTP.REQ.URL.PATH_AND_QUERY.STARTSWITH (\"/default\")" -action mylb_cs_action_443 add cb vserver mylb_CSVS:443 SSL 10.17.18.19 443 bind ssl vserver mylb_CSVS:443 -certkeyName servercert_Mar2022 set ssl vserver mylb_CSVS:443 -ssl3 DISABLED bind cs vserver mylb_CSVS:443 -policyName mylb_cs_policy_443 -priority 100 Note: In the above example for command number Seven (7) and (12), I am binding a certificate already present on Citrix ADC. Please follow the “Install SSL certificate on Citrix ADC Instance” section on Citrix ADM documentation covering steps on how to install a Server certificate on a managed Citrix ADC using Citrix ADM service. To create a configuration template: Select the section which the service group name “mylb_SVG:443” to convert it to a variable. Once the dollar sign appears enclosing the variable’s value, bring cursor over green dollar enclosed variable and click on it to edit. On righthand side small window under “Define Variable” section, edit and add in Name, Display Name and Keep “Type” field as “Text Field” and click on Done. Similarly, as per step 6 convert the Service Group name into variable for first 4 command with reference to service group name. Change the backend service IP address into variable with a unique names and the field “Type” to be set as “IP Address Field”. Follow the same steps to define the variable for load balancing virtual sever, Content switch action, content switch policy and content switch virtual server PS: Please find below location to download the config template’s json file created for this configuration job workflow here: https://citrix.sharefile.com/d-sab5397f46b644209a75776f4ec07dc2f This can be used as reference template by importing it on ADM service Configuration template page. In order to upload a template, navigate to Infrastructure > Configuration > Configuration Jobs > Configuration Template and Click on “Import” to upload the configuration json file by navigating to file’s saved location from the local desktop/laptop. Push the configuration to one or more Citrix ADC using the said configuration template Navigate to Infrastructure > Configuration > Configuration Jobs. Click on “Create Job” if this is your first time to this page. and if already have other configuration jobs, then click on “Create Job” as shown below. On the Create Job page, select the custom job parameters such as the name of the job, the instance type, and the configuration type. Within Configuration Editor, the “Configuration Source” will by default be showing “Configuration Templates” which further shows all the custom configuration templates created. Drag and drop the configuration template “CSVserver_Template” created earlier or Click on “+” sign to add it. Click on “Preview Variable” to see all the eight (8) variables that has been defined so far. Click on “Next” button. Under “Select Instances” tab, click on “Add Instance” to add the ADC Instance(s) where the selected configuration needs to be pushed. Click on “Next” button. In the “Specify Variable Values”, one need to add the values for all the variables that are part of configuration template. In our configuration sample as quoted earlier eight variable (step#5) needs to have their values filled in. To perform this step, select the “Upload input file for variable values” and click on “Download Input Key File” to download a csv file which needs to be filled with values and uploaded back. Above step will download the following csv file with all the variable for which values needs to be filed in. Define the values for all variables against the instance(s) and save the file. Note: In case of multiple instances selected for which same configuration need to be pushed one need to input values for respective variable. Go back to ADM service page and on the “Specify Variable Values” tab upload the csv file by navigating to file’s saved location on local desktop/laptop. PS: If the file upload has all the required fields filled up a success message will pop-up on top of ADM service web browser stating: “Input key file “CSVIP_config_job_variable_input_key_file_csv” uploaded successfully.” Click on “Next” button and briefly one can see “Fetching Job Preview” message on screen On the “Job Preview” one can select the individual Instance from “Select an Instance to preview” to view the config that will pushed under this configuration job. Click on “Next” button. Under “Execute” Tab, once can select to execute the “Now” or schedule it for “Later” by selecting under “Execution Mode”. One can also choose what action Citrix ADM service must take if the command fails “On Command Failure” and if one would like to send an Email/Slack notification regarding the success or failure of the job along with other details. Click on “Finish”. PS: In this workflow, job is selected to be executed immediately by ignoring errors. Citrix ADM Service will take back to “Jobs” page and there it will the live progress in percentage. Once the job is executed successfully you will see the “Completed” status under “EXECUTION STATUS” On the “Jobs” page click on “Details” button and then on “Variable Details” to view the details of variables added to the selected Job Details of configuration job executed can be downloaded by clicking on “Download” under “ACTIONS”

Rapidly migrate your NetScalers in under an hour using ADM Service NetScaler migration Have you bought a new NetScaler with more throughput or is it time to refresh? Are you migrating from old NetScalers to new ones? Sounds difficult? Then you've stumbled across the right article. Installing new NetScalers into production and migrating services may seem like a complicated process, but once you know how to do it, it's much easier than you might think. This blog post will walk you through the steps needed. In this article, we will assume that you have an existing source NetScaler that's running in production and a new target NetScaler that you want to prepare in a non-production environment and migrate to under a change control process. Prepare for the migration (existing NetScaler) ADM Service's migration feature requires NetScaler firmware version 13.1.17 or above. To prepare for the upcoming migration, upgrade your existing NetScalers to 13.1.17 firmware so that both the source and target NetScaler have the same firmware version and support for ADM Service's migration feature. NetScaler recommends upgrading to the same firmware on the source and target NetScalers to minimize the number of changes during the migration and, in the unlikely event of any issues, to identify the cause more easily. You can upgrade your existing NetScaler to 13.1 firmware using the instructions here. Prepare for the migration (target NetScaler) First, install your new target NetScaler into a non-production environment without a connection to the production network. Ensuring the non-production and production networks are isolated from each other will prevent unintended interactions such as IP address conflicts. Once you have installed the new NetScaler, connect it to ADM Service. To do this, you need to add an ADM Agent into the non-production environment. You can install a new ADM Agent using the instructions here and add you can connect the new NetScaler to ADM Service using these instructions. Next, install a license onto your new NetScaler. You can apply a pooled license to your new NetScaler using the instructions here. Replicate the configuration After ensuring that the new NetScaler is not connected to any production networks, follow these steps to replicate the configuration. 1. Select Infrastructure, Instances, NetScaler, and NetScaler form factor In the screenshot, we have selected the "VPX" form factor and can see two NetScalers, "192.168.6.30" and "192.168.1.230". In this article, we will use "192.168.6.30" as the source NetScaler and will replicate its configuration onto the target NetScaler on "192.168.1.230". 2. Tick the source NetScaler, click "Select Action", and then choose "Replicate Configuration" 3. Move the target NetScaler instance from the "Available" to "Configured" window using the arrow button and click "Create" ADM Service will now replicate the configuration from your source to target NetScaler. Note: Any existing configuration on the target instance will be erased and replaced with the configuration of the source NetScaler. The source and target NetScalers should be on separate networks to prevent IP conflicts. Validate the network configuration on the target NetScaler Although the source and target NetScalers will now have the same configuration, the management NSIP and network interface layout may differ. If you installed your new NetScaler into the non-production environment using a network interface layout that you don't want to use in production, you should change they layout now. To change the network configuration, log into the target NetScaler and ensure: That all unused interfaces are disabledThat you have created LACP channels as you requireThat a SNIP is bound to each VLANThat each VLAN is bound to an interface or channel using your intended network layoutYou can find instructions detailing how to disable unused interfaces, create LACP channels, and how to bind SNIPs and VLANs here. Replace the NSIP on the target NetScaler If you installed your new NetScaler into the non-production environment using an NSIP that differs from production and would prefer it was the same at the point of migration, you should change the NSIP now. You can change the NSIP using the following steps. Note: If you are migrating an HA pair, you must perform the commands below on both nodes at the same time and shutdown both nodes at the same time. Select the target NetScaler within the "Infrastructure, Instances, NetScaler" menu of ADM, choose "SNMP" from "Select Action", then disable SNMP.Note: We are disabling these services to ensure they are not configured to communicate with the pre-production ADM Agent, we will re-enable them after the migration is complete. Using the same process as "step 1" (above), disable Syslog and Analytics on the target NetScaler. Select the source NetScaler and repeat steps one and two (above) to disable SNMP, Syslog, and Analytics. Log into the target NetScaler's command line interface using SSH. You can find details of how to use SSH here. Change the NSIP of the target NetScaler using the commands shown in red. shellcd /nsconfigcp ns.conf ns.oldsed -i -e 's/192.168.1.230/192.168.6.30/g' ns.confNote: 192.168.6.30 is the source NetScaler in our example and 192.168.1.230 is the target NetScaler. You must replace the IP addresses within the command with those of your source and target NetScaler. Change references to the NSIP of the other HA node if applicablesed -i -e 's/192.168.1.231/192.168.6.31/g' ns.confNote: You will run the sed command twice on both HA nodes. Firstly, to replace the NSIP of the current node. Secondly, to replace all references to that changed NSIP on its HA partner. Shutdown the NetScalershutdown nowNote: If you shutdown the NetScaler using an alternative command then you should reply "no" when ask if you wish to save the running configuration. The output will look like the text below. Note: In this example, we have included an example HA node with an NSIP of 192.168.1.231 that needs to be replaced with 192.168.6.31 - the replacement needs to be executed on both HA nodes. ############################################################################### # # # WARNING: Access to this system is for authorized users only # # Disconnect IMMEDIATELY if you are not an authorized user! # # # ############################################################################### Password: Last login: Mon Dec 6 14:26:05 2021 from 192.168.6.20 Done nsroot@-Primary> shell Copyright © 1992-2013 The FreeBSD Project. Copyright © 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. root@ns# cd /nsconfig root@ns# cp ns.conf ns.old root@ns# sed -i -e 's/192.168.1.230/192.168.6.30/g' ns.conf root@ns# sed -i -e 's/192.168.1.231/192.168.6.31/g' ns.conf root@ns# shutdown now Shutdown NOW! shutdown: [pid 8140] root@ns# *** FINAL System shutdown message from root@ns *** System going down IMMEDIATELY System shutdown time has arrived Connection to 192.168.6.30 closed by remote host. Replace the NetScaler under change control With the existing source and target NetScalers now having identical configurations, you can replace them in compliance with your change control procedures. Many customers achieve that change using a two-stage replacement process. In the first stage, they work with their network teams to connect the new NetScalers to network switch ports that are administratively disabled or offline. In the second stage, during a change control window, the network team places the switch ports to the old NetScalers into a disabled or offline state, brings the interfaces connecting the new NetScalers online, and flushes the ARP cache on their router. The effect is that all hardware installation and readiness activities can be completed in-advance and the change control window to move from the old to new appliances is completed in minutes. This approach also provides an excellent rollback strategy as the network could disable the interfaces connecting the new NetScaler and reenable those for the old, which are still powered on and running without modification. Re-establish connectivity to ADM Service You have now replaced the source NetScaler with the target. ADM will show the source as online (as the target is using the source NetScaler's NSIP) and the target's old NSIP as offline. You should now complete the following steps to ensure the migrated NetScaler is correctly communicating with ADM Service. To ensure ADM is communicating with the new NetScaler that has taken over the NSIPs of the source, select the source NetScaler within "Infrastructure, Instances, NetScaler" menu of ADM and, choose "Rediscover" from "Select Action". After the rediscovery completes, re-enable the SNMP, Analytics, or Syslog capabilities that you disabled at step one of "Replace the NSIP on the target NetScaler". To enable each, select the NSIP of the source NetScaler and the capability under "Select Action". If you are using pooled licensing on the new NetScaler, you must also ensure it is using the production ADM Agent as its license server. To do this, log into the new NetScaler using SSH and enter the following commands:sh nslicenseserver If the new NetScaler is not using the production ADM Agent as its license server, you can correct the license server to production ADM using the following commands:add nslicenseserver <prod_agent_ip> -port 27000 -forceUpdateIP Remove the target NetScaler's old NSIP from ADM by ticking the old NSIP within the "Infrastructure, Instances, NetScaler" menu of ADM and pressing "Remove". Next steps You have now finished migrating from your old to new NetScalers and can begin using them in production.

Sometimes it can be helpful to limit the number of people interacting with backend servers. For example, when selling festival tickets or handling purchases on the first day of a product launch, you may want to restrict the number of people talking to your servers so you can quickly complete sales.Usually, load balancers track the total number or rate of TCP or HTTP connections. Here, we want to rate limit another metric, the number of concurrent people, as tracked by session cookie. NetScaler can store variables such as session cookies and reference them with policy logic. In this article, we will demonstrate how to use variables and policy expressions to limit the number of people able to access your store. Thanks to Ibrahim Mubasher at DU Telecom, who helped develop this article. Note: we will assume you already have a queuing system that presents a friendly screen to waiting users and that all users will arrive with a JSESSIONID cookie. What is an HTTP Session? An HTTP session is the traffic, context, and state of a conversation between client and server. For example, a web browser opening a website may request many pages, images, and resources over several TCP connections. All these connections and requests would be a part of that session. Initially, it may seem that we can reduce the load on the NetScalers by rate-limiting TCP connections and backend servers by limiting HTTP requests. However, users will likely make repeated requests for web pages, images, etc., until they've purchased their newly released gadget or festival ticket. To be effective, it's the concurrent sessions that we need to limit. Online stores typically track users with session cookies. A session cookie is a cookie that expires once the current session ends. Stores usually use these cookies to maintain state, track shopping baskets, and bill for the correct items. Here, we will track sessions using the "JSESSIONID" cookie produced by J2EE web applications, but you could use any session cookie. How can we limit the number of HTTP sessions?First, we will create a variable map on the NetScaler that can store 10 32-character keys and a ulong value for each. We will set the variable to expire keys that the NetScaler hasn't requested in the last 300 seconds. add ns variable JID_Session_Map -type "map(text(32),ulong,10)" -ifFull undef -expires 300 We will use the map variable to store the content of the "JSESSIONID" cookie as a key. We can conceptually visualise that map as a spreadsheet with ten keys and a value for each. The map needs to be large enough to store all sessions the NetScaler keeps track of; in this article, we will allow ten sessions. We don't plan to use the value for each key and will set it to "1". KeyValueJSESSIONID11JSESSIONID21JSESSIONID31 Next, we will define two assignment actions that responder policies can call to add and remove keys from the variable map. add ns assignment JID_Add_Session -variable "$JID_Session_Map[http.req.cookie.value(\"JSESSIONID\")]" -set 1 add ns assignment JID_Delete_Session -variable "$JID_Session_Map[http.req.cookie.value(\"JSESSIONID\")]" -clear To trigger the assignment action to add a JSESSIONID cookie to our map variable, we will create a responder policy that evaluates true when the user requests a login page and their JSESSIONID cookie does not already exist as a key in the map. We will add a second responder policy that evaluates true when a logout page is requested. This second policy will trigger the assignment action to remove the cookie from our map. add responder policy JID_Add_Session_ResPol "http.req.url.contains(\"login\") && $JID_Session_Map.valueExists(http.req.cookie.value(\"JSESSIONID\")).NOT" JID_Add_Session add responder policy JID_Delete_Session_ResPol "http.req.url.contains(\"logout\")" JID_Delete_Session With the NetScaler now able to add and remove JSESSIONID cookie values from our map when users retrieve login and logout pages, we will create a policy that prevents new sessions from accessing the store when the existing number reaches a defined limit. In this article, we will use a limit of 10 sessions. We assume the queuing system or the normal flow of the web store will cause users to access the login page initially and to log out after their purchase is complete. That could be achieved with dedicated pages or by changing the responder policy rules to reference existing page the user will retrieve at the start and end of their purchase. We define a 'new session' as a JSESSIONID cookie value that we haven't already told the NetScaler to store within the memory map. Then, we determine the session limit has been reached by counting the number of JSESSIONID values in the memory map. The policy that we will create to limit the number of sessions will call an action to display a status message informing your queuing system. add responder action SendLimitMessage respondwith q{"HTTP/1.1 200 OK\r\n\r\nThe limit of 10 sessions has been reached and the supplied JSESSIONID value \n" + HTTP.REQ.COOKIE.VALUE("JSESSIONID") + " is not on the allowed list."} add responder policy Check_Existing_Sessions_ResPol " $JID_Session_Map.valueExists(http.req.cookie.value(\"JSESSIONID\")).NOT && $JID_Session_Map.valueCount >= 10 " SendLimitMessage With our policies, actions, and variables created, we can now bind the policies to our load balancing vServer. As the queuing system passes a user to the Store, the NetScaler will check if the user has an existing session and the maximum session count. If the NetScaler hasn't reached the maximum, it will allow the user access and process requests for the login and log out pages we defined earlier. bind lb vserver MyOnlineStore -policyName Check_Existing_Sessions_ResPol -priority 10 -gotoPriorityExpression END -type REQUEST bind lb vserver MyOnlineStore -policyName JID_Add_Session_ResPol -priority 20 -gotoPriorityExpression NEXT -type REQUEST bind lb vserver MyOnlineStore -policyName JID_Delete_Session_ResPol -priority 30 -gotoPriorityExpression END -type REQUEST ResultsWe can see that ten users (tracked by JSESSIONID cookie) will now be able to access our web store, but the NetScaler will block other sessions until the existing sessions timeout or explicitly call the logout URL. We have also seen that we can define the session timeout by modifying the expiry time in seconds within the "add ns variable" command. If we would prefer the NetScaler to redirect additional sessions rather than produce an error message that our queuing system can consume, we can the "add responder action SendLimitMessage" responder policy. add responder action SendLimitMessage redirect "\"https://www.google.com\"" -responseStatusCode 302 Extending our solution to remove the explicit login page Our solution relies on the user calling a specific login and logout URL. You may ask, "How can we remove the login URL?". The answer to that question has a subtlety. We could change the expression on the " JID_Add_Session_ResPo l" responder policy, which logs in users by adding their JSESSIONID cookie to the variable map, to "true" and avoid the need for a login page. However, web browsers often display "favorite icon" representing a website. You will usually see that icon in the top left of the browser window, your website history, or your bookmarks list. The browser gains the icon by asking each website for a "favicon.png" file, usually after requesting a webpage. The "favicon.png" file is important as if we change our logic to replace the "login" page with a request for anything, and the browser silently requests the "favicon.png" file immediately after a user visits our logout page, that additional request will cause the user to log in again. We can avoid that scenario by modifying the expression on the "JID_Add_Session_ResPol" responder policy to ignore requests for the “favicon.png” file. add responder policy JID_Add_Session_ResPol "http.req.url.contains(\"favicon.ico\").NOT && $JID_Session_Map.valueExists(http.req.cookie.value(\"JSESSIONID\")).NOT" JID_Add_Session Next stepsWe have seen how to limit the number of users tracked by the JSESSIONID cookie, how to provide a response a queuing system can consume, how to redirect the user to an alternative website, and how we could extend the solution to remove the need for a login URL. The NetScaler could also track other variables if desired. For example, if we replaced the references to "http.req.cookie.value(\"JSESSIONID\")" with "client.ip.src.typecase_text_t", the NetScaler would convert the client's IP address into text and track that value within the variable map. add responder policy JID_Add_Session_ResPol "http.req.url.contains(\"favicon.ico\").NOT && $JID_Session_Map.valueExists(http.req.cookie.value(\"JSESSIONID\")).NOT" JID_Add_Session add responder policy JID_Add_Session_ResPol "http.req.url.contains(\"favicon.ico\").NOT && $JID_Session_Map.valueExists(client.ip.src.typecase_text_t).NOT" JID_Add_Session As NetScaler has many expressions that will return text (or values that can be typecase as text using the expression above), you can expand this solution as required. More details about NetScaler policy expressions can be found here: https://developer-docs.citrix.com/projects/citrix-NetScaler-advanced-policy-expression-reference/en/latest/

In our first article, we proposed an architecture allowing for multiple datacenters that was scalable to 100 million HTTP requests per second at each. We designed each datacenter to operate with a standardized design so that it's simple and easy to update and maintain.Now, we will look to automate that design. What is Terraform?We will automate using custom configuration scripts executed by Terraform. However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as:AnsibleAWS CloudFormationAzure Resource ManagerGoogle Cloud Deployment ManagerADM ServiceNitro REST API to allow for custom scripts. Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage servers in popular service providers such as AWS, Azure, and GCP as well as in-house solutions, including virtual and physical NetScalers. Terraform uses configuration files to define your intended configuration, to plan the changes needed to reach that apply those settings and can automatically reconfigure your systems. Using Terraform, we can create an "infrastructure as code" approach in which we manage and provision our datacenter through configuration files. How can I install Terraform?To install Terraform, you can follow the instructions and video here. Documentation for the NetScaler "provider" within the Terraform can be found here. Terraform will install the NetScaler "provider" automatically and you don't need to download or install it separately. For performance, we plan to use NetScaler MPX appliances in our environment. However, it is worth noting that Terraform can integrate with hosting environments and deploy virtual NetScaler instances (VPX) on demand. Preparing Terraform to connect to the NetScaler appliancesIn this article, we will configure the three NetScalers that we covered last time. We are writing about three NetScalers only for brevity, the same process can be followed for larger number of appliances.We will assume our NetScaler MPX appliances are already powered on and available with the IP addresses "192.168.1.10", "1.11", and "1.12". Terraform uses NetScaler's Nitro API to collect configuration information and implement changes. To allow that connection, we need to supply connection details, including credentials. Its good practice to keep these details separate from resource definitions to reduce the likelihood they are inadvertently copied to an insecure location. To start, we will create three variable files to hold the connection details needed to access our NetScalers. In production, the username/password would be for a service user account. netscaler1.tfvars (netscaler2.tfvar and netscaler3.tfvar will follow the same format) nsip = "192.168.1.10"[/code]username = "nsroot"[/code]password = "notnsroot"[/code] [/code]Next, we will create a provider file to define how Terraform should access the NetScalers and to tell Terraform about the variables within the files we made a moment ago. The "provider" definition within "provider.tf" tells Terraform to use the Citrix ADC / NetScaler provider from the Terraform registry. Terraform will automatically download the provider module for you. provider.tf terraform { required_providers { citrixadc = { source = "citrix/citrixadc" } } } variable "nsip" { type = string } variable "username" { type = string } variable "password" { type = string } provider "citrixadc" { endpoint = format("https://%s", var.nsip) username = format("%s", var.username) password = format("%s", var.password) insecure_skip_verify = true } Defining the configuration that Terraform will deployAfter creating the "provider.tf" file, we need to define the configuration Terraform will deploy onto the NetScalers. To do so, we need to create a file called "resources.tf". In this article, we will deploy a Load Balancing vServer and a basic ZebOS OSPF configuration. However, Terraform can create a wide range of configuration on the NetScaler and, you can find examples here. resources.tf resource "citrixadc_lbvserver" "tf_lbvserver" {[/code] [/code] name = "tf_lbvserver"[/code] ipv46 = "10.0.0.10"[/code] lbmethod = "LEASTLOAD"[/code] port = 80[/code] servicetype = "HTTP"[/code]}[/code]resource "citrixadc_service" "tf_service1" {[/code] name = "tf_service1"[/code] ip = "10.0.1.1"[/code] servicetype = "HTTP"[/code] port = 80[/code]}[/code]resource "citrixadc_service" "tf_service2" {[/code] name = "tf_service2"[/code] ip = "10.0.1.2"[/code] servicetype = "HTTP"[/code] port = 80[/code]}[/code] [/code]resource "citrixadc_lbvserver_service_binding" "tf_binding1" {[/code] name = citrixadc_lbvserver.tf_lbvserver.name[/code] servicename = citrixadc_service.tf_service1.name[/code] weight = 1[/code]}[/code]resource "citrixadc_lbvserver_service_binding" "tf_binding2" {[/code] name = citrixadc_lbvserver.tf_lbvserver.name[/code] servicename = citrixadc_service.tf_service2.name[/code] weight = 1[/code]}[/code]resource "citrixadc_nsip" "tf_nsip" {[/code] ipaddress = "10.0.1.10"[/code] type = "VIP"[/code] netmask = "255.255.255.255"[/code] icmp = "ENABLED"[/code] hostroute = "ENABLED"[/code] vserverrhilevel = "ALL_VSERVERS"[/code] arp = "DISABLED"[/code]}[/code]resource "citrixadc_routerdynamicrouting" "tf_dynamicrouting" {[/code] commandlines = [[/code] "router ospf",[/code] "network 192.168.1.0/24 area 0",[/code] "redistribute static",[/code] "redistribute kernel",[/code] ][/code]}[/code] Using Terraform to push our configuration to the NetScalersWith the configuration files created, we are ready to push configuration to our NetScalers. As we mentioned, you could expand this Terraform implementation to integrate with the hosting environment and create servers or virtual NetScaler instances (VPX) if you wish. To begin, we will prepare the directory containing the files we created using the "init" command. "init" will perform various steps, including creating a hidden subdirectory used to manage cached provider plugins and modules. terraform workspace initAfter initialization, we will create one Terraform workspace for each NetScaler. Workspaces are locations for Terraform to plan changes and track the NetScaler's state. terraform workspace new netscaler1 terraform workspace new netscaler2 terraform workspace new netscaler3 Terraform stores state in an internal database that persists between calls, which allows for smart "planning" behaviors. With the initial plan complete, Terraform then provisions the precise steps required to reach the desired end state without any human interaction. While not applicable to this article, as both our NetScaler and state are currently blank, if you can refresh Terraform's tracked state with the command "terraform refresh -var-file netscaler1.tfvars". You can learn more about managing NetScaler states using Terraform here. Enter the following commands to select our first NetScaler workspace and tell Terraform to plan the configuration changes. terraform workspace select netscaler1 terraform plan -var-file netscaler1.tfvars "Terraform plan" will list each of the settings it intends to implement and the settings for each object affected. The output will look like the text below. We have truncated the output shown here to save space in the article, and your output will be longer. % terraform plan -var-file netscaler1.tfvars Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # citrixadc_lbvserver.tf_lbvserver will be created + resource "citrixadc_lbvserver" "tf_lbvserver" { + appflowlog = (known after apply) + authentication = (known after apply) + authenticationhost = (known after apply) + authn401 = (known after apply) + authnprofile = (known after apply) + authnvsname = (known after apply) + backuplbmethod = (known after apply) + ipv46 = "10.202.11.11" + lbmethod = "ROUNDROBIN" + name = "tf_lbvserver" + port = 80 + servicetype = "HTTP" To apply the settings to the NetScaler, we need to run "terraform apply" terraform apply -var-file netscaler1.tfvars"Terraform apply" will prompt for confirmation and the apply the settings with output similar to the text below. Do you want to perform these actions in workspace "netscaler1"? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes citrixadc_lbvserver.tf_lbvserver: Creating... citrixadc_lbvserver.tf_lbvserver: Creation complete after 0s [id=tf_lbvserver] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. ValidationIf we now login to each of our NetScalers, we will observe that Terraform has deployed our configuration and that NetScaler has an OSPF neighbour relationship with our router. By repeating the per NetScaler "select workspace", "plan", and "apply" commands, perhaps in a script, we can implement the same configuration on each appliance. To apply future changes, you can now edit the resources.tf file once, and by rerunning the "plan" and "apply" commands, Terraform will reconfigure your NetScalers. Next stepsIn our first article, we saw how to create a scalable NetScaler solution that allows for multiple datacenters, scales to 100M L7 requests/sec at each, and supports both physical and virtual NetScalers. We have now seen how we can build a simple Terraform configuration to manage and deploy that configuration across many NetScalers. We have also seen that the Terraform configuration is expandable and could be integrated with other systems such as VMWare, AWS, Azure, or GCP to deploy servers and virtual NetScalers.

NetScalers are used by some of the world's largest service providers to handle inbound application traffic. In this article, I will show how you can deploy an NetScaler environment that will scale to millions of HTTP requests and SSL handshakes per second. In the next, I will show you how to automate that environment with a basic introduction to Terraform. The suggested architecture will allow for multiple datacenters. In addition, it will provide load balancing, SSL offload, and the capability to secure and optimize your web application traffic. The architecture will also be scalable and fully automatable. We will begin by reviewing technologies to get traffic to your datacenter, how to distribute that traffic across NetScalers once it arrives, and how you can configure your NetScalers with code using in-house deployment scripts. Assuming you have sized the rest of your environment appropriately, this design should allow you to process around 100 million HTTP requests per second at each datacenter. However, the layout is scalable and can be used for smaller implementations without significant changes. Directing users to their most appropriate datacenter Historically, there have been two approaches to directing users to their closest datacenter: dynamic routing (anycast) and GSLB. Dynamic routing (anycast) With the dynamic routing / anycast approach, the routers at each datacenter announce the same IP ranges and rely on the Internet's routing protocol (BGP) to take the shortest path. The approach operates on the assumption that a British user will have a shorter path to your British datacenter and your American user to an American datacenter. The method also assumes that a short path will be the best route. The concept is illustrated in simplistic terms in the diagram below. Users, sending traffic to their ISP, will have that traffic forwarded to the datacenter that is the smallest number of hops away. LinkedIn's engineering team wrote an excellent article on this topic some years ago, which is well worth reading. They found that global anycast showed cross-continent problems but was good when used within a continent. Their recommendation was to use DNS to return an IP address within the user's continent and to use anycast within each region. Users would perform a DNS lookup and be directed to the best continent to service their request. If you had multiple datacenters in that continent, these would share the same IP addresses using anycast, and users would then take the shortest path. GSLB (Global Server Load Balancing) / CADS Service (Citrix Application Delivery and Security Service) GSLB, Global Server Load Balancing, is a DNS based technology used to direct users to datacenters. Administrators commonly configure GSLB to ensure users receive the IP address of a datacenter in their region (or a redundant location if they are not using anycast). NetScaler fully supports GSLB, and you can read about configuring it here. However, GSLB configurations are often simplistic as they rely on relatively few data points. CADS Service (Citrix Application Delivery and Security Service) includes a feature known as ATM (Intelligent Traffic Management), which is the next generation of GSLB and an "Internet Aware" SaaS offering. While GSLB only takes metrics from your locations, CADS also allows you to integrate the radar tag into your website and collect metrics from end-users. To highlight the difference, if there were a problem at a regional ISP entirely separate from your environment, a GSLB configuration would examine metrics from your datacenters, conclude everything was healthy, and take no action. In contrast, ITM would see that users from the regional ISP are unable to reach your datacenter and immediately redirect them to a more performant location. If you intend to deploy without anycast, CADS is a must-have capability to detect and avoid connectivity issues between your users and datacenters. However, with anycast, CADS remains of significant benefit as it operates on objective user metrics to direct users to the location that provides the best possible service, avoiding both intercontinental link issues that LinkedIn wrote about and temporary connectivity problems. In this solution, we will assume that you have implemented anycast within each region and are using CADS’s ITM feature to ensure DNS sends users to the region where they receive the lowest latency and the fastest application response. A standardized repeatable design at each datacenter With traffic now arriving at the most appropriate datacenter, we can focus on handling the resultant traffic. We will design each datacenter to operate with a standardized design so that it's simple and easy to update and maintain. We will use physical hardware NetScalers known as "MPXs" for their high performance. However, our design is equally suitable for virtual NetScalers "VPX" if you favor a complete infrastructure-as-code approach. In our design, traffic (directed by CADS) will enter the network at the router/firewall. The router will then distribute these inbound connections to our NetScalers. Next, the NetScalers will terminate the connections, process the requests, and deliver them to backend servers. Finally, the servers will service the requests, potentially using local data or a storage solution replicated between your datacenters. All of the configuration will be automated using Terraform (covered in the next article). However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as: Ansible Citrix ADM Service Nitro REST API to allow for custom scripts. We assume you have appropriately sized your routers, firewalls, and storage. And you have ensured your storage is replicated between datacenters if your servers so require. This article shows a single router as a simplification. Splitting traffic within the datacenter Within the datacenter, the router/firewall will divide connections between our NetScalers using dynamic routing. We will be automating the configuration in the next article but, it would be helpful to first understand how this could be achieved manually. In our dynamic routing configuration, each NetScalers "announces" the IP addresses of the virtual servers (vServers) they host to the router. As each NetScaler will host vServers with the same IP addresses, the router will distribute the inbound connections between the NetScalers. On a Cisco router, the distribution of traffic between OSPF neighbors will likely occur in hardware using a technology known as CEF, Cisco Express Forwarding, to reduce the impact on CPU. You must take care to ensure your router has sufficient resources to handle the distribution. Additionally, to ensure all packets from a user are directed to the same NetScaler, you should consult your routers manual to ensure it is using per-destination load-sharing, which is the default on a Cisco router. To implement our dynamic routing configuration, we first need to configure our router to have an OSFP neighbor relationship with the NetScalers. On a Cisco router, a very simplistic OSPF configuration would look as follows: enable router# config t router(config)# router ospf 100 router(config-router)# network 192.168.1.0 0.0.0.255 area 0 router(config-router)# maximum-paths 16 router(config-router)# end After we enter the "network 192.168.1.0 0.0.0.255 area 0" command, our router will enable OSPF on all of its interface addresses within that network range. Next, on our NetScalers, we need to enable the OSPF routing facility with the command "enable ns feature ospf".After enabling OSPF, we can configure the NetScaler's included routing platform known as "ZebOS", by entering the command "vtysh". The ZebOS configuration on each NetScaler would look as follows: NetScaler# config terminal NetScaler(config)# router ospf NetScaler(config-router)# network 192.168.1.0/24 area 0 NetScaler(config-router)# redistribute kernel With the OSPF configuration created, we can tell each NetScaler to advertise the IP address of a vServer (such Load Balancing vServer) using the following command: set ns ip 10.0.0.10 255.255.255.255 -hostroute ENABLED -vserverRHILevel ALL_VSERVERS This command will tell the NetScaler to announce the IP address 10.0.0.10 to the router. With each of our NetScalers now announcing itself to our router as a path to the Load Balancing vServer, the router will distribute inbound traffic to each of the NetScalers. After enabling OSPF, you can validate communication between the NetScalers and router using the following commands on the router. Here, we can see three NetScalers on "192.168.1.10", "1.11", and "1.12" advertising the LB vServer on "10.0.0.10". R1# sh ip ospf | s 10.0.0.10 > 10.0.0.10/32, Intra, cost 1, area 0 via 192.168.1.10, GigabitEthernet1.1 via 192.168.1.11, GigabitEthernet1.2 via 192.168.1.12, GigabitEthernet1.3 R1# sh ip cef 10.0.0.1 detail 10.0.0.1/32, epoch 0, per-destination sharing nexthop 192.168.1.10 GigabitEthernet1.1 nexthop 192.168.1.11 GigabitEthernet1.2 nexthop 192.168.1.12 GigabitEthernet1.3 We have used a relatively simplistic configuration with a single router and each NetScaler acting as an OSPF path. As routers are generally limited to 16 OSPF paths, this design will likely be limited to 16 NetScalers. More advanced implementations will be able to expand on this simple design. 16 low-end appliances (such as the MPX9130) will equate to around 33 million L7 HTTP requests per second. Using high-end appliances (such as the MPX26200-50S), this design should service over 100 million L7 requests/sec at each datacenter. Note: You will have multiple routers in a production environment to meet redundancy and throughput requirements and may need to work with your network team to adjust the proposed configuration. Testing To test, our configuration, we will configure each NetScaler to return a different colored web page. The colors will make it immediately apparent which NetScaler is servicing our traffic. To return the colored web page, we need to add the green, blue, or purple responder policy to each of our NetScalers and bind it to the LB vServer. #NetScaler 1 add responder action Green_Responder_Act respondwith q{"HTTP/1.1 200 OK\r\n\r\n<html><body><p style=\"background-color:#78c445;\">Green NetScaler</p></body></html>\n"} add responder policy Green_Responder_Pol true Green_Responder_Act bind lb vserver LB_vServer -policyName Green_Responder_Pol -priority 1 -gotoPriorityExpression END -type REQUEST # NetScaler 2 add responder action Blue_Responder_Act respondwith q{"HTTP/1.1 200 OK\r\n\r\n<html><body><p style=\"background-color:#00c9cc;\">Green NetScaler</p></body></html>\n"} add responder policy Blue_Responder_Pol true Blue_Responder_Act bind lb vserver LB_vServer -policyName Green_Responder_Pol -priority 1 -gotoPriorityExpression END -type REQUEST # NetScaler 3 add responder action Purple_Responder_Act respondwith q{"HTTP/1.1 200 OK\r\n\r\n<html><body><p style=\"background-color:#a87bd9;\">Purple NetScaler</p></body></html>\n"} add responder policy Green_Responder_Pol true Purple_Responder_Act bind lb vserver LB_vServer -policyName Green_Responder_Pol -priority 1 -gotoPriorityExpression END -type REQUEST With the responder rules now added we can test traffic from different locations before removing the responder to restore usual service. Next steps We have seen how to create a scalable NetScaler solution that allows for multiple datacenters, scales to 100M L7 requests/sec at each, and supports both physical and virtual NetScalers. In the next article, we will explore how to automate the design using Terraform.

StyleBook automation using Citrix ADM's NITRO API: A DevOps primerOverviewCitrix Application Delivery Management (ADM) is a centralized management solution that enables you to operate, upgrade, report on, and troubleshoot all your NetScaler's across a global estate. One of Citrix ADM's best features is StyleBooks - declarative descriptions of a desired state written in YAML and consisting of input parameters and components driven by those inputs. A StyleBook causes Citrix ADM to read your current NetScaler configuration, compute the differences between your current and target state, and implement the required settings. While you can write StyleBooks yourself, Citrix ADM comes with more than 35 default StyleBooks, covering everything from building a simple HTTP load balancer to all of the content switching, load balancing, and authentication settings needed to support Exchange 2016. Learn about using default StyleBooks and creating custom StyleBooks. In this blog post, I focus on simple HTTP load balancers but provide enough detail so you can apply the logic to any StyleBook. Specifically, I'm going to focus on a StyleBook called "lb" with the namespace "com.citrix.adc.stylebooks"which you can find by selecting Applications, Configuration, StyleBooks within Citrix ADM. This post is the first in a two-part series that describes how to rapidly create services on your NetScaler's using Citrix ADM StyleBook automation, accelerating your migrations. We'll create 300 load balancers across an estate of several NetScaler's within 15 minutes. This post is aimed at DevOps employees who want to start working with Citrix ADM and need a brief introduction with examples (which you can download here). In my second post, I target sysadmins and focus on using preprepared scripts to achieve our goal of creating the 300 load balancers. Citrix ADM and the NITRO API The second best thing about Citrix ADM is that you can access it using API calls, which is how we'e going to automate our StyleBook. You can see more on that below but, you can also find more information here: Citrix ADM 13.1 NITRO API Reference documentationUsing APIs to create configurations from StyleBooksTo call your script to log in using the NITRO API, you need an ID and secret. You can create the ID and secret for Citrix ADM service by doing the following steps: Sign in to the Citrix Cloud portal.Select Identity and Access Management from the menu.Select the API Access tab.Name your Secure Client, and click Create Client. NITRO API request for a session ID After you have the ID and secret, you can sign in and collect a session ID using the following Python commands (this isn't my real ID and secret): Python import http.clientimport jsonid = "f20e7432-7a83-416d-a171-9347b92761f4"secret = "7-rotB6QMFZTuOgpPajPBg=="conn = http.client.HTTPSConnection("adm.cloud.com") headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': '' }request = { "login": { "ID": id, "Secret": secret }}payload = json.dumps(request)conn.request("POST", "/nitro/v2/config/login", payload, headers)response = conn.getresponse()data = response.read()if response.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1)jsondata = json.loads(data)sessionid = jsondata["login"][0]['sessionid']print (sessionid)Outputia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM NITRO API request using the session ID to retrieve a list of NetScaler identifiersNow that you have a session ID token, you can request a list of NetScaler's and find the unique identifier of the instance on which you would like to operate. The session ID is taken from the preceding script. Python import http.clientimport jsonsessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM"conn = http.client.HTTPSConnection("adm.cloud.com")headers = { 'isCloud': 'true', 'Content-Type': 'text/plain', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid }conn.request("GET", "/nitro/v2/config/managed_device", None, headers)response = conn.getresponse()data = response.read()if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1)jsondata = json.loads(data)#print('\n\nHere is the full JSON output from ADM.\n')#print (jsondata["managed_device"])print('\n\nHere is a list of ADCs.\n')for device in jsondata["managed_device"]: name = device['hostname'] ip = device['ip_address'] id = device['id'] type = device['type'] version = device['version'] status = device['status'] if status == "Success": print (name + " " + ip + " " + id + " " + type)OutputHere is a list of ADCs.myhostname 192.168.1.240 a1fe50be-e187-44bb-871b-d60f112b0aff nsvpx Knowing the parameters to submit to a StyleBookNow that we have a session ID and the identifier of an NetScaler to target, we can run a StyleBook. First, however, we need to understand the various parameters that it expects us to submit before doing so. You can gather the parameters by programmatically requesting the StyleBook's schema, which describes all valid parameters and which are optional, or by reviewing the source YAML for the StyleBook and its dependencies. Here are the Python commands to request a StyleBook's schema. Python import http.clientimport jsonsessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM"base_types = ["string", "ipaddress", "number", "boolean", "tcp-port", "password", "file", "certfile", "keyfile", "certkey", "ipnetwork"]def get_config_parameters(parameters): configpack_params = {} for parameter in parameters: parameter_name = parameter["name"] parameter_type = parameter["type"] parameter_required = parameter["required"] if parameter_type in base_types: if parameter_required: parameter_type += "*" if parameter_type.endswith("[]"): configpack_params[parameter_name] = [parameter_type] else: configpack_params[parameter_name] = parameter_type else: if "parameters" in parameter: subparameters = parameter["parameters"] datatype = get_config_parameters(subparameters) else: datatype = parameter_type if parameter_type.endswith("[]"): configpack_params[parameter_name] = [datatype] else: configpack_params[parameter_name] = datatype return configpack_paramsconn = http.client.HTTPSConnection("adm.cloud.com")stylebook = "com.citrix.adc.stylebooks/1.1/lb"headers = { 'Cookie': 'NITRO_AUTH_TOKEN= ' + sessionid}conn.request("GET", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/schema", None, headers)res = conn.getresponse()data = res.read()if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1)schema = json.loads(data, "utf-8")parameters = schema["schema"]["parameters"]config_parameters = { "parameters": get_config_parameters(parameters)}print(json.dumps(config_parameters)) The output from this schema request is the format of the parameters section of a config pack. From this output, you can remove parameters you don't intend to use. As a clue, the required parameters are marked with a "*" at the end of their type. Another (and possibly easier) way to achieve the same thing is to use the Citrix ADM UI to create a template that shows only the parameters I want to use. To create the template, I open Citrix ADM's web interface and run my selected StyleBook with dummy values against a non-production NetScaler in a lab. In doing so, I create a config pack with each of the values I would need for a production service. Having created the config pack, I now select Applications > Configuration > Config Packs from within the Citrix ADM web interface and export the resultant configuration. After downloading and opening the exported ZIP file, I find that it contains a single JSON file with all the parameters that I need to submit to the API. NITRO API request using the session ID to automate a StyleBook and target an NetScaler identifier Finally, having retrieved a session ID, the unique identifier of the NetScaler to operate on, and clarity into the parameters needed to submit to the StyleBook, we are ready to script execution of the StyleBook itself. Note: Delete the template configuration pack from Citrix ADM if you want to use the same input parameters within your Python script. Python import http.clientimport jsonsessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM"target_adc = "a1fe50be-e187-44bb-871b-d60f112b0aff"stylebook = "com.citrix.adc.stylebooks/1.1/lb"conn = http.client.HTTPSConnection("adm.cloud.com")payload = json.dumps({ "configpack": { "targets": [ { "id": target_adc } ], "parameters": { "lb-appname": "ApplicationName", "lb-virtual-ip": "192.168.50.10", "lb-virtual-port": "80", "lb-service-type": "HTTP", "svc-service-type": "HTTP", "svc-servers": [ { "ip": "192.168.5.20", "port": 80, "add-server": True }, { "ip": "192.168.5.21", "port": 80, "add-server": True } ], } }})headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid}conn.request("POST", "/stylebook/nitro/v1/config/stylebooks/" + stylebook + "/configpacks", payload, headers)res = conn.getresponse()data = res.read()if res.status != 200: print("Error:") print(" Status code: " + str(res.status)) print(" Error payload: " + data) exit(1)payload = json.loads(data, "utf-8")jobid = payload["configpack"]["job_id"]print("Configuration Job " + jobid + " has started.")OutputConfiguration Job 204770189 has started. Viewing a job_idAt this point you are able to see the config pack that your script created within the ADM GUI and the resultant configuration on your NetScaler. If that didn't happen, you can view the details of the "job_id" and understand its status. We can retrieve the outcome of the job by sending a GET request. To illustrate, I purposely ran another StyleBook targeted at a non-existent NetScaler ID. Python import http.clientimport jsonsessionid = "ia6bMEB9xn_XUW0Rfy_86zNbKlG1rJgZrd-DM3u3MIM"jobid = "2476087615"headers = { 'Content-Type': 'application/json', 'Accept': 'application/json', 'Cookie': 'NITRO_AUTH_TOKEN=' + sessionid}conn = http.client.HTTPSConnection("adm.cloud.com")job_status = Nonewhile job_status != "completed" and job_status != "failed": conn.request("GET", "/stylebook/nitro/v1/config/jobs/" + jobid, None, headers) response = conn.getresponse() data = response.read() if response.status != 202: print("Error:") print(" Status code: " + str(response.status)) print(" Error payload: " + data) exit(1) payload = json.loads(data, "utf-8") job_status = payload["job"]["status"]config_id = payload["job"]["progress_info"][0]["id"]if job_status == "completed": print("Success: configuration " + config_id + " successully created")else: print("Error: Failed to create configuration.") progress_info = payload["job"]["progress_info"] for step in progress_info: error_details = step["message"] print(" " + json.dumps(error_details)) exit(1)Output (failure)Error: Failed to create configuration."Validating the parameters""Configuration pack process failed. Instance with id g4702445-2286-4b6d-8fb1-c231a0ee17b1 not found on the system." Here, we can observe a clear explanation of the issue. We can observe success messages on jobs that run without issue. Output (success)Success: configuration 204770189 successfully created Next stepsWe have now seen how we can programmatically authenticate to Citrix ADM, retrieve a list of identifiers for the NetScaler's that it manages, prepare and run StyleBooks against a chosen NetScaler, and retrieve the result. We have also observed how you can find the namespace, version, and name of alternative StyleBooks and retrieve the various properties they require. Using this knowledge, your DevOps team is able to integrate Citrix ADM StyleBooks and their automated NetScaler configuration into your existing workflows. In the second of this two-part series, using the example of creating 300 load balancers across an estate of several NetScaler's within 15 minutes, I share prebuilt scripts that allow those without a DevOps team to create business services rapidly during a platform migration.

StyleBook automation using Citrix ADM's NITRO API: A SysAdmin primerIn this two-part blog series, I describe how to rapidly create services on your NetScaler's using Citrix Application Delivery Management (ADM) StyleBook automation to accelerate migrations. In the example, we create 300 load balancers across an estate of several NetScaler's within 15 minutes. My first post was aimed at DevOps. In this post, I target Sysadmin employees and focus on a pre-prepared script that I wrote. The script helps you to achieve our goal of rapidly creating load balancing virtual servers. You can download the script and its two supporting files here. Note: The script requires Python 3 and jinja2, that you can install with pip using the command "pip install jinja2". Prepare the StyleBook template("template.txt")The first of the script's two supporting files is named template.txt. The template.txt file is already preprepared for the simple HTTP load balancers that we create in this post, and you don't need to modify it. However, I do want to explain how you can make those modifications, in case you want to target other StyleBooks or introduce extra parameters. On the second line of the template.txt, you can find the namespace, version, and the name of the targeted StyleBook. StyleBook NAMESPACE/VERSION/STYLEBOOKcom.citrix.adc.stylebooks/1.1/lb If you want to change the template file's target, you can view the namespace, version, and the name of another StyleBook within Citrix ADM's web interface. On lines five to 12 of the template.txt, you can find placeholder names that correspond to the columns in a CSV file called lbvserver.csv, which is also included with the script. CSV columns targetadcnamelbvirtualiplbvirtualportlbservicetypesvcservicetypesvcservers: listsvcserverport As you can see from the placeholder named svcservers: list, if the columns in the CSV file contain a list of items, such as a list of back-end servers for a load balancing virtual servers, we note this by appending ": list" to the placeholder's name. You can find a template with each value that our script must submit to the StyleBook at line 14. Each item in the template matches one of the placeholder values you created earlier, and the template includes a loop where it encounters a list. Template { "configpack": { "targets": [{"id": "<<targetadc>>"}], "parameters": { "lb-appname": "<<name>>", "lb-virtual-ip": "<<lbvirtualip>>", "lb-virtual-port": "<<lbvirtualport>>", "lb-service-type": "<<lbservicetype>>", "svc-service-type": "<<svcservicetype>>", "svc-servers": [ <% for server in svcservers %> { "port": << svcserverport >>, "ip": "<< server >>", "add-server": true }<<,>> <% endfor %> ] } }} Again, you need to edit the template file only you're modifying the script to target an alternative StyleBook or you're introducing extra parameters. Preparing the script As we saw in Part 1, you need an ID and secret to allow the script to log in into your Citrix ADM environment. If you're using the Citrix ADM service, you can create the ID and secret by doing the following steps: Sign in to the Citrix Cloud portal.Select Identity and Access Management from the menu.Select the API Access tab.Name your Secure Client, and click Create Client. After you have the ID and secret, replace the placeholder example values within the create2.py python script. # Set my login variables for ADM# You can create an ID and secret by selecting the Identity and Access Management tab in your Cloud portalId = "1707d2ff-e200-46be-848a-93faf2076d6b"Secret = "DIH1CzF_GSZG2zNnEVShLA=="` Preparing the Excel CSV templateNow that you've set the ID and secret values, you can run the script with the "-a" parameter to display a list of NetScaler's and their identifiers. Command python create2.py -a Output The following NetScaler instances IDs can be referenced within your CSV file.Name - IP address - ID for use in CSV.1\. bob - 192.168.1.240 - a0de40bb-e186-44bb-871b-d60f112b0a3a Here, we can observe that I have an NetScaler with the host name "bob" on IP address "192.168.1.240" with the unique identifier "a0de40bb-e186-44bb-871b-d60f112b0a3a." You must note each identifier because we need them later. Prepare a list of load balancing virtual server ("lbvserver.csv")The second of the script's two supporting files is named lbvserver.csv. Each row in the lbvserver.csv file includes the properties for a single StyleBook, in this case, the properties for a single HTTP Load Balancing virtual servers. Each column in the lbvserver.csv file is the source for the placeholders within the template.txt file. The columns and placeholders are in the same order, with the first column being the first placeholder and so on. I have expanded the lbvserver.csv file to contain 300 rows (300 HTTP Load Balancing virtual servers) for this post. If my customer had a similar requirement, I would likely populate an initial CSV with all the expected values. Then, they would forward the CSV internally to ensure that the names, naming conventions, and IP addresses were all as desired. As you can observe by comparing the columns to the template.txt file, column A specifies the unique identifier of the NetScaler that we target, and columns B to H are the input parameters for the StyleBook. You can include lists in the parameters with ":" used as a separator. After modifying the lbvserver.csv file to reflect the virtual servers that you want to create, you must save in "CSV" rather than "CSV (UTF8)" format. Running the script (python create2.py) Having populated the script with an ID and secret that allow access to your Citrix ADM environment and updated the columns to reflect which NetScaler you want to build on and the properties, we are now ready to create our load balancing virtual servers. We can now run the script and create a Citrix ADM config pack along with the resultant NetScaler configuration. Command python create2.py Output (shortened) Creating ADM Config Pack based on CSV row: 1{"configpack": {"job_id": "1926702141"}}Creating ADM Config Pack based on CSV row: 2{"configpack": {"job_id": "2520889571"}}Creating ADM Config Pack based on CSV row: 3{"configpack": {"job_id": "2581540139"}}Creating ADM Config Pack based on CSV row: 4{"configpack": {"job_id": "1440135035"}}Creating ADM Config Pack based on CSV row: 5{"configpack": {"job_id": "3264746514"}} The script takes around two to three seconds per row and completes the 300 rows in just under 15 minutes. After completion, the resultant Config Packs will be visible within ADM. Ongoing maintenanceIn addition to creating the virtual servers rapidly from a pre-agreed Excel CSV file, this script-based approach also allows management of the resultant virtual servers using ADM and future changes without a need to understand the underlying commands on their NetScaler's. You can view each of the resultant Config Packs by selecting Applications, Configuration, Config Packs from within the Citrix ADM web interface. By selecting Edit, you can then manage and update the Config Pack as your needs change. Next stepsWe have now created 300 load balancers across our estate using an adaptable template that allows you not only to create load balancers but any other StyleBook based configuration. If you want to understand how the script works, you can read through the first article in this series .

Simplify your NetScaler Access Control Lists using datasetsNetScaler Access Control Lists (ACLs) examine the traffic arriving on interfaces and allow or deny it based on the configuration. By restricting access to virtual servers and management interfaces, they can be a superb extra layer to overall security. It's straight forward to write ACLs that allow a single continuous block of IP addresses to communicate with one another, the number of ACLs required can grow quickly when there are many client IPs, which can add numerous complexities. NetScaler has a simple and elegant solution that reduces avoid this complexity by defining datasets to represent the IP addresses and ports to be blocked or allowed and then calling them as an entity in ACL rules. In this blog post, we'll look at that solution and how it simplifies complex ACL configurations. ACLs and Datasets With recent NetScaler releases, you can use datasets instead of specifying a range of IP addresses and ports in the source and destination fields of each rule. This makes an environment much simpler to configure and maintain. Below are two examples of how the same NetScaler is configured with and without datasets Example 1 (ACLs without datasets)add ns acl ClientRange1-HTTP ALLOW -srcIP = 10.10.10.10-10.10.10.19 -destIP =10.100.100.10-10.100.100.19 -destPort = 80 -protocol TCP -priority 10 add ns acl ClientRange2-HTTP ALLOW -srcIP = 10.20.20.10-10.20.20.19 -destIP =10.100.100.10-10.100.100.19 -destPort = 80 -protocol TCP -priority 20 add ns acl ClientRange1-SSL ALLOW -srcIP = 10.10.10.10-10.10.10.19 -destIP =10.100.100.10-10.100.100.19 -destPort = 443 -protocol TCP -priority 30 add ns acl ClientRange2-SSL ALLOW -srcIP = 10.20.20.10-10.20.20.19 -destIP =10.100.100.10-10.100.100.19 -destPort = 443 -protocol TCP -priority 40 apply ns acls Example 2 (ACLs with datasets)add policy dataset WebClients_dataset ipv4 add policy dataset WebServers_dataset ipv4 add policy dataset dstport_dset numberbind policy dataset WebClients_dataset 10.10.10.10 -index 1 -endRange 10.10.10.19 bind policy dataset WebClients_dataset 10.20.20.10 -index 2 -endRange 10.20.20.19 bind policy dataset WebServers_dataset 10.100.100.110 -index 1 -endRange 10.100.100.119 bind policy dataset WebPorts_dset 22 -index 1bind policy dataset WebPorts_dset 443 -index 2add ns acl ClientWebRequests ALLOW -srcIP = WebClients_dataset -destIP = WebServers_dataset -destPort = WebPorts_dset -protocol TCP -priority 10apply ns acls As you can see, the first example needs one ACL for each block of IP addresses and ports on the client and server. If we added a second range of server IP addresses, we'd need twice as many ACLs. In the second example, we end up with a single ACL. Plus, adding another range of server IPs to "WebServers_dataset" requires just a single bind command. Imagine how much time and complexity you can save in an environment with lots of clients and servers. How Do ACLs and Datasets Work? As you configure ACLs with datasets, NetScaler builds "effective ACLs" in the background. In our first example, the "ACL Count" and "Effective ACL Count" match because the NetScaler didn't need to create additional ACLs based on the dataset. In the second example, the "ACL Count" is one and the "Effective ACL Count" is four because the NetScaler created the same ACLs that we would have manually created. The command "stat ACL" shows you both the "ACL count" and the "Effective ACL count". NetScaler can handle a total of 10,000 ACLs, giving you a tremendous amount of flexibility as you design and build your layered security approach. nsroot@bob-Primary> stat aclACL Statistics Rate (/s) Total Allow ACL hits 0 0NAT ACL hits 0 0Deny ACL hits 0 0Bridge ACL hits 0 0ACL hits 0 0ACL misses 0 0ACL Count -- 1Effective ACL Count -- 4DFD ACL Count -- 0DFD ACL hits 0 0DFD ACL misses 0 0 Done Learn More We've shown how datasets and NetScaler ACLs can simplify complex ACL configurations. Learn more about NetScaler ACLs and datasets.

DoS & DDoS protection with NetScaler Access Control Lists (ACLs)NetScaler provides many capabilities to protect against both DoS and DDoS attacks. These capabilities include Access Control Lists (ACLs), layer 3 and 4 protection mechanisms, HTTP protocol validation, IP reputation protection to identify likely bad actors, automated bot detection, rate-limiting, a full web application firewall with TCP signature scanning and L7 rules, and a cloud-based DDoS protection service. What is a Denial of Service (DoS) and Distributed Denial of Service attack? A Denial of Service (DoS) attack is an action intended to make a service or platform unavailable, generally by consuming one or more constrained resources by flooding the service with a large number of requests or by sending requests known to trigger a crash. A Distributed Denial of Service or DDoS attack is an attack that originates from multiple sources, perhaps each sending a small number of requests which when added together overwhelm the platform. DDoS attacks are more difficult to protect against as it can be challenging to identify and separate the illegitimate traffic from perfectly valid requests. You can learn more about DDoS attacks and their motivations in Pankaj Gupta's excellent blog in "Info Security" magazine. How can NetScaler help you?Because DDoS attacks happen at all layers, we need a mitigation strategy for each. Alone, each strategy would have weaknesses, and it is only by bringing protection capabilities together that we reap the full benefits. Attacks can have different vectors. The most common are: Volumetric, simply filling the network by sending too much traffic.Protocol, creating excessive sessions to exhaust the resources on networking devices.Application, abusing an application's capabilities to bring it down.Increasingly, attackers are using multi-vector attacks to defeat individual mitigations and to divert attention away from their real target. The most common attacks happen at the Network (layer 3), Transport (layer 4), Presentation (layer 6) and Application (layer 7). Open Systems Interconnection (OSI) Model #LayerApplicationDescriptionAttack Example7ApplicationDataNetwork process to applicationsHTTP flood, SQL Injection, Buffer overflow6PresentationDataData representation and encryptionSSL and TLS (BEAST, POODLE. Heartbleed)5SessionDataInterhost communicationSession hi-jacking4TransportSegmentsEnd-to-end connections and reliabilitySYN flood, UDP flood, TCP session hi-jacking3NetworkPacketsPath determination and logical addressingICMP flood, MITM2DatalinkFramesPhysical addressingARP Spoofing1PhysicalBitsMedia, signal, and binary transmissionWire cutters While the OSI model is useful way to think about attacks, it's common for modern applications to have functions that encompass layers 5-7 which makes it tricky to categorize an application into a single layer. In practice, layers 5-7 are generally thought of as "the upper layers", layers 2-3 as the "infrastructure layer", and layer 1 as "the physical connection". With this in mind, NetScaler can offer the following protection capabilities: Can Citrix offer any further help?Beyond the capabilities of individual NetScaler, Citrix Web App and API Protection (CWAAP) service provides a ready to go DDoS mitigation offering with one of the world's largest DDoS scrubbing capacities. Its scrubbing service cleans incoming requests of superfluous DDoS traffic before it reaches your platforms. CWAAP has a scrubbing capacity of 12 Tbps and can protect against the very largest volumetric DDoS attack. You can find out more about Citrix's cloud-based protection service here. Access Control ListsThe first protection capability that I'm going to talk about, and the focus of this article, is Access Control Lists (ACLs). ACLs are a good way to prevent unwanted requests tying up processing capacity at higher layers. They won't prevent a volumetric attack but, they are helpful when combined with other techniques as part of a broader program of holistic protection. ACLs provide an extremely basic mechanism of blocking traffic as it arrives at your NetScaler and preventing further processing. ACLs are network filters that control incoming and outgoing traffic. ACLs are used to improve security by restricting or allowing communication from specified hosts. They can be extremely helpful when you need to block illegitimate request from known sources that are tying up processing capacity at the higher layers of the computing stack. ACL Security Models ACLs are used in either a default allow or deny security model. In a default allow (block list) model, all hosts except those specified will be able to communicate with the NetScaler. This default allow model is well suited to a public website where you wish to provide services to the whole Internet except for a few bad actors. In a default deny (allow list) model, all hosts are restricted and only able to connect if your ACLs allow it. This is a good model for situations where you want to limit management traffic to a small number of hosts or a few partners need access to a website. The default deny security model is generally considered to be more secure by virtue of its restrictive nature but, it will take more work to maintain. NetScaler supports Access Control Lists (ACLs), defaults to an allow/blocklist model, but can be configured to use either model. I have provided some examples of each in the sections that follow. DatasetsNetScaler's ACLs support both explicit values, like a range of IP addresses or ports, and datasets which reference a collection of values. Datasets enable you to create and manage a small number of ACLs that reference a collection of values such as "management IP addresses", "management ports", or "legitimate clients". Here, we will create a small number of datasets to use in the default allow and deny configuration examples that follow. Here's an example of how to create a dataset. Creation of a datasetadd policy dataset RestrictedClients IPV4bind dataset RestrictedClients 10.0.1.0 -endrange 10.0.1.255add policy dataset RestrictedPorts NUMbind dataset RestrictedPorts 22bind dataset RestrictedPorts 110bind dataset RestrictedPorts 143add policy dataset ADC_Management_IPs IPV4bind dataset ADC_Management_IPs 192.168.11.1 -endrange 192.168.11.10add policy dataset ADC_Management_Clients IPV4bind dataset ADC_Management_IPs 10.0.5.1 -endrange 10.0.5.21add policy dataset ADC_Management_Ports NUMbind dataset ADC_Management_Ports 22bind dataset ADC_Management_Ports 443 Configuring ACLs in a default allow (block list) modelYou can configure ACLs within the NetScaler's GUI by selecting System, Network, ACLs, Extended ACLs and clicking the Add button. You can also configure ACLs using the CLI. Here are two examples in which client IP addresses are blocked using explicit and dataset based ACL rules: Creation of two deny ACLs using datasetsadd ns acl Example1 DENY -protocol TCP -srcIP 10.0.0.0-10.0.0.255 -srcPort 1024-65535 -destIP = 192.168.10.0-192.168.10.255 -destPort 443 -priority 10add ns acl Example2 DENY -protocol TCP -srcIP RestrictedClients -destPort RestrictedPorts -priority 11apply ACLs Here, we are explicitly telling the NetScaler to deny communication between the listed source and destination addresses. Addresses that are not specified within the ACL will be allowed access. Note the order of ACL evaluation begins with the lowest priority number and finishes with the highest. The "protocol", "srcIP", "srcPort", "destIP", "destPort", and "priority" parameters are optional and serve to further restrict the ACL rule. Configuring ACLs in default deny (allow list) model To implement a default deny ACL list, we start by explicitly allowing communication from our management range at the lowest priority and then denying all other traffic at the highest possible priority number (remembering that ACLs with a lower priority number are evaluated first). Creation of an allow ACL using datasetsadd ns acl Allow_Management_Range ALLOW -protocol TCP -srcIP ADC_Management_Clients -destIP = ADC_Management_IPs -destPort ADC_Management_Ports -priority 1add ns ACL Deny_All DENY -priority 100000apply ACLs Now, having restricted all access to the NetScaler beyond that of our management range, we can create additional ACLs allowing communication to and from legitimate hosts. add ns acl Example1 ALLOW -protocol TCP -srcIP 10.0.0.0-10.0.0.255 -destIP 192.168.10.5 -destPort 443 -priority 10add ns acl Example2 ALLOW -protocol TCP -srcIP Good_Clients_Dataset_Name -destIP Business_Service1_IPs_Dataset_Name -destPort Business_Service1_Ports_Dataset_Name -priority 11apply ACLS ACL Logging with log rate limiting NetScaler does not log ACL violations to Syslog by default. To change this behavior, you need to modify the global syslog parameters to activate ACL logging and then enable logging on each ACL for which you desire a log entry. The following command will modify the global syslog parameters to record ACL logs: set audit syslogParams -acl ENABLED To enable logging on an ACL, we need only include the -logstate ENABLEDparameter. However, to ensure that we don't overwhelm the Syslog server, it is usually wise to also include the optional -ratelimit parameter to specify a maximum number of log entries that should be created each second. The following command will create an ACL with logging enabled and a rate limit of 200 log entries per second. add ns acl Example1 ALLOW -protocol TCP -srcIP 10.0.0.0-10.0.0.255 -destIP 192.168.10.5 -destPort 443 -priority 10 -logstate ENABLED -ratelimit 200 Implicit ACLsBy default, a NetScaler has a small number of implicit ALLOW ACLs and will not block some types of important HA and clustering traffic even when a deny ACL is configured. These implicit ACLS are: ModeSourceDestinationDestination PortProtocolPurposeAllAllNSIP & SNIPs3008-3011TCPMetric Exchange and synchronizationAllAllNSIP & SNIPs179TCPBGPAllAllNSIP & SNIPs520UDPRIPHAHA PartnerNSIP3003TCPHA HeartbeatHAHA PartnerNSIP4001TCPZebra routing configuration syncHAHA PartnerNSIP22TCPSSH/SCPClusterCluster nodesNSIP7000UDPCluster heartbeatIf you wish to disable the implicit ACLs, perhaps to replace them with more restrictive rules, you can do so using the following command: set l3param -implicitACLallow DISABLED Next stepsYou can see the power of ACLs in blocking unwanted traffic. This can be extremely useful on your network, especially if you want to prevent certain users from accessing sites or servers. Remember, though, that ACLs should only be used as a supplement to other security measures. They should not be relied on as the only security measure on your network.

Deploying Citrix Gateway using Citrix ADM and StylebooksCitrix ADM does not currently have a Stylebook for Gateway. However, that may soon change, as I have written one and it is on track to be included in the defaults. This article describes how to use the Gateway Stylebook in its current form to deploy a complete Citrix Unified Gateway implementation that includes LDAP authentication and Citrix Cloud integration. In the future, I hope to offer RADIUS MFA and SAML as options. Citrix ADM, for those who have not used it before, is Citrix's Application Delivery Management platform. ADM allows you to monitor, report on, and configure all of your NetScaler (regardless of form factor) and their services from a central location. Stylebooks are YAML configuration files defining declaratively the various objects necessary to implement a particular configuration, in this case the implementation of a Unified Gateway. In this article we will assume that you already have a Citrix ADM deployment, if you do not, I recommend that you sign up for a Citrix ADM Service Express license here. Download and import the Gateway Stylebook The first step is to download the Gateway Stylebook from my GitHub repository by clicking here. Once you have downloaded the "Gateway LDAP Only" Stylebook, you should import it into ADM by selecting "Applications > Configuration > StyleBooks", and then by clicking "Import New StyleBook". Select the Stylebook file and click "Create" on the "Import Stylebook" screen. The Stylebook will be displayed under "Custom Stylebooks" after it has been imported. Run the Gateway Stylebook Click the "Create Configuration" button located on the lower left-hand side of the imported "Gateway" Stylebook. The Stylebook will now prompt you to enter the following values: A friendly name for the GatewayThe name can be anything and is only used as the basis for the names of objects within the NetScaler. The Fully Qualified Domain Name for your Gateway. Three new IP addresses. These IP addresses will be used to create the Gateway and the LDAP and StoreFront load balancers that the Gateway will rely on. Next, select "Authentication Settings" and replace the default values with details of the service account, a filter for limiting LDAP searches, and a list of Active Directory servers to be used by the Gateway. Select "Gateway vServer configuration" and replace the defaults with the details of your STA and StoreFront servers. Optionally, if you would like to use this on-premises Gateway as an identity provider for Citrix Cloud, include the details provided by Citrix Cloud. Enter the name of an existing SSL certificate or choose to import a new certificate and key file. Next, select your NetScaler and click "Create". You will now be presented with progress of each step. After the operation is complete, click "Close". Should you wish to alter any of the settings that you have entered, you can select the configuration pack created by the Stylebook and click "Edit". The Resultant Gateway configuration The Gateway Stylebook will deploy a Unified Gateway Content Switch (CS) listening for HTTPS requests on the specified IP address, as well as a HTTP CS that redirects users who mistakenly attempt unencrypted access. The Stylebook will also deploy a Gateway vServer that processes requests received by the CS vServer. The Gateway vServer is configured with standard ICA Proxy session policies for both Citrix Workspace and web browser access. Additionally, the Stylebook has deployed a load-balancer vServer with monitoring to ensure that requests are redundantly distributed between StoreFront servers. The Gateway vServer is configured to perform authentication using a new AAA vServer. The AAA vServer, which also has been deployed by Stylebook, has been automatically configured to perform LDAP authentication against the servers that you provided. The Stylebook has also deployed a load-balancer vServer equipped with monitoring to ensure that requests are distributed redundantly. Next steps We have seen how to download the Gateway Stylebook, upload it into ADM, and deploy a new Unified Gateway configuration in around ten minutes. I hope that this Stylebook will significantly reduce the time needed to deploy new NetScaler. If you are pursuing a DevOps approach, your team might be interested in my recent blog article on how to automate Stylebooks and apply configuration across a global estate.