Jump to content
Welcome to our new Citrix community!
  • Building a highspeed and scalable NetScaler solution to handle millions of requests per second: part 2

    Steven Wright
    • Validation Status: Work In Progress
      Summary: Following our article about deploy an NetScaler environment that will scale to millions of HTTP requests and SSL handshakes per second, we now look to automate that environment.
      Has Video?: No

    In our first article, we proposed an architecture allowing for multiple datacenters that was scalable to 100 million HTTP requests per second at each. We designed each datacenter to operate with a standardized design so that it's simple and easy to update and maintain.

    Now, we will look to automate that design.



    What is Terraform?

    We will automate using custom configuration scripts executed by Terraform. However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as:
    • Ansible
    • AWS CloudFormation
    • Azure Resource Manager
    • Google Cloud Deployment Manager
    • ADM Service
    • Nitro REST API to allow for custom scripts.

    Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage servers in popular service providers such as AWS, Azure, and GCP as well as in-house solutions, including virtual and physical NetScalers.


    Terraform uses configuration files to define your intended configuration, to plan the changes needed to reach that apply those settings and can automatically reconfigure your systems.

    Using Terraform, we can create an "infrastructure as code" approach in which we manage and provision our datacenter through configuration files.






    How can I install Terraform?To install Terraform, you can follow the instructions and video here.

    Documentation for the NetScaler "provider" within the Terraform can be found here.


    Terraform will install the NetScaler "provider" automatically and you don't need to download or install it separately.


    For performance, we plan to use NetScaler MPX appliances in our environment. However, it is worth noting that Terraform can integrate with hosting environments and deploy virtual NetScaler instances (VPX) on demand.




    Preparing Terraform to connect to the NetScaler appliances

    In this article, we will configure the three NetScalers that we covered last time. We are writing about three NetScalers only for brevity, the same process can be followed for larger number of appliances.

    We will assume our NetScaler MPX appliances are already powered on and available with the IP addresses "", "1.11", and "1.12".


    Terraform uses NetScaler's Nitro API to collect configuration information and implement changes. To allow that connection, we need to supply connection details, including credentials. Its good practice to keep these details separate from resource definitions to reduce the likelihood they are inadvertently copied to an insecure location.


    To start, we will create three variable files to hold the connection details needed to access our NetScalers. In production, the username/password would be for a service user account.


    netscaler1.tfvars (netscaler2.tfvar and netscaler3.tfvar will follow the same format)

    nsip = ""[/code]
    username = "nsroot"[/code]
    password = "notnsroot"[/code]

    Next, we will create a provider file to define how Terraform should access the NetScalers and to tell Terraform about the variables within the files we made a moment ago.

    The "provider" definition within "provider.tf" tells Terraform to use the Citrix ADC / NetScaler provider from the Terraform registry. Terraform will automatically download the provider module for you.



    terraform {

        required_providers {

            citrixadc = {

                source = "citrix/citrixadc"





    variable "nsip" {

      type = string


    variable "username" {

      type = string


    variable "password" {

      type = string



    provider "citrixadc" {

      endpoint = format("https://%s", var.nsip)

      username = format("%s", var.username)

      password = format("%s", var.password)

      insecure_skip_verify = true




    Defining the configuration that Terraform will deploy

    After creating the "provider.tf" file, we need to define the configuration Terraform will deploy onto the NetScalers. To do so, we need to create a file called "resources.tf".


    In this article, we will deploy a Load Balancing vServer and a basic ZebOS OSPF configuration. However, Terraform can create a wide range of configuration on the NetScaler and, you can find examples here.



    resource "citrixadc_lbvserver" "tf_lbvserver" {[/code]
      name = "tf_lbvserver"[/code]
      ipv46 = ""[/code]
      lbmethod = "LEASTLOAD"[/code]
      port = 80[/code]
      servicetype = "HTTP"[/code]
    resource "citrixadc_service" "tf_service1" {[/code]
      name = "tf_service1"[/code]
      ip = ""[/code]
      servicetype = "HTTP"[/code]
      port = 80[/code]
    resource "citrixadc_service" "tf_service2" {[/code]
      name = "tf_service2"[/code]
      ip = ""[/code]
      servicetype = "HTTP"[/code]
      port = 80[/code]
    resource "citrixadc_lbvserver_service_binding" "tf_binding1" {[/code]
      name = citrixadc_lbvserver.tf_lbvserver.name[/code]
      servicename = citrixadc_service.tf_service1.name[/code]
      weight = 1[/code]
    resource "citrixadc_lbvserver_service_binding" "tf_binding2" {[/code]
      name = citrixadc_lbvserver.tf_lbvserver.name[/code]
      servicename = citrixadc_service.tf_service2.name[/code]
      weight = 1[/code]
    resource "citrixadc_nsip" "tf_nsip" {[/code]
        ipaddress = ""[/code]
        type = "VIP"[/code]
        netmask = ""[/code]
        icmp = "ENABLED"[/code]
       hostroute = "ENABLED"[/code]
       vserverrhilevel = "ALL_VSERVERS"[/code]
       arp = "DISABLED"[/code]
    resource "citrixadc_routerdynamicrouting" "tf_dynamicrouting" {[/code]
        commandlines = [[/code]
            "router ospf",[/code]
            "network area 0",[/code]
            "redistribute static",[/code]
            "redistribute kernel",[/code]

    Using Terraform to push our configuration to the NetScalers

    With the configuration files created, we are ready to push configuration to our NetScalers. As we mentioned, you could expand this Terraform implementation to integrate with the hosting environment and create servers or virtual NetScaler instances (VPX) if you wish.


    To begin, we will prepare the directory containing the files we created using the "init" command. "init" will perform various steps, including creating a hidden subdirectory used to manage cached provider plugins and modules.


    terraform workspace init

    After initialization, we will create one Terraform workspace for each NetScaler. Workspaces are locations for Terraform to plan changes and track the NetScaler's state.


    terraform workspace new netscaler1 

    terraform workspace new netscaler2

    terraform workspace new netscaler3


    Terraform stores state in an internal database that persists between calls, which allows for smart "planning" behaviors. With the initial plan complete, Terraform then provisions the precise steps required to reach the desired end state without any human interaction.


    While not applicable to this article, as both our NetScaler and state are currently blank, if you can refresh Terraform's tracked state with the command "terraform refresh -var-file netscaler1.tfvars". You can learn more about managing NetScaler states using Terraform here.


    Enter the following commands to select our first NetScaler workspace and tell Terraform to plan the configuration changes.


    terraform workspace select netscaler1 

    terraform plan -var-file netscaler1.tfvars


    "Terraform plan" will list each of the settings it intends to implement and the settings for each object affected. The output will look like the text below. We have truncated the output shown here to save space in the article, and your output will be longer.


    % terraform plan -var-file netscaler1.tfvars


    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

      + create


    Terraform will perform the following actions:


      # citrixadc_lbvserver.tf_lbvserver will be created

      + resource "citrixadc_lbvserver" "tf_lbvserver" {

          + appflowlog                         = (known after apply)

          + authentication                     = (known after apply)

          + authenticationhost                 = (known after apply)

          + authn401                           = (known after apply)

          + authnprofile                       = (known after apply)

          + authnvsname                        = (known after apply)

          + backuplbmethod                     = (known after apply)

          + ipv46                              = ""

          + lbmethod                           = "ROUNDROBIN"

          + name                               = "tf_lbvserver"

          + port                               = 80

          + servicetype                        = "HTTP"



    To apply the settings to the NetScaler, we need to run "terraform apply"


    terraform apply -var-file netscaler1.tfvars

    "Terraform apply" will prompt for confirmation and the apply the settings with output similar to the text below.

    Do you want to perform these actions in workspace "netscaler1"?

      Terraform will perform the actions described above.

      Only 'yes' will be accepted to approve.


      Enter a value: yes


    citrixadc_lbvserver.tf_lbvserver: Creating...

    citrixadc_lbvserver.tf_lbvserver: Creation complete after 0s [id=tf_lbvserver]


    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.





    If we now login to each of our NetScalers, we will observe that Terraform has deployed our configuration and that NetScaler has an OSPF neighbour relationship with our router.


    By repeating the per NetScaler "select workspace", "plan", and "apply" commands, perhaps in a script, we can implement the same configuration on each appliance. To apply future changes, you can now edit the resources.tf file once, and by rerunning the "plan" and "apply" commands, Terraform will reconfigure your NetScalers.



    Next steps

    In our first article, we saw how to create a scalable NetScaler solution that allows for multiple datacenters, scales to 100M L7 requests/sec at each, and supports both physical and virtual NetScalers.


    We have now seen how we can build a simple Terraform configuration to manage and deploy that configuration across many NetScalers. We have also seen that the Terraform configuration is expandable and could be integrated with other systems such as VMWare, AWS, Azure, or GCP to deploy servers and virtual NetScalers.




    User Feedback

    Recommended Comments

    There are no comments to display.

    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!

    Register a new account

    Sign in

    Already have an account? Sign in here.

    Sign In Now

  • Create New...