HTTP Callouts

New in NetScaler 9.0 is the ability to perform a callout using HTTP to an external server. An HTTP Callout is a means to process incoming packets on the NetScaler using an external service that can be a virtual server on the NetScaler itself, a back-end server, or an third party service.

Traditionally, the NetScaler used to verify these packets internally using in-built policies but with specialized services being available for validation, they can be integrated with the NetScaler using this feature.

An HTTP callout will consist of a NetScaler policy expression that can send a simple HTTP request to an external service, wait for the response and then parse the response to produce a simple result. The result will then be used like any other policy expression evaluation result.

The HTTP callout expression:

SYS.HTTP_CALLOUT(<name of HTTP Callout>)

To define the HTTP callout:

set policy httpCallout <name>
	[-IPAddress < ip_addr|ipv6_addr>]
	[-port <port>]
      	[-vServer <string>]
	[-returnType <returnType>]
	[-httpMethod ( GET | POST )]
	[-hostExpr <string>]
	[-urlStemExpr <string>]
	[-headers <name(value)> ...]
	[-parameters <name(value)> ...]
	[-fullReqExpr <string>]
	[-resultExpr <string>]

Where:

-returnType must be one of TEXT, NUM or BOOL.

-IPAddress IP address of the server to which callout is made

-port Port of the server to which callout is made

-vserver must be one of the vservers added using the "add lb/cs/cr vserver" command. The service type of the vserver must be HTTP.

-httpMethod could be GET or POST.

-hostExpr Complex PI string expression for value of the Host header.

-urlStemExpr Complex PI string expression for generating the URL stem.

-headers Every header name must have a corresponding value. These headers will be inserted in the request. Header name is string. Header values are Complex PI Expressions.

-parameters Every parameter name must have a corresponding value. These parameter names are put in the URL query if the request has a GET method or they are put in the body if the request has a POST method. One must not rely on the order in which the parameters are inserted. Parameter name is a string. The parameter values can be computed using Complex PI String expressions. The parameter values will be URL encoded.

-fullReqExpr A complex PI String expressions computes the entire request. It is the user's responsibility to provide a well formed and sane HTTP request. The system will not do any sanity checking. If full request is specified then none of the other arguments can be specified.

HTTP callouts are available with HTTP or TCP Content Switching, Responder and Rewrite functionality.

The basic communication flow for HTTP callout is:

1. User sends request
2. Policy sends HTTP request to an external service
3. Result used like any other policy evaluation result
4. Available for multiple features

HTTP Callout Deployment Scenarios

The examples in this section illustrate how to use HTTP callouts to perform various tasks. In all cases, the NetScaler performs a callout to an external server where a callout agent is configured to respond to the request from the NetScaler based on the data that is present on the external server.

This section describes how to configure HTTP callouts in the following scenarios:

1. Filter clients based on an IP blacklist.
2. Fetch and update content on the fly using Edge Side Includes (ESI) markup language.
3. Authenticate users and control access to resources.
4. Filter Outlook Web Access (OWA) spam.

Filtering clients based on an IP blacklist

HTTP callouts can be used to block requests from clients that are blacklisted by the administrator. This list of clients can either be a publicly known blacklist or one that is maintained specifically by the administrator or a combination of both.

The source IP address of the incoming client request is checked against the external pre-configured blacklist and based on whether the IP address has been blacklisted or not, the transaction is either blocked by the NetScaler or the NetScaler continues to process the transaction normally.

The HTTP callout feature facilitates this by allowing the NetScaler to communicate with the external server that maintains a database of such blacklisted IP addresses.

The following outlines the requirements to implement this configuration:
1. Enable Responder on the NetScaler.
2. Create an HTTP callout on the NetScaler and configure it with details about the external server and other required parameters.
3. Create a Responder policy to analyze the response.
4. Bind the Responder policy globally on the NetScaler.
5. Create a callout agent on the remote server.

ESI support for fetching and updating content dynamically

Edge Side Includes (ESI) is a markup language for edge-level dynamic Web content assembly. It helps in accelerating dynamic Web-based applications by defining a simple markup language to describe cacheable and non-cacheable Web page components that can be aggregated, assembled, and delivered at the network edge.

Using HTTP callouts on the NetScaler, you can read through the ESI constructs and aggregate or assemble content dynamically.

The following outlines the requirements to implement this configuration:
1. Enable Rewrite on the NetScaler.
2. Create an HTTP callout on the NetScaler and configure it with details about the external server and other required parameters.
3. Create a Rewrite action to replace the ESI content with the callout response body.
4. Bind the Rewrite action to a Rewrite policy.
5. Bind the Rewrite policy globally on the NetScaler.

Access Control and Authentication

In high security environments, it may be mandatory to externally authenticate a user before a resource is accessed by clients. On the NetScaler, you can use HTTP callouts to externally authenticate a user based on supplied credentials. There are different ways that authentication credentials might be supplied; the client could be sending the user name and password in HTTP headers in the request, or, the credentials could be fetched from the URL or the HTTP body.

The following outlines the requirements to implement this configuration:
1. Enable Responder on the NetScaler.
2. Create an HTTP callout on the NetScaler and configure it with details about the external server and other required parameters.
3. Create a Responder policy to analyze the response.
4. Bind the Responder policy globally on the NetScaler.
5. Create a callout agent on the remote server.

OWA-based spam filtering

Spam filtering is the ability to dynamically block emails that are not from a known or trusted source or has inappropriate content. Spam filtering requires business logic that indicates a particular kind of message is a spam.

Using HTTP callouts, you can take out any portion of the incoming message and check with the configured external callout server that has the rules to detect if the message is a legitimate email or spam. In case of a spam email, the sender will not be notified that the email is marked as spam because it will only alert spammers to modify their messages.

The following outlines the requirements to implement this configuration:
1. Enable Responder on the NetScaler.
2. Create an HTTP callout on the NetScaler and configure it with details about the external server and other required parameters.
3. Create a Responder policy to analyze the response.
4. Bind the Responder policy globally on the NetScaler.
5. Create a callout agent on the remote server.

Read about the Citrix Application Switch with Version 9.0 here.

Try the Citrix Application Switch with Version 9.0 here.

Tap into the power of AppExpert!

An easy step up to IPv6

IPv6 has been available on NetScaler since April 2007, but only to select customers, and with a limited feature set.

Today, with NetScaler version 9.0, the IPv6 feature set is complete, with support for IPv6 communication all the way back to the application servers that the NetScaler is protecting and optimizing. Now that the IPv6 feature has matured, it has been released with the latest version of software! NetScaler version 9.0 includes IPv6 communication to the application servers, and all the usual tools use for troubleshooting will be present, such as ping6, traceroute6, etc.

The "IPv4 Dinosaur" may well be a term used in the future to describe a site which doesn't have an IPv6 representation on the internet. It's not a label one would want if they consider themselves to be keeping up to date with the latest and greatest technologies, as that of the Citrix NetScaler Application Switch.

Do keep in mind, running an IPv6 ONLY network, is probably still an arms length away and not very easy to migrate to. What would be required is a hybrid approach - and this is where NetScaler version 9.0 can provide a quick solution.

It is possible to use IPv6 communication from the internet to your NetScaler, and then use IPv4 from the NetScaler to the application servers. This will provide an IPv6 presence on the internet for your external website, without having to use time, resources, and budget to rebuild your entire environment right away.

Think of this as IPv6 offload, if you will. The fact that the application and back end systems are running IPv4 will be fully hidden from the end user. You can then, in your own time, port your back end infrastructure over to IPv6 step by step, making testing and roll-back a cinch.

Of course, full IPv6 end-to-end communication is equally important, especially for those government accounts which require this box to be checked-off for any new hardware going into the racks. This is the newest part of this feature, which is also now available in NetScaler version 9.0.

Read about the Citrix Application Switch with Version 9.0 here.

Try the Citrix Application Switch with Version 9.0 here.

Tap into the power of AppExpert!

NetScaler supports the chaining of Intermediate SSL Certificates

Up to 10 Chained Certificates to be exact, one Server Certificate and nine CA Certificates.

Verisign recently posted an advisory stating the discontinuance of Unchained SSL Certificates, and that all Verisign SSL Certificates issued after Dec 11, 2008 will be chained to Root CAs to align with security best practices - Read the advisory here.

Chaining of Certificates is done with Intermediate Certificates. What are Intermediate Certificates?

They sit in the middle, between the Public Trusted Certificate Authority (CA) and your Server, in our case the Citrix NetScaler.

The Citrix NetScaler Application Switch supports the chaining of SSL Certificates just for this very purpose, and to show how easy it is to obtain an SSL Certificate from a Trusted Certificate Authority, such as Verisign, and install it into the Citrix NetScaler, we developed the following deployment guide to walk you through the process.

Verisign Certificate Authority w/ Citrix NetScaler SSL Deployment Guide.

Tap into the Power of AppExpert!

Try it!

Wan Optimization Survey:

Take this quick survey to tell us more about the solutions your organization uses to optimize your WAN.

1. Do you currently have a WAN optimization solution in your IT environment?	Choose
Yes (go to Q2)
No (go to Q3)

2. Which, if any, of the following WAN optimization solutions does your organization currently use?	Choose
Citrix WAN opt product(s) (WANScaler, Branch Repeater, Branch Repeater w\Windows Server)
Riverbed
Blue Coat
Expand
Other (please specify in a comment)
I don't know

3. Approximately what average percentage of your organization's overall network traffic is via XenApp (ICA)?	Choose
0%
1-25%
26-50%
51-75%
76-99%
100%
I don't know

Thank you If there is anything else you would like to tell us, please leave a comment.

Not very long ago I published a series on how to become an Application Expert. Citrix NetScaler 9.0 makes it easier with AppExpert Templates. NetScaler AppExpert Templates - introduced in NetScaler 9.0 - provide an application-centric view of the NetScaler system's policy configurations. From a single place within the GUI (AppExpert -> Applications) NetScaler administrators can: 1) Configure the various AppExpert features the NetScaler is fronting, 2) View which NetScaler functional modules (e.g., compression, caching, application firewall) are optimized and active for a given application unit.

Additionally, AppExpert Templates allow you to drill down and see which individual NetScaler policies are active, and what policies are inactive but available, by application component and NetScaler module. From this same view, individual policies can be created, activated and deactivated.

AppExpert Templates can be downloaded, imported, modified and exported AppExpert Templates page of the Citrix Community Website. Administrators can download AppExpert Templates built by Citrix, Citrix Partners and members of the NetScaler community from the Citrix Community Website. These templates are easily imported into any NetScaler running NetScaler 9.0 or higher, jump starting the configuration and deployment process. Templates developed in-house can be easily exported and shared within your organization, or posted back to the Citrix Community Website for others to view and improve.

See the new AppExpert Templates page here!

Tap into the power of AppExpert!

NetScaler 9 is officially here. Well, actually, it's officially announced. It won't be officially available to download from mycitrix.com until November 27^th. Yes, I know that's Thanksgiving. However, Citrix is a global company, and what better way to prove it than to post the NetScaler 9 code on a major US holiday? And, there is a chance that it might show up a day or two before the 27^th.

NetScaler 9 is a pretty big release. Looking at the detailed feature tracker, it contains over 350 new features and feature enhancements. I'm not going to go through all of them in this post, because that's what release notes are for. However, I do want to highlight some of the major new features that folks seem to be most excited about, and point you to some additional resources on this site that go into a bit more detail on some of them.

I like to think that NetScaler acts as the bridge between the network and the applications that run on it, making each of them work better with the other. NetScaler 9 furthers this.  A lot of the new capabilities and features making NetScaler more application-saavy than it already is. This is not to say that there aren't any hardcore networking enhancements in NetScaler 9, because there are a lot of them. These include everything from end-to-end support for IPv6 to enhancements to our GSLB functionality to the ability to tunnel IP within IP.

But in the end our networks are there to run applications, and it's the new AppExpert features in NetScaler 9 that seem to be generating the most interest.

AppExpert Templates make a given application the "first class citizen" within NetScaler. They do this by encapsulating everything about a NetScaler configuration that is specific to a given application, including:

1. The different application components (e.g., pages, files, archives, Web Services) NetScaler is managing
2. The various NetScaler entities and settings (e.g., VServers/VIPs, load-balancing algorithms, health checks, persistence methods, SSL offload settings) defined for these application components
3. The specific NetScaler policies (e.g., caching, compression, application firewall, rewrite) used for the application

All of this is presented in a way that puts the application front and center, and configuration and policy changes can be made from there as well. So, while today understanding the entire NetScaler configuration for Microsoft SharePoint (for example) involves moving around between the various NetScaler GUI tabs, with AppExpert Templates everything is centralized in one place.

AppExpert Templates can be imported and exported as well, so they make it pretty easy to move app-specific configurations between different systems. More broadly, several folks have told us that this, and the general look and feel of AppExpert Templates, will help with knowledge transfer within their organizations. You can see an example of the Microsoft SharePoint template being imported and then applied here.

If you go here when NetScaler 9 becomes available in a couple of weeks, you'll be able to download AppExpert Templates we've already built. And, as you'll quickly notice, AppExpert Templates aren't static. The underlying infrastructure makes it really easy for you tweak a template to your own specific needs, or to improve the template by adding to it. Hopefully, you'll all post any improvements and modifications you make back to the community site so that others can benefit. And definitely look for additional AppExpert Templates to be made available by us, but Citrix partners, and hopefully by other NetScaler users.  

With AppExpert rate controls, we've integrated the concept of data rate into the core NetScaler policy infrastructure.  This allows building policies that are only triggered when a defined data rate is exceeded.  And since it's integrated with the core policy infrastructure, it can be used with the various NetScaler functional modules (e.g., content switching, responder), so you're not limited to just dropping traffic as an action.

There's a number of ways folks have told us they're going to use AppExpert rate controls. Of course straight-up rate limiting (e.g., DNS rate-limiting, limiting traffic originating from a single subnet) is one example. Ensuring a given resource (e.g., anything from a VServer to a specific URL) isn't overwhelmed by requests is another. Two specific examples are:

1. One customer allows some of its partners to scrape its website so the partners can republish content on their own sites. However, the customer wants to ensure that overly aggressive scraping by the partners doesn't overwhelm the website and degrade the site's performance. AppExpert rate controls can be used to limit how much scraping each partner can do. This same approach could be used to ensure that websites that publish APIs -- so that partners can do mashups, for example -- aren't overwhelmed by any particular partner's use of the API.
2. Another example is a customer that was having problems with a couple of users FTPing a few too many large files at the same time. By using AppExpert rate controls to build an expression around bandwidth consumed per sourceIP, they can drop any additional FTP requests coming from a sourceIP (aka a user) that already has too much FTP activity. A more generalized use could also do something along the lines of limiting the amount of concurrent file downloading for a given SharePoint site, to ensure that downloads don't drown out other SharePoint (or other application) activity.

AppExpert service callouts make NetScaler policies extensible, and will allow you to integrate logic or functionality available in other systems and applications into NetScaler policies. Specifically, using an AppExpert service callout, a policy can send (over HTTP or HTTPS) any part of an incoming request to an external service. The result returned by the external service is then used like any other policy evaluation result.

As an example, one beta customer has an application that identifies and tracks IP addresses that are scraping its site's content. No, this is not the same customer that is interested in AppExpert rate controls. In earlier case, scraping is encouraged, they just needed to control it. In this case, the scraping of content amounts to theft, and the customer want to prevent as much of it as possible. Unfortunately, the IP addresses doing scraping change constantly (hence the reason they had to build an app), so statically defining them within the policy itself isn't practical. However, a service callout can query the application in real-time, and NetScaler then uses the response to either pass or drop the request.

Other use cases customers have mentioned include:

• Passing content to an external transformation engine
• Integration with UDDI or other directory services
• Geo-targeting or other token-based switching decisions, where the logic for the content switch is available in an external application  

NetScaler 9 has the first availability of the XML technology we acquired from QuickTree last year. New XML protections in the NetScaler Application Firewall module will now be able to inspect and protect XML as well as HTML traffic. In addition to protecting XML-based applications from attack, this can also be used to ensure that incoming XML traffic conforms to various standards (e.g., XML syntax, schema, WSDL validation). With XML, sometimes "bad" traffic isn't malicious but is just a mistake. Either way, the XML capabilities in the app firewall will catch it.

We've had the ability to rewrite payloads within the TCP header or payload since NetScaler 8.0. However, in NetScaler 9.0 we've added a URL transformation 'mini-module' to our generalized rewrite functionality specifically for rewriting HREFs. While this function is often thought of in the context of either SSL VPN or application firewall, it has uses beyond these as well. For example, onboarding apps acquired through M&A activity, simplifying change management or "Akamai-zing" graphics content.

Again, NetScaler 9.0 is big release. There is a lot more than the app-centric things mentioned above. There is a pretty comprehensive What's New in NetScaler 9 writeup here for those of you that want a more comprehensive overview.

Updated November 12, 2008:

I received a question via comments asking about Access Gateway Enterprise enhancements. As many of you know, Access Gateway Enterprise is in essence another module in NetScaler. So, all Access Gateway Enterprise functionality is included in NetScaler, which is why NetScaler is such a great solution for Citrix XenApp and XenDesktop. There are definitely enhancement to Access Gateway Enterprise in NetScaler 9. At a high level, they are:

• Support for IPv6 XenApp Client Connections
• Single sign-on to file shares, so your users won't get get as annoyed by as many authentication prompts (unless you want them to be)
• Full clientless access to Microsoft SharePoint 2003 and 2007 so users can access SharePoint sites from any browser
• Historical charting which allows you to see trend data on system activity

Citrix Systems is closing the gap on the Number 1 Load Balancer for Web Applications. They are certainly a leader and not going to relent on the pace. Check out the Gartner Magic Quadrant. Further proving a commitment to Application Delivery, Citrix teamed with Akamai to extend Application Delivery from the datacenter into the cloud. Combining Akamai's efficiency in the cloud with Citrix's efficiency in the datacenter provides the ultimate in global acceleration of applications.

Citrix & Akamai Load Balancing Deployment Guide.

Tap into the power of AppExpert!

Read about the Citrix Load Balancer here.

Buy the Citrix Load Balancer here.

WAN Load Balancing by Elfiq Networks is a perfect fit for the Citrix WanScaler WAN Optimization Engine product. The Citrix NetScaler already performs Server Load Balancing on inbound connections, and can even perform Link Load Balancing on outbound connections. However, when it comes to managing link resiliency directly at the WAN Links, at layer 2, this is where Elfiq shines. The Elfiq Layer 2 implementation allows the insertion of the Elfiq unit between the firewall and the primary link router without any change to their configuration for an easy deployment. For private WAN Links, Elfiq will redirect packets to all links at Layer 2 on a per session basis. Another great advantage with Elfiq is the low price point to get this type of functionality. When connectivity is being deployed to multiple sites with multiple links, Elfiq SitePathMTPX can be used with IPSec VPN Tunnels and VoIP along side of enterprise applications for greater performance and resilience.

Citrix & Elfiq Networks Deployment Guide!

WAN Failover Video Tip:

WAN Load Balancing Video Tip:

NetScaler Developer Network!

The St.Bernard iPrism works with Citrix's Application Virtualization platform - XenApp, and works quite well. Seen as a perfect complement to each other the Citrix NetScaler and XenApp products were tested with the St.Bernard iPrism Web Filter. Both companies offer architectures of one-arm (out-of-band) and two-arm (in-band) deployments. At Citrixlabs in Santa Clara, CA, USA, we tested both the out-of-band and in-band configuration of the iPrism Web Filter. We loved the fact that the iPrism is auto-discovered by the management software, so no console cable was needed.

With NetScaler:

We deployed the iPrism Web Filter behind the NetScaler in our proof of concept datacenter in Santa Clara, CA, USA, and configured the NetScaler for NAT (Reverse NAT) for outbound connections to the Internet. NAT is often performed by the Firewall. The Web Application Firewall, also part of the Citrix NetScaler, was configured for protection of inbound security threats to websites and web applications.

The iPrism was configured to monitor outbound traffic from the internal subnet of 172.16.104.0/24, and block all traffic to offensive websites, and monitor traffic to all other websites. The Real-Time monitor in iPrism gave us a detailed report on the users and IP Addresses that were going out to which sites on the internet. We could see who was accessing what, and which content was being blocked. Particularly nice, was the fact that the iPrism automatically authenticated each user to the Citrixlabs domain controller, every time they surfed a new website, without them knowing it. This was very useful for keeping a tight grip on security and for compliance reporting.

With XenApp:

The powerful value is in the integration with XenApp. We plugged the iPrism in as an in-line device, and configured it to work with Citrix XenApp©, formerly known as Citrix Presentation Server. One of the key questions that will arise in this situation is with all of those Citrix XenApp thin clients logging into the XenApp and then launching browsers to the internet, how does iPrism keep track of them. By adding the XenApp IP Address to the iPrism configuration, the users are tracked using "Session Based Authentication" - this catches each individual user and IP Address in each browser session and in the reports. We were impressed by this and determined the iPrism to be an excellent fit into a datacenter outfitted with Citrix.


Citrix & St.Bernard Deployment Guide!

Network Diagram:

Watch this video tip:

NetScaler Developer Network!

AppExpert is now a useful tag used by Citrix Systems, Inc., to qualify articles and content on the web as that which pertains to the art of delivering applications to an end user. Becoming an "Application Expert" is not only an art, but as with anything else in the internet industry, takes time to learn. Becoming an Application Expert and using the knowledge takes time, drive and patience. In an effort to make this art easier, Citrix embarked on a series of product enhancements targeted toward the Application Expert, which started with the use of the NetScaler Policy Engine.

The first in many enhancements directed toward making application delivery easier for the individual using the product. Other product nomenclature was created for other features such as "AppCompress" and "AppCache", but "AppExpert" seemed to stick and a community website was born to promote the development of policies and expressions used in the course of business on the NetScaler at customer sites, by partners and in the labs at Citrix Systems.

The concept of the community site grew with favor as a much needed interactive forum for the exchange of ideas, policies, blogs, video tips and information to expand the knowledge of those using the Citrix products. Thus, the AppExpert community site was born to express this direction of growth in knowledge.

In an effort to create a community site that is both practical and useful for our customers, and in keeping aligned with the principles of ease of use and name recognition, the community site formerly named "AppExpert", has been renamed to "NetScaler Developer Network" to allow for a more easier fit among other Citrix product lines, as the Citrix community continues to grow.

NetScaler Developer Network!