rewrite policy to delete header elements in response doesn't work

[Cristina Marletta Livi]

In Netscaler ADC VPX 12, we have a problem due to a rewrite policy that does not work.
This is the policy definition:

add rewrite action CSP_remove_X-AspNet-Version delete_http_header X-AspNet-Version

add rewrite policy CSP_remove_X-AspNet-Version "http.res.header(\"X-AspNet-Version\").exists" CSP_remove_X-AspNet-Version

bind lb vserver VIP_www.xxx.it_28.11:80 -policyName CSP_remove_X-AspNet-Version -priority 100 -gotoPriorityExpression end -type RESPONSE

 

the statistics show that the policy hits but, looking at the http stream with a browser plug-in, I see that the response header always contains "X-AspNet-Version".

 

Why?

 

Regards

Cristina

 

[Rhonda Rowland1709152125]

If there is a duplicate header, only one will be removed.

If the web server is cmp responses, then the response time rewrite may not apply (despite the policy hit); you may need to prevent the server from doing compression (by corrupting/remove accept-encoding header during request or otherwise turning off compression on the web server.)

 

Not your exact rewrite scenario, but this article goes over the impact of compressed responses on the rewrite engine:  https://support.citrix.com/article/CTX122916

 

 

 

 

[Cristina Marletta Livi]

Thank you Rhonda,

I added a rewrite policy to remove Accept-Encoding from the request.

despite this, I continue to see X-AspNet-Version in the response.
I wonder: if I don't have multiple headers in the answer, what could be the reason?

[Rhonda Rowland1709152125]

You're sure this policy is being hit?  Since you have the GoTo expression set to END, it will not find additional policies after this one. But If any higher priority policy is hit first, then this and are also set to END, then this one will not run either. If its being hit, that probably isn't a problem. But you may need change the goto to NEXT if other rewrites need to be performed.

 

The server may still be compressing traffic, so you may have to check that first.  Intercept the response from the server before it gets to the Netscaler to see if a content-encoding header is returned. (Or connect a client directly to server to observe responses.)

 

And try some other rewrite on a simpler expression to see if any other response time rewrite is working.  Or change from delete to replace and see if you can change the value or at least insert a custom header to figure it if all rewrite to responses are failing, or just this one.

[Cristina Marletta Livi]

Hi Rhonda,

the rewrite policy on request hits (I can see the hit number in statistics). No more rewrite policies on request (an appfw policy whith security checks in 'log' status and a responder policy IP Reputation).

I have tried another rewrite to remove X-Powered-By, the behavior is the same.

I have tried to corrupt 'Accept-Encoding' header in request instead of canceling it but the rewrite policy on the response hits but it doesn't work anyway.

I'm sure that I' don't have multiple header in response because the HTTP Header Live in Firefox does not show.

 

[Rhonda Rowland1709152125]

You might need to confirm the web server isn't doing compression no matter what (regardless whether header accepts it or not).   Otherwise, something else may be going on. 

 

Which version NS are you running (I know you said 12, but someone might know if there is a version specific issue for a specific build)?

As there was at least one bug with rewrites on a certain 12.1 build:  https://discussions.citrix.com/topic/400796-netscaler-121-build-5028-and-rewrite-policies-not-applying/

It may just give us more information to look at; or you could check with support.

 

And we'll see if anyone else thinks of something before I do.

[Cristina Marletta Livi]

Hi Rhonda,

you're precious!

 

My NS version is: 12.1 49.23.

 

Well, if I try this curl command:

curl  -D - http://www.xxx.it/ScriptResource.axd?d=thSmMF8ofXsF-LxFIG5NGjiy8U6T3iQfLxn7MTpSSqVWBmI3_SiKd4FLNQCiD8abSb0pYA42I-PwiCp06pcAiAXuRaFoYBWXfcxWsiM-4vPd7Wg_0YRLvqXVSv7z-ajMdGDwcHWwOXBLU3nCYWfh2nHtXZltZI_sMZowsvthp1I1&t=ffffffffb3aa5c5a

 

I can see Content-encoding in response http header:

 

GET: HTTP/1.1 200 OK
Date: Fri, 21 Jun 2019 10:22:13 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
Content-Encoding: gzip
Cache-Control: public
Expires: Fri, 19 Jun 2020 18:38:48 GMT
Last-Modified: Thu, 20 Jun 2019 18:38:48 GMT
Content-Type: application/x-javascript
Content-Length: 4848

 

Then, the web server is doing compression even if the request does not contain "Accept-Encoding: gzip, deflate"!

 

[Cristina Marletta Livi]

Hi,

in this article:

https://support.citrix.com/article/CTX131488

they says

Request Headers Dropped by the Application Firewall

Many of the request headers related to caching might be dropped by Application Firewall in order to view every request within the context of a session. Similarly, if the request includes an encoding header to allow the web server to send compressed responses, the Application Firewall deletes this header so the contents in the uncompressed server response can be inspected by Application Firewall to prevent any leakage of sensitive data to the client.

The Application Firewall drops the following request headers: 

......

Accept-Encoding – What encoding methods are allowed for a particular object, such as gzip.

 

Then, if the Application Firewall drop the Accept-Encoding header,  why 'Content-Encoding: gzip' is present in the answer?

[Rhonda Rowland1709152125]

Are you looking at the response from the NS to client or the response from Web Server to Netscaler?

You didn't mention AppFw before, is your traffic being processed by an appfw profile. If not, appfw doesn't apply.

 

IF caching is enabled, then you need to flush cache or none of your existing rewrites are probably occurring.

[Cristina Marletta Livi]

Hi Rhonda,

I'm looking at the response from the NS to client.

Now I have 2 APPFW policies on request and 1 Rewrite policy on response.

It's a test.

Following the article at https://support.citrix.com/article/CTX131488,

Accept-Encoding should be removed from the appfw.

Unfortunately, even following this mode, I still see 'Content-Encoding: gzip' in the response from NS to the client.

I'm really confused.

 

[Rhonda Rowland1709152125]

Ok - If the NetScaler feature for compression is enabled, it will compress NS to client.  What we're stopping is the SERVER to NS compression as this is what prevents rewrite/appfw body transformations from working.

 

REQUEST:

client --> NS --> server

RESPONSE

client <-- NS <-- Server

 

If the server does compression (seen in the server to ns segment of the response), this will prevent NS from doing response time rewrites/appfw body transformations.

NS via appFw, a setting in compression, or using a request time rewrite will corrupt or remove this header to prevent the SERVER from doing Compression.

If the NS feature for compression is enabled, it does NOT stop the NS from compressing the compressible traffic itself for the NS to client segment of the response (NS knows it saw a valid compression header and will compress).

The NS doing compression on this segment does NOT break rewrite or appfw, because those features have already processed.

 

So you can't like at the client side response to see if the server is or isn't doing compression. You have to look at the response leaving the server before it gets back to the NS.

 

 

 

 

[Cristina Marletta Livi]

Hi Rhonda,

the feature for compression isn't enabled.

 

I've tried to run the test directly vs the web server.

I can still see Content-encoding:gzip in response http header.

 

Now, if appfw on request drop the Accept-Encoding header and the NS feature for compression isn't enabled, why

in my test:

curl  -D - http://www.xxx.it/ScriptResource.axd?d=thSmMF8ofXsF-LxFIG5NGjiy8U6T3iQfLxn7MTpSSqVWBmI3_SiKd4FLNQCiD8abSb0pYA42I-PwiCp06pcAiAXuRaFoYBWXfcxWsiM-4vPd7Wg_0YRLvqXVSv7z-ajMdGDwcHWwOXBLU3nCYWfh2nHtXZltZI_sMZowsvthp1I1&t=ffffffffb3aa5c5a

where WWW.xxx.it corresponds to the VIP on NS,

 

the response contains Content-encoding:gzip?