Rewriting json contents in a HTTP response

[Kari Ruissalo]

Hi,

A web server is responding with a json (/config/config.json) file that contains a static IP in the payload with http protocol similar to this:

{
  "restUrl": "http://10.123.123.123:8081"
}

Apparently some JavaScript is using this value and it's not rewritten on NetScaler. I would make the contents look sth like this:

{
  "restUrl": "https://cool.reverse.proxy"
}

I have tried a replace all similar to this, but it doesn't seem to do the trick:

add rewrite action rw_act_cool.reverse.proxy-config.json replace_all "HTTP.RES.BODY(120000)" "\"gotcha\"" -search "text(\"rest\")"

... but it doesn't seem to do anything.

I already made sure the content isn't encoded, so that shouldn't prevent the rewrite. Any ideas?

[Rick Davis]

add rewrite action kr_test replace_all HTTP.RES.BODY(120000) q{"https://cool.reverse.proxy"} -search text("http://10.123.123.123:8081")

Probably best to use Rewrite since this is a json response.  Be sure to bind it as a response rewrite to a vserver type of SSL or HTTP.y

Reference:

https://docs.netscaler.com/en-us/citrix-adc/current-release/appexpert/rewrite.html

[MBP_NetScaler]

Hi Kari,

 

I think you need to use XPATH_JSON when you want to search within json
Let me know if you need more help

 

[Rick Davis]

8 minutes ago, MBP_NetScaler said:

Hi Kari,

 

I think you need to use XPATH_JSON when you want to search within json
Let me know if you need more help

 

 

xpath is a powerful search tool and worth knowing about. 
Definitely useful if there are multiple keys with the same value but you only want to edit one of them.  

[Juliano F. Reckziegel]

Make sure your response is not compressed, otherwise the NetScaler will not "see" the content of the response and will not find the string you are looking for.

[Kari Ruissalo]

16 hours ago, Rick Davis said:

 

xpath is a powerful search tool and worth knowing about. 
Definitely useful if there are multiple keys with the same value but you only want to edit one of them.  

Gotcha, got the similar result, but how should I set the actual rewrite, I tried REPLACE_ALL just to change the string "http" to "https", but it doesn't seem to do so. My policy expression is set to "true" so that it will hit all the req/res for now on that level (will narrow it down naturally when I get this bit working).

 

15 hours ago, Juliano F. Reckziegel said:

Make sure your response is not compressed, otherwise the NetScaler will not "see" the content of the response and will not find the string you are looking for.

As I mentioned in my initial post "I already made sure the content isn't encoded, so that shouldn't prevent the rewrite.", this is what the response looks like:

[Rick Davis]

I can't craft the XPATH without being able to see where RestUrl exists in json response payload.   But I don't see how XPATH is necessary for your use case.  

Use this instead:

add rewrite action kr_test replace_all HTTP.RES.BODY(120000) q{"https://cool.reverse.proxy"} -search text("http://10.123.123.123:8081")

 

[Kari Ruissalo]

55 minutes ago, Rick Davis said:

I can't craft the XPATH without being able to see where RestUrl exists in json response payload.   But I don't see how XPATH is necessary for your use case.  

Use this instead:

add rewrite action kr_test replace_all HTTP.RES.BODY(120000) q{"https://cool.reverse.proxy"} -search text("http://10.123.123.123:8081")

 

This is what I started with, but the rewrite doesn't happen. I have also flushed the cache as we can see in the response header "Via" the value "NS-CACHE-10.0: 10", but that doesn't seem to do anything either. It's pretty weird that the cache header is showing up as the appliance is on Advanced licensing and it's just a TM/Load Balancer.

I do understand the benefits on the XPATH stuff now and can leverage it later for other use cases!

[Kari Ruissalo]

I actually did another rewrite and can clearly see now that I'm able to rewrite an extra HTTP header in the responses for other file types, like this .js file

... but the header doesn't appear in the .json responses:

[Rick Davis]

You are writing about  a REQUEST rewrite.  Your post indicated the need for a RESPONSE rewrite in order to to change a server response payload . 

 

Reasons a Rewrite policy may not get a hit?

Answers:

any of the following:

1) If the  configured Rewrite policy Expression is incorrect.

2) If the Rewrite policy is not bound to the right Vserver (Load balancing or Content Switching or NetScaler Gateway Vserver).

3) if the Rewrite is bound in the wrong direction (Request vs Response).

4) If more than one Rewrite policy is bound under Vserver with the expression as "ns_true" or "TRUE" and if the GOTO EXPRESSION is set to END instead of NEXT.
 

[Kari Ruissalo]

1 hour ago, Rick Davis said:

You are writing about  a REQUEST rewrite.  Your post indicated the need for a RESPONSE rewrite in order to to change a server response payload . 

 

Reasons a Rewrite policy may not get a hit?

Answers:

any of the following:

1) If the  configured Rewrite policy Expression is incorrect.

2) If the Rewrite policy is not bound to the right Vserver (Load balancing or Content Switching or NetScaler Gateway Vserver).

3) if the Rewrite is bound in the wrong direction (Request vs Response).

4) If more than one Rewrite policy is bound under Vserver with the expression as "ns_true" or "TRUE" and if the GOTO EXPRESSION is set to END instead of NEXT.
 

In the example above I'm rewriting a header to the response, not sure if you can add one to a JSON file at all. I'm familiar with the concept of response rewrite. This is how I have configured the LB vServer:

add rewrite action rw_act_domain.dom-config.json-test replace_all "HTTP.RES.BODY(10000)" "\"https\"" -search "text(\"http\")"
add rewrite policy rw_pol_domain.dom-config.json-test true rw_act_domain.dom-config.json-test
bind lb vserver lb_http_domain.dom -policyName rw_pol_domain.dom-config.json-test -priority 100 -gotoPriorityExpression END -type RESPONSE

The .json file response looks like this:

{
  "restUrl": "http://10.123.123.10:8081"
}

Shouldn't the response rewrite above replace the "http" with "https" (trying to avoid any escape characters and such at this point, therefore not rewriting the whole string)?

[Rick Davis]

When I tested this, I got a "304 Not Modified" answer from the server which means it's not actually sending the payload. 

When I disable NetScaler Integrated caching and force the browser to reload the page, ignoring cached content, it works. 

 

[Kari Ruissalo]

11 hours ago, Rick Davis said:

When I tested this, I got a "304 Not Modified" answer from the server which means it's not actually sending the payload. 

When I disable NetScaler Integrated caching and force the browser to reload the page, ignoring cached content, it works. 

 

I have cleared the cache (nsapimgr_wr.sh -ys call=ns_ic_flush) several times even that the NetScaler is running Advanced licensing. It's really weird it even sends the "via" header in that sense...

[Rick Davis]

Please post a packet capture.

[Kari Ruissalo]

Many thanks to you @Rick Davis, I opened a support case of this one due to security reasons. Seems to me that for some peculiar reason the "Via" header just stays there and I can't see my request hitting the web server and it's just served from the NetScaler cache even it's not licensed.

I even did delete all the load balancing configuration related to this specific URL and recreated them, but the header just stays there.

[Rick Davis]

Yeah, between the browser, NetScaler IC, and intermediate proxies, cached content can create a frustrating situation when trying to craft Rewrite actions.  

I'm glad you could tell this was occurring by by looking at the requests coming (or not coming in this case) to the server. 

Just to recap a couple ways of dealing with this situation (for future readers):

- If you use Private browsing mode in your browser, your browser will not send the via header or cookies it learned from other browsing sessions. 

- It's good you cleared the cache on the NetScaler at one point, but you can also allocate zero memory to the integrated caching module to disable it.  (though it's not enabled so it should only be caching for the gateway module) 

- Rewrite can also be used to remove via headers.  

[Kari Ruissalo]

Seems that this is somekind of bug. I got the rewrite working partly after we upgraded the environment from 13.1-52.19 to 13.1-53.17, I managed to rewrite the json file once so that now the protocol is https instead of http, but now the response is again from the IC. This drove me to reboot the secondary and try this after failover, but unfortunately that didn't help.

I have gone through the listed things @Rick Davis and it seems I'm unable to rewrite the header and it just persistently stays there. I'll keep you updated of what we'll find, the case itself was escalated already by support.

[Kari Ruissalo]

The support instructed me to create a NOCACHE policy and apply it to the config.json. Once I did that, the rewrite started working flawlessly.

Support stated

Quote

As the NetScaler has to optimize itself to perform better, the IC feature itself is used on Gateway VIPs and where ever necessary.

... and ofc I asked them what are the cases "wherever necessary" :D