Jump to content
Welcome to our new Citrix community!

How to stop Netscaler encoding to percent values on backend,


Keith Drone

Recommended Posts

Let me start by saying its been a long.... long.... long week.   So I fully expect I've missed something blatantly obvious.

Build

NS11.1 61.7.nc

 

 

Netscaler, with no App-FW on the VIP enabled is encoding URL to percent encoding

 

Customer >  Inputs URL   https://Website.com/api/anotherapi/1.0/bin/apis/    >   NETSCALER  VIP  >   Backend server gets  https://Website.com/api%2anotherapi%2f1.0%2fbin%2fapis%2

 

 

Nothing fancy here, just passing through the VIP and SSL termination.   We do have a basic 'Transform' policy that changes one of the characters for a legacy service, normalizing it on the backend

 

Can't figure out why its percent-encoding on the backend

 

 

Link to comment
Share on other sites

That was my thought too...

If the traffic didn't go through this transform, would it arrive encoded or not encoded.

So if the transform isn't doing this, I would check to see if any other rewrite features are in use? We might be able to use rewrite to fix this...but I think the original transform might be doing more "encoding" than expected.

 

Link to comment
Share on other sites

I removed the transform policy.  It did not resolve the issue.

 

And now we have more interesting items come up - they are adding some new characters which are being % encoded.

 

I'm starting to think the answer is going to be to do a REWRITE that forces the Netscaler to treat the text as "Thou Shalt Not Encode"

 

Example

Input is

https://website.com/@top?catalogId=1234&responseFormat=json&langId=-1&depthAndLimit=

 

The servers on the backend are getting % encoded, such as the @top becomes %40top

 

Trying a REQ Rewrite as follows is not doing it for me  -  and I've tried numerous options just to shotgun it (including safe characters, encoded/noencoded, etc).

 

HTTP.REQ.URL.PATH_AND_QUERY

to

HTTP.REQ.URL.PATH_AND_QUERY.SET_TEXT_MODE(NOURLENCODED).DECODE_USING_TEXT_MODE

 

The intent here was to force it to not perform any operations on the text

 

I've also attempted a Transform policy and a Rewrite policy to grab the encoded parameters and change them back to their intended characters.   In terms of the order-of-operations in Netscaler, the Rewrite should be the final 'thingy' that it could be affecting, since this is all HTTP and not HTTP/2 so no encoding is occurring there.

 

This almost seems like some odd bug or another, but we can't update the VPX version for a bit unfortunately.

Link to comment
Share on other sites

I don't know if it will help the current issue, but try your rewrite with this expression instead of the one you have above:

HTTP.REQ.URL.PATH_AND_QUERY.DECODE_USING_TEXT_MODE

 

The set_text_mode(nourlencoded) changes the input behaviof of the following operator and it isn't needed in this case. (Just like set_text_mode(ignorecase).eq("something") makes the following operator eq("") do a case-insensitive comparison.  The set_text_mode(nourlencoded) actually tells the following operator to not do decodings (as no encodings are present; used to supress decodings for things that would otherwise appear like an encoding, is my understanding).

 

Good time to confirm if caching is in effect that you also flush cache between test cases/policy changes (and make sure client browser isn't "helping" by caching too...that always adds to this type of fun).

--

Even if the rewrite "Fixes" the issue, it doesn't explain where or how the encodings are happening in the first place.

 

(This is the "assumption" checking phase again...)

Are you absolutely certain there is nothing coming from the client to ADC or between ADC and the backend that could be interfering? Or client side script or any external calls/links affecting additional requests made by the client? 

Can you test a direct connection to the server with no NS/ADC in the middle? (or a different ADC in the middle?)

A trace might be required and or a proxy to inspect at different points.

 

If you think it might be a bug, contacting support is the best idea.

 

You said you had no appfw enabled on the vip in question (and it shouldn't cause a problem), but you're sure there are no other global policy bindings (for appfw or url transform)?
An appfw profile has a canonicalize html response setting that is on by default, but I've never seen it break an app before (but there is a first time for everything).

 

Otherwise, I'm stumped (if it makes you feel in better; doesn't solve your problem but you are not alone.)

  • Like 1
Link to comment
Share on other sites

Epilogue

 

In my limited time I'll try to keep this as..... simple as I can.

 

It wasn't the Netscaler.  

 

What was happening in the background

Input >  VIP  >  Backend Server  > bunch of backend stuff > back to Netscaler VIP (same VIP) as UTF encoding, validating the same call (instead of a loopback or something on that local server)  >  Backend server again > Sent to another system, translated to Base64  >  Sent to ANOTHER SYSTEM validated other data-field > Sent to first server behind VIP but not through Netscaler  to be decoded from base64 and then converted to ASCII > sent BACK THROUGH THE NETSCALER VIP > original server, then presents the perceived issue of percent decoding.

 

So.... when I was looking at the traffic and the packet captures, I saw all the same flippity floppity back and forth through the VIP and 'assumed' it was the calls I was making to test.

 

I frankly feel silly not catching this, but considering the Rube Goldberg method which follows I think I am somewhat validated in my misses here 

 

Thank you all so much for your assistance, I appriciate it!   Issue will be resolved by the developers correcting their app-traffic flow

 

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...