Jump to content
Welcome to our new Citrix community!

Variables, assignments and HTTP Callouts


Recommended Posts

I have been working on a project and have been greatly assisted by @Rhonda Rowland, but am not quite over the finish line yet.

 

Previous thread:

https://discussions.citrix.com/topic/412707-http-callout-returned-data/

 

The crux is, I have a webservice that I can reach out to that returns data in the following example format - TRUE,123456,SomeName. this is an ASN based "Blacklist" to combat service abuse. The data returned by the service is as follows:

 

FIRST value - Was ASN found in a known violator table in the db?

SECOND value - What is the ASN number?

THIRD value - What is the ASN company name?

 

This is functional and actionable by a basic responder policy. My problem is that I need to make this Callout data available to other policies, syslog and weblogs. My understanding is that variables should be able to help here. Doc below, and maybe it's me, but I find the assignment section quite confusing.

https://docs.citrix.com/en-us/citrix-adc/current-release/appexpert/variables/configuring-using-variables.html

 

Here is what I need to accomplish:

 

- Trigger the callout based on the presence of a header value

-Load the returned value into a variable. It is formatted as: TRUE,123456,SomeName

-Rewrite the last 2 positions of the variable data as a header value to pass on to weblogs for the VIP it would be bound to

-Use the value in the first position (IF this value is TRUE) to send to rate limiting (limiting based on the value in the second position)

 

We have a public proxy so all policies are based on XFF. XFF comes in looking like actualCLIENTip,PROXYip. Here is my relevant config at this point:

 

Variable:

add ns variable var_ASN_Index -type "text(64)"

 

Assignment:

add ns assignment assn_ASV_returned_data -variable "$var_ASN_Index" -set "SYS.HTTP_CALLOUT(callout_http_ASNLookup)"

 

HTTP Callout:

add policy httpCallout callout_http_ASNLookup -vServer lb_vsrv_TESTING_ASNCalloutService -returnType TEXT -hostExpr "\"99.98.97.96:80\"" -urlStemExpr "\"/IPBlock/IPBlocked/\"+HTTP.REQ.HEADER(\"X-Forwarded-For\").BEFORE_STR(\",\")" -headers Request("Callout Request") -scheme http -resultExpr "HTTP.RES.BODY(10000)"

 

Rewrite Policy:

add rewrite policy pol_rw_CaseDetails_InsertASN "(HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).PATH_AND_QUERY.CONTAINS(\"/somepage1.html\")||HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).PATH_AND_QUERY.CONTAINS(\"/SomePage2.html\"))" act_rw_insert_ASNData

 

Rewrite Action (This is where I am trying to trigger the callout and load the var):

add rewrite action act_rw_insert_ASNData insert_http_header ASN "$var_ASN_Index.AFTER_STR(\",\")"

 

Responder Policy:

add responder policy pol_res_BadASNCheck "$var_ASN_Index.BEFORE_STR(\",\").SET_TEXT_MODE(IGNORECASE).CONTAINS(\"TRUE\")&&sys.CHECK_LIMIT(\"rt_lmt_idt_by_ASN_\")" act_res_send_to_maintpage -logAction msg_aud_act_ASN_Rate_Limit
 

Rate Limit Selector:

add stream selector sel_rl_ASN "$var_ASN_Index.AFTER_STR(\",\")"

 

Rate Limit Identifier:

add ns limitIdentifier rt_lmt_idt_by_ASN_ -threshold 4 -timeSlice 120000 -selectorName sel_rl_ASN

 

Audit Message:

"Policy Violation: ASN Rate Limit |" + " XFF IP: " + HTTP.REQ.HEADER("X-Forwarded-For") + " | ASN: " + $var_ASN_Index.AFTER_STR(",") + " | HOST: " + HTTP.REQ.HEADER("Host") + " | URL: " + HTTP.REQ.URL + " | USER AGENT: " + HTTP.REQ.HEADER("user-agent") + " | REFERER: " + HTTP.REQ.HEADER("Referer") + " | NSAction: NOOP"

 

 

Rewrites trigger, but neither the header property nor value are injected. The Callout never gets hit.

 

My takeaway from reading is to:

 - Create the var

 - Create an assignment, bind the var and define the callout as the source of the data

 - Call the var with the policy engine which initiates the callout

 - Use the var globally

 

Am I WAYYYY off base here?

 

Thanks as always!

 

 

Link to comment
Share on other sites

You know more about variables then I do.  So, glad you made a little progress. (I was hoping someone would have info for you; but decided to weigh in.)

 

The only thing I'm not sure about are your actions (even if I ignore the variable part of this).

You won't be able to do rewrite and responder on the same transaction.

You also won't be able to evaluate if a header inserted by rewrite is present on this transaction on this adc.  1) Responder runs first so it won't see a header inserted by rewrite unless you daisy chain traffic between two adcs.  2) All rewrites are perfomed at once at end of request side flow so the rewrites are evaluated on original traffic, you can't use rewrite1 to insert a value that rewrite2 (or other feature) looks for.

 

Are you basing the trigger for responder (with ratelimit) and the rewrite (header insertion with logging) on the same callout results or different?
I read this as if the callout returns "true" you are trying to do the rewrite header insert, custom logging, and trigger a rate limit...which can't be done on a single transaction.

 

If instead, the callout invokes one value and you want rewrite to occur, but if the callout has  a different value you want the responder action....then maybe. 

 

If I misunderstood the requirements, then feel free to clarify. 

 

 

 

 

Link to comment
Share on other sites

10 hours ago, Rhonda Rowland1709152125 said:

You know more about variables then I do.  So, glad you made a little progress. (I was hoping someone would have info for you; but decided to weigh in.)

 

The only thing I'm not sure about are your actions (even if I ignore the variable part of this).

You won't be able to do rewrite and responder on the same transaction.

You also won't be able to evaluate if a header inserted by rewrite is present on this transaction on this adc.  1) Responder runs first so it won't see a header inserted by rewrite unless you daisy chain traffic between two adcs.  2) All rewrites are perfomed at once at end of request side flow so the rewrites are evaluated on original traffic, you can't use rewrite1 to insert a value that rewrite2 (or other feature) looks for.

Sadly, I discovered this yesterday. I do have the luxury of having both DMZ and internal ADC's, so that may be an option. How about injecting a header at a CS with a rewrite and then reading it at the LB VIP behind? I also learned that this causes a delay by one call in updating variable contents, but this my be due to the variable type I have selected.

 

 

 

10 hours ago, Rhonda Rowland said:

 

Are you basing the trigger for responder (with ratelimit) and the rewrite (header insertion with logging) on the same callout results or different?
I read this as if the callout returns "true" you are trying to do the rewrite header insert, custom logging, and trigger a rate limit...which can't be done on a single transaction.

 

Sadly, I learned this too. 

 

10 hours ago, Rhonda Rowland said:

If instead, the callout invokes one value and you want rewrite to occur, but if the callout has  a different value you want the responder action....then maybe. 

 

If I misunderstood the requirements, then feel free to clarify. 

 

 

 

 

 

 I will update this thread with the final, working (if I can make that a reality) config.

 

 

Link to comment
Share on other sites

The daisy chain won't work from cs to lb.

If you want to do a rewrite and insert header and trigger logging, before a responder action is invoked, it would have to be from ADC1 to ADC2.  And at that point you'd be better off with running the callout twice as opposed to variables anyway.

 

So even with content swtiching, then lb, the processing flow is 

cs decision is evaluated first to know which lb it will end up on (though its not actually content switched yet)

The policy processing begins:  appfw, responder, caching,...then rewrite

The policy processing for a feature looks to resolve any global override, then cs/lb vserver, then global default, and then moves to next feature.

 

The other problem you have with rewrite in general is this aside from it running after responder:

1) When rewrite policy processing is performed (and it s a little vague where in the flow it is evaluated), all rewrite policies are compared against original (unmodified) traffic to find all applicable policies. All policy matches found basically at once and not in series.  You can't have policy 1 do something that triggers policy 2 on this transaction.

2) Rewrites don't apply until end of processing flow when no further evaluation of original traffic is needed. Then all rewrites apply at once.    We don't "Modify" the packet until we're done with all other inspections.

 

So, the only way I know to get rewrite to insert header, that a responder (or any other feature would see) on the current request would be:
lb vserver1 on NS1 gets rewrite to insert header and then traffic is sent to lb vserver 2 on NS2 and the next process occurs.

 

Also, the point of your rewrite is to trigger a logging header (audit policy) and to insert a header the backend server CAN log as well.

But the point of your responder policy will prevent the traffic from reaching the server...so you still can't do both actions at same time.

 

I think you have to decide what the goal is logging on backend or filtering at ADC.

Or change the nature of the callout to have two different ones; one to identify traffic for rewrites and header insertion (and or leads to the server inserting a cookie that can be inspected for presence either safe traffic or bad traffic) in later transaction so the responder can kick in separately.

 

Good luck; but I think the scenario you are trying to do isn't compatible with the two thinks you want to do.  It may mean you rethink the callout or how to use it. (Regardless of the variable issue.)

 

 

 

 

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...