Jump to content
Welcome to our new Citrix community!
  • 0

WAF profile blocking certain images from loading


Question

We have an odd one, we have a waf profile bound to a vip, when bound, certain icons are blank. When I unbind the waf profile the site works fine. I've unblocked all security checks, blanked out the sigs, checked and unchecked everything in the profile. It's certain SVG files even though other ones seem to load fine. Any ideas? Also, it's not logging anything flagging for appfw for these in ns.log.

Link to comment

23 answers to this question

Recommended Posts

  • 0

Anything not covered by start urls (or url closure) will be blocked.  

1) .svg extensions are not in the default list of closure is OFF

2) OR the path for your media files is not covered by closure or are coming from a separate import server or something other exotic.

 

First, start by looking at syslog or use learning to figure out the blocks being applied to the svg graphics are in fact coming from start urls.

 

Then depending on whether

a) Start URL Closure is ON or OFF

b) Whether the graphics are coming from a specific directory, same server, or different server would affect how you would allow them.

To test this, you can use the browser F12 or other web tools to confirm for the page and object requests for each .svg which path they are coming from and if they are based on relative or absolute path (possible referencing a remote media server.)

 

 

But start with syslog and the start url closure setting and then we can go from tehre.

 

Second, since you said you are not seeing anything in syslog, check the following:

1) are you sure you are in /var/log/ns.log and not /var/nslog/ns.log (not the same file)  And are you getting any appfw events at all?

2) Check your local syslog parameters to be sure what logging level is included

3) Check signatures / start url / deny url to ensure the "logging" is in fact enabled for each security check

4) Finally, confirm if the appfw logs were configured to log separately from syslog (per this article):    https://support.citrix.com/article/CTX138973

 

shell

cd /var/log

tail -f ns.log | grep APPFW

 

 

Link to comment
  • 0

Wow!  I appreciate the response on a Saturday.

 

We're not blocking Start URL's, at least in the Security Check section, unless I'm missing something?  The graphics are coming from the web server itself in a deeper folder, no remote call.  I am most definitely tailing the /var/log/ns.log file and am seeing appfw hits, just nothing when these icons don't load after refreshing the page.  Local syslog parameters, are you referring to the System -> Auditing -> syslog stuff?  They're all set to ALL, we send those to a separate server but APPFW should write the same info to ns.log right?

Link to comment
  • 0

What is the config of your profile then?  Blocking OFF but log/stats on?

Enable learning and see what it is seeing.

Look for any appfw block or "not blocked" event to see if there is some other security check like content types (logged as secure commerce) causing a problem.

 

And again, load in a browser when its not protected to look at paths and calls for the icons/objects to see how to "allow" them.

 

But again,

1) What is the Start URL settings for: url closure on or off and referer header validation on/off?

2) Do you have the default start urls AND any inital start url patterns defined?

3) Does learning find any rules to allow this graphics

4) Are any other appfw violations occurring resulting the graphics being blocked that aren't start urls.

 

Depending on how familiar with appfw you are, it will block anything/everything not covered by the start urls.  Now, even if blocking is off there may be something else going on...but hard to comment without some config info to go on.  So, now you want to ensure logging is in fact enabled on signatures and other security checks in use and see if any violations are logged for this content type to help figure out cause and how to resolve.

 

You can also enable the appfw "trace" option in the profile properties and then create an nstrace with the "appfw" option enabled for this vserver to see network events and appfw events in trace if you need more info.  (Both a profile and trace property. I would filter the trace to just this vserver or just your test client source ip and the vserver vip destination with "trace filtered connection's peer traffic" also enabled.)

Link to comment
  • 0

I really do need answers for all questions I've been asking, even if you don't want to share the exact urls.  To know what is configured is really important.

 

1) enable log/stats while leaving block off to see if events are related for Start URL

2) In your start URL relaxations, a) are the default rules enabled/disabled and b) do you have any custom rules defined.

 

Typically, you ALWAYS start with start urls and make sure page loads properly.  Even though you have no blocking now, certain security checks are DEPENDENT on start urls being configured. This is the one step you should never skip.

 

3) Can you share the complete event log message you are seeing, so we can see the full message. I'm fine if you want to omit the <path/query> in use, but it would help to see what "disallow illegal url" message you are getting and which security check is logging it.

Link to comment
  • 0

May 22 17:39:44 <local0.info> x.x.x.x CEF:0|Citrix|NetScaler|NS12.1|APPFW|APPFW_STARTURL|6|src=x.x.x.x geolocation=Unknown spt=63973 method=POST request=https://xxxx.x.x/xxxx/Clinical/CareTeam/LoadExternal?hfrId\=&sources\=&actions\=&ComponentNumber\=2&noCache\=0.32327597945235986 msg=Disallow Illegal URL. cn1=857648 cn2=5414719 cs1=APPFW_web_Profile cs2=PPE0 cs4=ALERT cs5=2021 act=not blocked

 

Here's what I see when I enable log for Start URL.

 

I've attached what is in Start URL relaxation rules (I added svg)

mychopqaafw3.jpg

Link to comment
  • 0

So, even though Start URL blocking was OFF, you are getting violations.

 

With Closure OFF, your start urls act as a URL whitelist and anything not covered by the default rules is in violation.

The two default rules allow a URL of any static content (based on one of the extensions listed) without query parameters and the other rule allows any of the dynamic extensions listed with or without query parameters.  (Start URLs behave differently with closure)

 

This specific request is denied, because the URL pattern does not match the regex because there is no "extension" or expected value in the path portion of the URL:
https://xxxx.x.x/xxxx/Clinical/CareTeam/LoadExternal?hfrId\=&sources\=&actions\=&ComponentNumber\=2&noCache\=0.32327597945235986

<path>:  ends with LoadExternal and no recognized extension.

<query> therefore isn't expected either.

 

While learning, may help, in general there are certain things you should ALWAYS do prior to deploying the AppFirewall.

 

1) Always create the default "main" URL to identify known good traffic.  Therefore, learning will learn the things that are NOT working for the additional relaxations you need.

Example:

If your website has the following url examples:

http://someweb.demo.com/dir1/somepath.cgi

https://someweb.demo.com/dir1/somepath.cgi?a1=b1&a2=b2

http://someweb.demo.com/dir1/someotherpage.htm

 

Then you at least need the base url. Recommended to be narrowly defined like:

^https?://someweb[.]demo[.]com/$

This will exactly match http:// or https:// and the FQDN in use.  Then the other start urls will allow most common content/patterns.

 

Then learning can help you see what else you need.

 

However, your website is navigating by pages without extensions therefore most of the "heavy lifting" done by the default URL patterns will not apply.

As a result, you may have to go broader with the allowed URL patterns to allow the wide range of allowed directory structures and query parameters.

After the initial URLs are added, clear the current learning and learn again.

But you may have to use the learning visualizer to deploy some very broad rules so the site will navigate.

 

Switching to URL Closure may help, but they would be a different discussion.

 

2) You still may end up with a fair amount of additional URL patterns needed, but you can ensure the main site/default page works first and then find what else is needed.  

Otherwise, "everything" already in violation is a much bigger issue and harder to sort good from bad.

 

Since your above URL is also a POST there could be some other dependencies.

 

 

 

 

Link to comment
  • 0

One last point, the "not blocked" messages are not being stopped. Just observed that they are in violation of your start urls.

You may to identify in a web browser/web developer tool which objects are not being retrieved and then look for specific violations in the log related to those paths.

Or look at the requesting objects "page source" to see if the dependent object is coming from somewhere else (like an import from a media server, which would need to be included in the allowed start url patterns.)

Link to comment
  • 0

I think this is the frustrating part, normally I can tail the ns.log and see an appfw block and hone in and remediate...this particular issue isn't producing any blocks, so I have no idea where to hone in.  Whatever code they deployed for this particular upgrade already caused issues a few weeks ago by breaching the buffer overflow policy, which needed to be increased...but at least with that I was able to see a block and find it.  The page source is local to the server, you don't think it has to do with cookies and having persistence enabled on the vip do you?  Like maybe it can't see inside the cookie, therefore it bombs?

Link to comment
  • 0

One thing you can try is just creating a rule that allows .svg extensions.

I would also check to see if the test client has caching enabled and disabled OR the vserver has caching enabled and be sure to disable ic or clear the cache.

^[^?]+[.](svg)([?].*)?$

basically a rule for the .svg extension with query parameters allowed as your current default rules do not accept it.  This is modelled after the dynamic script rule (in case I fat-fingered it).

 

But remember, you have LOGGING off on your start urls so logging may not be hitting.

 

While load balancing and persistence should not be a problem, that assumes that appfw policy is at the lb vserver level (and that you aren't also doing content switching).

That your appfw policy applies to all traffic on the vserver and not just some.

 

 

 

Link to comment
  • 0

It's possible; i've had a few customers whose default syslog local settings were set to only error and higher and so were missing all the appfw events AND they were logging externally, so the local log didn't have what they expected.  Usually, that would result in no appfw events in the local syslog. So while something interesting might be going on and its always worth checking, it doesn't seem to be the case with what I know so far.  

 

1) The way to check is to see if the config settings noted in the article are in fact present on your system. If not, you shouldn't have a problem of the separate log file.

1b) Check for the presence of any other audit policies bound to either the system global or other vservers. Usually, with audit policies all audit policy destinations will be hit (so it usually results) in multiple logging locations...but it can be checked if there is an audit policy logging some content externally.

 

2) However, ANY security check that doesn't have LOG enabled is not logging events to syslog, so there could be events or security checks not logged depending on you security check values.  Usually, if you are not processing that check logging (and all other actions will be disabled). But as I mentioned you always want your start urls defined in most cases.

 

3) Check to see if any other appfw policies are in effect by checking global appfw bind points and any other vservers.  If your media content is coming from lb_vsrv_2 instead of lb_vsrv_1, there might be some additional appfw policies in effect (I doubt it; but things I would check for).

 

Normally, in testing, disable the appfw profile. Do a trace to see the original unaffected traffic. Then enable the profile and compare the trace (with appfw trace enabled) (You may not be able to do the first part if you are actively protecting a production website.  Simplify the profile and test only a subset of protections at a time. For the web services content, confirm your wsdl/schema files.  And see if by adding protections a few at a time you can find which area the errors are occurring.  

 

4) Check if you have any responder or ip reputation policies that could be filtering traffic and if so, add an audit message and add "user configurable log messages" to syslog and see if some other feature besides appfw is preventing the requests.

 

 

Review the appfw engine settings and profile settings for anything unexpected.

 

 

 

 

Link to comment
  • 0

So here's where we're at at this point:

 

Initially I was able to get the issue to resolve half the time by choosing "ALL" for Strip XML Comments in the Profile settings of the WAF profile.  This, however, was only resolving it half the time, you would refresh a few times and it would break, refresh, would work, etc.  After talking with Citrix support, he noticed that it could possibly resolve the issue if we disabled XML Format, XML Attachment and XML SOAP Fault Filtering completely (block, log and stats).  This is now allowing the site to load fully without any issues.  We're still trying to figure out why having these on, even with block off, is breaking the XML in the page when it shouldn't be.

Link to comment
  • 0

There's not a lot of documentation on what Soap Fault is triggering on beyond "soap faults" returned by the web server. 

Beyond its a response-time protection with either BLOCK or REMOVE actions.

You may still need to go through support for why it is not logging as expected; but you may need to check if the server is generating responses that look invalid Or if the security check is not working as expected.  Or make a new post to see if anyone has ideas about this check specifically.  I'm sorry, I don't have any info for you. 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...