Igel Thin Client - 403 error with Imprivata PIE agent logging into storefront through a load balancer

[sortola27]

Good morning,

 

I've been working on an issue with Citrix Support and Imprivata support over the last 6-7 months and recently was told by Citrix Support we that we should post about the issue here.  

 

We are using an Imprivata Embedded agent on igel thin clients.

 

If we configure the Imprivata agent to connect to our storefront server through 2 KEMP load balancers, we are randomly getting a desktop unavailable error.   If disable the Imprivata PIE Agent and use just a simple Igel Active Directory login, it works fine.  Imprivata however says it's a Citrix issue.

 

Here's the error in the Imprivata Agent Log:
 

2024-01-05 07:52:55,982 - Agent(_log,1613) - DEBUG: CitrixWebResponse: httpStatusCode = 403, dperrorId = None, contentLength = 1233, contentType = text/html; charset=utf-8
2024-01-05 07:52:55,982 - Agent - ERROR: Access is denied.
2024-01-05 07:52:55,983 - Agent - ERROR: You do not have permission to view this directory or page using the credentials that you supplied.
2024-01-05 07:52:55,983 - Agent(_log,1613) - DEBUG: Traceback (most recent call last):
  File "CitrixStoreFrontWebApiClient.py", line 220, in _authenticate
  File "CitrixStoreFrontWebApiClient.py", line 112, in send
  File "CitrixWebResponse.py", line 98, in Parse
packages.Vdi.CitrixBase.CitrixWebApi.CitrixWebApiError.AccessIsDenied: Your desktop is not currently available. Try again.

2024-01-05 07:52:55,983 - Agent - WARNING: Failed to connect to https://storefront.homebank.internal/citrix/desktopsweb/ citrix server. Error: Your desktop is not currently available. Try again.
2024-01-05 07:52:55,983 - Agent - ERROR: Citrix server is not available.
2024-01-05 07:52:55,983 - Agent - ERROR: Unable to connect to XenDesktop. Try again or contact your administrator for assistance.

 

Imprivata had been looking into the issue since July but they have asked us to engage with Citrix, which we have done twice.

 

Imprivata recently asked us to engage Citrix to have the information reviewed by engineering team to see why a 403 error is being returned.  Here's what Imprivata said in that request:

 

"I completed the review. At present, using the diagnostic PiE, there is no problem in the Imprivata workflow.
However, because the 403 error is presented by Citrix, we will need their technical support team to evaluate for reasons why the 403 error is returned.
Can you work with Citrix team, then let me know of your progress?"

 

diagnosis:

Logs still show that Citrix responses with error “403 - Forbidden: Access is denied.”
Logs show that when PiE ask for ICA data then Citrix responses 'reason="notoken" and PIE tries to reauthenticate.
- Reauthentication is successful.
 

Then PiE ask for ICA data once more and Citrix now responses with 403 error:"

 

After we opened our latest case we worked with Sr Escalation team to capture storefront traces when the issue occured.  Here's the feedback from that.

 

"I see more instances of the error I mentioned in the previous email:

It looks similar to a cookie / persistence issue, but I think you would see this across the board instead of just with Linux machines. I unfortunately am not finding any errors in the Store or Authentication logs at the time of the error you documented. 

 

Can you confirm for me if there is an ADC between the SF server and the client? Also, are there any customizations on the SF server? I reached out to one of the product managers to see if there is any insight they could provide in regards to this scenario and she want to verify these questions. More than likely, this will need to go to the SDK forums I mentioned previously."

 

We have tried setting the load balancers to use cookie persistence instead of IP, but the issue still occurs.  

 

So as suggested we're posting the issue here in case anyone can help. If you need any more information, please let me know.

[Jonathan Adams1709163648]

Does it work OK if you remove the load balancer and the client connects direcctly to StoreFront? If so then it's an issue with your load balancer configuration

 

What about if you set your load balancer to always direct traffic to a single StoreFront server? If that works then it's almost certainly due to load balancer session persistence not being configured correctly.

 

My guess of what is happeneing is that in the session you've first authenticated to one server, then when you make a later request it's gone to a different server. This won't work. On the first request to the web api responds ASP.NET_SessionId with a unique SessionId. Every subsequent request with that SessionId needs to be directed to the same StoreFront server.

 

Normally the way to achieve this is with cookie based session persistence. On the first call to the load balancer it generates an additional session persistence cookie. This must have a timeout of the same (or greater than) the session timeout configured within StoreFront. For each request within the session, the client must send the same session persistence cookie. The load balancer reads this and sends all such requests to the same StoreFront server. Web browers include cookies automatically. If your application is not running in a web browser then you need to code your application to remember the session persistence cookie and send it with every request (along with the cookies coming from StoreFront).

[sortola27]

Quote

Does it work OK if you remove the load balancer and the client connects direcctly to StoreFront? If so then it's an issue with your load balancer configuration

 

Yes. if we update the Imprivata Policy to point logins directly to the storefront server machine names, logins works fine.

 

However if we remove the Imprivata PIE Agent from the Igels and set an Active Directory login that's pointed to the load balancers, it works fine.

 

Quote

What about if you set your load balancer to always direct traffic to a single StoreFront server? If that works then it's almost certainly due to load balancer session persistence not being configured correctly.

 

We tried this this as well, but even with just a single load balancer running, the issue still randomly occurs.  We've tried IP Persistence and Cookie persistence with the same results