Jump to content
Welcome to our new Citrix community!
  • 0

Increased slowness as the day progresses


IMG Support

Question

We are in the progress of migrating from XenApp 6.5 on Server 2008 R2 to XenApp 7.15 on Server 2016.  We publish full desktops for our users.  My users are reporting that everything gets slower as the day progresses.  Im checking everything I can think of.  CPU times and available RAM is fine on my PVS, DDC, and XenApp servers.  PVS retries are low or 0.  ICA RTT and Latency in director is low.  I tried disabling all of the CPU, Disk, Network DFSS on my XenApp hosts.  Users reported in the morning that everything was quicker but still progressively got worse as the day progressed.  Any ideas?

Link to comment

8 answers to this question

Recommended Posts

  • 1

Screenshots still would help to help verify that the loads you claim to be perfectly ok are really ok but I have a feeling they are. Therefore I feel you already know where this problem will end up being at: Backend/frontend slowness of the specific EMR application you are using. So my money is on the application vendor.

 

Nevertheless I would go through the default process of elimination in order to rule out the application by setting up and testing specific scenario's that together allow you to isolate and prove -most importantly- where the problem resides (to your application vendor). I'll give some practical examples of such isolation tests and scenario's:

 

1. Set up an isolated delivery group for a handful users or even only a single 1 but make it point to just 1 isolated vm. This way you can prove that multiple users on a VM xenapp server or just 1 make no difference since they experience the same slowness at the end of the workday

 

2. Set and define a benchmark for disk response speed, cpu load and response and memory using different local (on the xenapp server) applications and time them in morning and at the end of the workday. If they are the same but only the EMR application becomes slow you have proof it's not related to the xenapp scaling nor server capacity

 

3. install the application on a standalone physical machine and have a user work on that for a day. See if the slowness happens there too

 

Continue further narrowing down by inventing more and more precise scenario's in this way (and collect all data as proof) in order to eliminate and deduce more and more until you know exactly where the differences are happening. It will require a close eye and followup. Good luck !

  • Like 1
Link to comment
  • 0

hey,

 

you'll need to be alot more specific:

 

1.  What exactly do users experience and have you verified with your own eyes ? How is this 'felt' ? Which exact operations behave different and how ?

 

2. do you have screenshots of your problematic virtual xenapp servers under load later on the day ? Showing memory usage, cpu usage etc

 

3. How are your vm xenapp servers sized currently and on what host hardware ?

 

4. How many users you have per vm xenapp and per physical host server ?

 

Let's start from there

Link to comment
  • 0

Thank you for your quick response Andy.

 

1.  Our primary application is an EMR application and all of the data resides on a backend DB server.  I do not suspect any of the backend stuff to be an issue because most of my company is still running in XenApp 6.5 and not reporting any issues.  Yes, I have seen the slowness with my own eyes.  In the morning, clicking around in the application the data on each screen loads quickly.  In the afternoon it starts taking 3-5 seconds for the data to load when they move screen to screen.  I have contacted the application vendor and they of course say it is Citrix and they don't support Citrix (even though we've been using Citrix for the last 12 years and they know this).  Looking at the process in task manager I do not see anything unusual like a memory leak.  Closing and reopening the application does not resolve the issue.  Logging off and back on to Citrix sometime offers temporary relief, but not always, and starts to slow down quickly.

 

2.  I do not have screenshots but I can get some this afternoon and provide them.

 

3.  My VMs are 4 vCPU (1 socket x 4 cores) with 32GB of RAM.  For PVS im caching on device hard drive.  I've been watching my .vdiskcache file and it doesn't seem to be growing at an usual rate and there is plenty of free space on the drive.  I am running 7 VMs on each host (24 pCPU/48 lCPU with 384GB RAM).  I am watching my CPU RDY times on the Citrix VMs in VMware and they are typically <2.  We run on Nutanix.

 

4.  I am currently getting 12-13 users per Citrix VM.  I did experience a loss in density from XenApp 6.5 on 2008 R2 to XenApp 7.15 on Server 2016 but this was expected.  On same sized Citrix VMs for XenApp 6.5 I get 23-25 users.  I did try getting up to 15-17 users per Citrix VM on XenApp 7.15 but I was maxing out my CPU times.

 

I pretty much have a vanilla install of Server 2016 on ESXi 6.0 with XenApp 7.15.  I did run the Citrix Optimization tool with the 2016 template on my PVS image.  Im looking at possibly disabling QoS on the vmxnet3 adapter.

Link to comment
  • 0

Given that the problem seems isolated specifically to users on the new environment, and assuming both environments connect to the same backend DB and/or other services, have you evaluated networking differences between the 6.5 and the 7.x environment?  It certainly does not sound like the issue is with the Citrix infrastructure.  Another possibility might be with Antivirus, did you remember to add the new environment servers to your AV exclusions container for Citrix as well as updating exclusions for Win2016?

Link to comment
  • 0

All of the Citrix VMs live on the same physical vmware hosts as the old environment.  Everything is the same on the next working side with any exception to what may or may not be configured in the Citrix VMs.  I know starting in Server 2012 Microsoft added disk and network DFSS which I have disabled.  I did get some positive response back from that.  However, my users are still saying things get slower as the day progresses.  I double-checked A/V and my config is the same.  We run Symantec Endpoint Protection.  I do and always have added the network and intrusion protection pieces but I might remove those just to see if it makes a difference.

Link to comment
  • 0

Citrix publishes a list of files/folders that you should setup exclusions to be exempt from Real Time AV scanning.  Depending on your Antivirus solution, you would typically configure your AV policies to exclude those files/folders scanning.  You would either do that Enterprise-wide, or setup a group/container that contains only XenApp servers.

Link to comment
  • 0

Ok, yeah I have all of that.  I referenced Citrix, Symantec (our AV), and Microsoft's A/V best practice articles and have all of the exclusions.  The way you worded it made me wonder if there was something special about Server 2016 that I was missing.

 

I did see that starting in Server 2012 Microsoft added disk and network DFSS.  I have disabled those and did get some positive feedback from my users. 

 

I did find out from our EMR vendor that their application is a WPF application and those don't play nice with terminal services/citrix.  One thing I haven't tried in that regards - our old environment was set to 16-bit graphics and the new environment is at 24-bit.  A lot of sites I found said for performance reasons it should be set to 16-bit.  Anyone have any experience or thoughts there?

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...