Jump to content
Welcome to our new Citrix community!
  • 0

Xenserver Migration 7.2 to 8.0 - High CPU Consumption do to interrupts


Ian Gostling

Question

We did a migration of Xenserver 7.2 to 8.0.

And notice that, when we use our usual applications,   the consumption of VMs (windows server 2016) goes to 100% cpu and checking process explorer we have observed that it is due to a high number of interrupts.

 

The VM has the same configuration on both Versions (16vCPU - 16gb RAM - SSD Storage).
The physical Server is the same on both. 
If we increment the Dom0 RAM or cpu the results are the same.
If we increment the vCPU & ram on VM, the results are the same

 

Even we tried a fresh-install of Hypervisor (8.0 + patches) and brand  New VM installation.
And we also have tried the latest xentools and previously version. Even xenproject PV drivers.

 

If we rollback to xenserver 7.2 all works properly.



 

Link to comment

12 answers to this question

Recommended Posts

  • 0

 

Hello Boby John.

 

Thanks for the answer. 

We have tested different versions of xentools, even a windows 2016 fresh-install with xentools provided by windows update, with no changes on the results.

At this moment i cant test uninstalling xentools,  i think the windows crash if you remove the xentools drivers. Probably i can do the test the next week.

 

We notice the heavy interrupts witch process explorer.

When we try lo locate the application with "latencymon" the process related is "wdf01000.sys"

 

The software what  are we running in the VM there are several graphics application, with 3d acceleration.

If we use only one application like a gpu benchmark app, the load is ok. When we use several applications, the performance goes to hell.

The VM has gpu pass-through (no virtualized gpu).

 

We have tried this week Xenserver 7.1-CU2 and no issues in the VMs. The performance  is good as expected, like xenserver 7.2

 

We think the issue can be related with some change in the new linux kernel versions.  Some change on modules, mitigation patch, tuneable on sysctl.

but we don't find out what it could be

 

Regards.

 

 

 

 

Link to comment
  • 0

 

Hello Tobias, thaks for the answer

 

we are using the right gpu drivers for each version of the xenserver / vm.

Is not a GPU issue, is a CPU issue with high consumption on xenserver 7.6 or 8.0

in previous versions (7.1cu2 / 7.2) there is no problem, but we need to upgrade for use new supported cards.

 

Regards.

 

Link to comment
  • 0

 I am assuming the following 

  • With or Without GPU card assigned to the VM - the Issue persists. 
  • XS80E003 is installed and VMs created post installing that hotfix shows the problem as well.  ( this is quite important ) 

What next

  • Take a process-dump of the process which causes 100% CPU and see if it can show "whats" causing. 
  • Look for errors in /var/log/daemon.log during the issue 

 

 

Link to comment
  • 0

we suffer from the same issue, which is kind of hard to find. We got a support ticket at citrix, but as always, no solution in sight...

 

We did a lot fresh, clean installs with XEN (on different hardware), HyperV and VMWare.

At the moment, it seems to be a Server2016/2019 vs. XEN 8.X problem..

 

no problems with hyperv and VMware nor 2008R2/2012 on XEN...

 

 

...one big Sh...

Link to comment
  • 0

We did a lot of research in the past six weeks. We had a customer with a full stop of new environment rollout. At the end we rollback to Server 2012R2, as the performance was not acceptable.

 

Some other tests with HyperV und VMware delivered an acceptable CPU / Latency behavior. Citrix Support has no clue and dropped some hopeless comment: ..we were unable to meet some customer needs in the past.

 

Main problem, which we discovered also under XEN7 LTS (fully patched), where sluggy feeling and slow cpu performance - but full load in xen.

 

In TOP we see high tapdisk load and xenTOP some high DOM0 load... nu clue if its ok or not. As we never had performance issues in the past.

 

PS.: DOM0 ist pinned to physicial CPUs in our cluster, to ensure highest perormance. iostat shows low disk utilization.

Link to comment
  • 0
On 5/19/2021 at 12:22 AM, Stephan Guumlnther said:

We did a lot of research in the past six weeks. We had a customer with a full stop of new environment rollout. At the end we rollback to Server 2012R2, as the performance was not acceptable.

 

Some other tests with HyperV und VMware delivered an acceptable CPU / Latency behavior. Citrix Support has no clue and dropped some hopeless comment: ..we were unable to meet some customer needs in the past.

 

Main problem, which we discovered also under XEN7 LTS (fully patched), where sluggy feeling and slow cpu performance - but full load in xen.

 

In TOP we see high tapdisk load and xenTOP some high DOM0 load... nu clue if its ok or not. As we never had performance issues in the past.

 

PS.: DOM0 ist pinned to physicial CPUs in our cluster, to ensure highest perormance. iostat shows low disk utilization.

I would love to discuss this with you. We are new to XenServer and experiencing the same issues. We are trying to move to XenServer in order to utilize vGPUs as we are unable to do this with Hyper-V and VMWare is too expensive. Has anyone been able to deliver acceptable performance with XenServer and Server 2016/2019?

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...