Jump to content
Welcome to our new Citrix community!
  • 0

Citrix Hypervisor Performance Tips


Julien DESTOMBES1709158802

Question

Hi all,

 

Few month ago I post this thread https://discussions.citrix.com/topic/407437-citrix-hypervisor-performance-tuning/#comment-2071495

 

Quote

Hi,

 

I need to configure Citrix Hypervisor 8.1 on HPe Synergy 480 Gen10 for XenApp Workload (no GPU).

 

I found HPe whitepaper on BIOS tuning for XenApp workload 

https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=a00049199enw

 

Can I also apply those settings to Citrix Hypervisor ? For instance, the whitepaper disable C-State but I read an excellent post of Tobias that says to enbale C-States. 

https://www.mycugc.org/blogs/tobias-kreidl/2019/03/07/tale-of-two-servers-bios-settings-affect-apps-gpu

 

Thanks for your advises !

 

Regards 

Julien

 

Now we are on migration and we observe that CPU consomption is very high (30 users take 90% of a 8 vCPUs VMs and 40% of the host CPU with only one VM on it).

 

The CPU are 2 x  Intel Xeon-Gold 6244 (3.6GHz/8-core/150W) so we use VMs (4 by host) with 8 vCPUs and all Citrix optimization from Citrix Optimizer.

 

WEM is also in place.

 

BIOS is configured like this:
 

Power Management (Power Profile): Maximum Performance

Power Management (Power Regulator): Static High Performance Mode (Default)

Minimum Processor Idle Power Core C-State: No C-states

Minimum Processor Idle Package C-State: No Package States

Intel QPI Link Power Management: Disabled

Energy Performance Bios: Maximum Performance

QPI Bandwidth Optimization (RTID): Optimized for I/O (Alternate RTID)

Hyper-Threading: Enabled

Intel Turbo Boost: Enabled

 

What is your suggestion to improve performances ? I don't know if we can improve something but I am quite disappointing to see that one VM take 40% of CPU of my host.

 

Thanks for helping me to troubleshoot this issue :)

Link to comment

13 answers to this question

Recommended Posts

  • 0

What sort of storage do you have? How is it configured? Now many VMs per SR do you have? Also, whats your dom0 memory set to use? Run top and xentop from the CLI and look for bottlenecks, like CPU saturation, swapping, etc. Also run iostat and look for disk I/O bottlenecks like high await and read/write times or saturated queues.

 

There are so many possible reasons for this, it's not just a simple do this or do that to fix it type of situation. People spend years studying how to optimize performance.

 

-=Tobias

Link to comment
  • 0

Hi Tobias,

 

 

I know there is not a simple answer to advise to me: tick this option and everything will be great :6_smile:

 

But your advises to check xentop, swapping and do iostat are great ideas. The customer pin me about this problem and I go on site on Thursday so I try to collect everything I can to gain time.

 

To answer your questions:

1- What sort of storage do you have?

We use SAN storage 

 

2- How is it configured?

We use fiber channel

 

3- Now many VMs per SR do you have?

We are at the beginning of the deployment so no more than 8 VMs are deployed on the SR with MCS

 

4- Also, whats your dom0 memory set to use? 

Dom0 was configured with default settings (so it should be 8 GiB according to Citrix docs "By default, Citrix Hypervisor allocates 1 GiB plus 5% of the total physical memory to the control domain, up to an initial maximum of 8 GiB"; we have 512 GiB on our servers).

 

Thanks very much for the time you take to answer me. I really appreciate. 

 

Regards

Julien

 

Link to comment
  • 0

Julien,

Make sure to check storage I/O stats. This is often one of the most overlooked areas and it is critical to good performance. With that small a number of VMs per SR, the queue depth should not be an issue. Consider also various read cache options, in particular being now on 8.X.  We used a NexentaStor array that with our provisioned MCS VMs, achieved a very high cache read hit rate and because it was all SSD drives, reads were super fast in any case.  With iostats, any delays in storage I/O over around 20 to 30 msec will start to be noticeable.

 

-=Tobias

Link to comment
  • 0
Quote

Power Management (Power Profile): Maximum Performance

Power Management (Power Regulator): Static High Performance Mode (Default)

 

Isn't it recommended (for a while now) to set it to OS?

 

Edit: Go to console (exit to shell) and use top to see if it's io-waits or how dom0 behaves.

(Press 1 to see it for each core you assigned to dom0)

Link to comment
  • 0

For storage. the iowait and access times are important. Overall performance is not just about the CPUs. As to the CPU, whether or not to set to maximum performance depends on other factors, such as if GPUs are being used or not. Enabling C-states is generally recommended. See for example https://www.mycugc.org/blogs/tobias-kreidl/2019/03/07/tale-of-two-servers-bios-settings-affect-apps-gpu  and the follow-up two articles https://www.mycugc.org/blogs/tobias-kreidl/2019/04/30/a-tale-of-two-servers-part-2 and https://www.mycugc.org/blogs/tobias-kreidl/2019/04/30/a-tale-of-two-servers-part-3. Also, when configuring large VMs, NUMA/vNUMA should be taken into consideration, in particular for older CPU architectures.

 

-=Tobias

Link to comment
  • 0
3 minutes ago, Christof Giesers said:

Right, I remember that if you disable all "savings features", e.g. C-states etc. it can't utilize the turbo states as other cores need to scale down to let a few scale up.

Indeed - you don't want to disable C-states, ever. Turbo mode is a matter of a whole separate discussion - see the references to my articles above, for example.

 

-=Tobias

Link to comment
  • 0

Hi all,

 

I want to share you my experience return.

 

We try different BIOS config and for my perspective OS control and turbo mode give a good improvement. 

 

However I want to try another config to disable C-State as I see on this blog it could impact CPU usage

https://www.wedelit.no/blog/2018/01/05/xenserver-c-state-or-not-to-c-state/

 

I was thinking that C-Stated was disabled, I checked BIOS and it is but if I run 

xenpm get-cpuidle-states | grep total | uniq

I still see 4 and not 2 as result. Is there a file or something to do to update CPU info ?

 

Thanks for your help.

Julien 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...