Design Decisions: Citrix DaaS for Azure

This document provides guidance and resources to help Citrix customers design Citrix DaaS solutions on Azure. The different sections contain a list of questions to help you better understand the design decisions that you need to make before deploying Citrix in Microsoft Azure. We will cover 4 design areas. System Level will cover Citrix and Azure cloud consideration. We then dive into design considerations for the workloads run in the system - the Citrix VDA’s. From there we jump into the user specific consideration. We wrap it up with network/security considerations.

Note:

This guide is not intended for Infrastructure as a Service (IaaS) deployments of Citrix in the Azure cloud. The guide focuses solely on deploying using the Citrix Cloud.

System Level Design Considerations

The System level refers to the infrastructure that is core to the Virtual Desktop Infrastructure technology. This is the base layer of the solution and needs to be crafted carefully. Put the required time in planning this layer before rushing into deploying the workload! In this section you will find a list of items to help you focus on the appropriate design decisions related to the Azure and Citrix Cloud control planes.

Azure Specific Considerations

Azure accounts are used for consolidated billing, but cannot contain Azure resources directly. Azure accounts contain one or more subscriptions. Subscriptions serve as security boundaries and they contain the actual Azure resources, such as virtual machines.

A subscription is an agreement with Microsoft to use one or more Microsoft cloud platforms or services. Charges accrue based on either a per-user license fee or on cloud-based resource consumption. Subscriptions can be used to further subdivide the costs or administrative access as required.

Management groups are used within Azure to efficiently manage access, policies, governance, and compliance across subscriptions. They are invaluable for operating multi-subscription tenants in Azure at scale. Each subscription automatically inherits the conditions, policies, and access of its parent management group.

Here are the questions that you need to answer about Azure infrastructure

How many Azure tenants do I need?

• Use a single Azure tenant for the Citrix resources and the users and devices that access those resources

• Use multiple tenants where multiple Azure Active Directories are required. Development/Test having a separate authentication directory or an enterprise that has multiple on-premises AD directory services are two examples.

• The Azure account owner must be associated to the same tenant where the subscriptions for the account are provisioned

• Azure account owners are automatically subscription owners for all the subscriptions in the account

What Microsoft license models should I use?

• Apply the Hybrid Use Benefit (HUB) of your current EA license if it includes Windows Server Software Assurance. HUB significantly reduces compute costs in the cloud. This licensing model can save you up to 40% of the hourly cost because you can use the base VM pricing for Windows Server or SQL Server instances in Azure.

• If using the Microsoft Office suite, use per user licenses that include the Windows 10 Virtual Desktop licenses such as the E3/E5 subscriptions

  • Microsoft 365 E3/E5: Includes Azure Virtual Desktop licenses and Microsoft Office licenses
  • Microsoft 365 Business Premium: Includes Azure Virtual Desktop licenses and Microsoft Office licenses
  • Windows 10 Enterprise E3/E5: Includes Azure Virtual Desktop licenses

How many Azure subscriptions will I need?

• All subscriptions within the same management group must trust the same Azure Active Directory tenant

• A subscription can be associated with only a single account at a time and must have an associated account owner

• Subscriptions cannot share networks, but they can communicate through VNET peering and Azure ExpressRoute

• Subscriptions are boundaries for Azure policies, management, governance and administrative, so plan subscriptions for business units that have separate administrative or billing requirements

• Multiple subscriptions reduce the blast radius and exposure in case credentials are compromised

• Plan to isolate development and test subscriptions from production subscriptions to provide extra performance, security, governance, and compliance

• Some environments such as production and user acceptance testing or preproduction can be shared in a single subscription

• Dedicating subscriptions to Citrix workloads simplify administration and policy management

• Citrix recommends limiting a subscription to 2,500 Virtual Delivery Agents (VDAs)

• Use subscriptions as scale units and scale them out as needed to support the required resources

• Microsoft sets limits on resources within a subscription and those limits must be considered when determining how many subscriptions are necessary to support the Citrix workloads

How many management groups will I need?

• Subscriptions can belong to only one management group at a time

• Management groups are associated with a single parent

• Management groups can be up to 6-levels deep and Microsoft recommends keeping the management group hierarchy as flat as possible

• Management groups are used for policies, not for billing or line-of-business groups. Create management groups based on policy requirements such as instance types, firewall rules, logging, storage, encryption, RBAC model, and so forth

• Limit the number of Azure policy assignments at the management group root, instead of placing them on the individual management groups

• Citrix recommends creating a management group for Citrix workload subscriptions

• Management groups are used for aggregating Azure Policies, so group subscriptions with similar policy requirements together under the same management group

• Use resource tags that can be referenced by Azure policy

For Citrix Cloud to connect and deploy machine catalogs in the Azure cloud, a service principal account is required. That account needs the correct permissions to create, delete, and maintain Citrix resources in each subscription. The service principal account is created through an application registration within the Azure AD tenant. The creation of the service principal account can be created automatically by Citrix or manually by an Azure AD global administrator.

The creation of the service principal object can be accomplished automatically by Citrix if the user running the Citrix Host Connection Wizard has contributor permissions on the subscription. During the host connection setup, the Wizard requests all the required permissions, including contributor permissions on the subscription, and keeps that acceptance for future connections.

Security-sensitive environments do not allow service principals to have contributor permissions at a subscription level. Citrix provides an alternative solution referred to as a Narrow Scope service principal. An Azure AD global administrator needs to manually create an application registration. Then a subscription administrator manually grants the service principal account appropriate permissions. Narrow-scoped service principals do not have contributor permissions on the entire subscription. Their permissions are scoped to just the resource groups, networks, and images that are required to create and manage the Machine Catalogs.

Here are the questions you need to answer regarding the service principal account:

Should I use a subscription-scope service principal account?

• Requires Azure AD global administrator permissions

• Contributor role for the entire subscription is created automatically and Azure will prompt for permissions approval at initial connection

• Use when information security allows a service principal account to be granted contributor permissions on the entire subscription and Citrix administrators have contributor access to the subscription

• Accounts used for authentication during the host connection creation must be at least co-administrators on the subscription and a member of the Azure Active Directory

• Recommended when subscriptions are dedicated to Citrix resources or the environment will contain many resource groups

• Use when a simple management experience is wanted

• Use when Citrix Studio is used to manage the environment more than PowerShell

• Preferred during proof-of-concept deployments

Should I use a narrow-scope service principal?

• The narrow-scope service principal is created manually by an Azure AD global administrator

• Before running the machine catalog Add Machines wizard, the target resource group must be precreated and granted these permissions:

  • Pre-Created Resource Group: Virtual machine contributor, Storage account contributor, and Disk snapshot contributor
  • Virtual Network: Virtual machine contributor
  • Storage Account: Virtual machine contributor

• Recommended when the number of resource groups is manageable either through the Azure console or through automation

• Recommended for higher-security environments where permissions are tightly controlled and fine-grained access control is prevalent

• Recommended when subscriptions cannot be dedicated to Citrix resources and are hosting other services

• Recommended when Azure administrators have different subscription permissions depending on their role

• For larger environments, consider using build scripts or ARM templates to pre-create resource groups and grant the required permissions

Should I use custom roles for the service principal?

• Citrix recommends the use of custom roles for setting permissions for the service principal when more than one subscription will be used

• Microsoft recommends setting the role permissions at the management group level through Azure policy

Citrix Cloud Considerations

Citrix Cloud Considerations

Citrix Cloud, much like Azure, supports multiple tenants and provides one or more Citrix Cloud services for use by the tenant. Each tenant is identified by one or more organizational identifiers (OrgIDs). Companies set up OrgIDs based on how they would like to manage their Citrix assets. OrgIDs are assigned to one of three Citrix control plane regions: United States, European Union, or South Asia Pacific.

Here are the questions that you need to answer about Citrix Cloud

How many Citrix OrgIDs do I need?

• Citrix customers can have one or more OrgIDs based on their organizational structure

• When multiple OrgIDs are in use and are configured in separate regions, use of multiple Citrix tenants can improve user experience.

• Use Citrix OrgIDs to isolate resource usage and for billing simplification

• A single Citrix Cloud OrgID has a limit of 100,000 concurrent users or 3,000 sessions per minute for Citrix DaaS.

• A Citrix Cloud OrgID only supports a single Azure AD tenant for authentication, though multiple Active Directory domains are supported

• Citrix Cloud customers must select a host region for each OrgID

• Citrix recommends using isolated OrgIDs for development and test to keep them separated from production environments

Which Citrix region should I select?

• The Citrix region selected identifies the location of the Citrix control plane. The location of the Citrix VDAs is independent and maps to Azure regions.

• The Citrix Cloud region for an OrgID should be as close as possible to the Azure regions that uses that control plane

• The Citrix Cloud region cannot be changed after it is selected

• Citrix Cloud regions are fully redundant within the geographical region where they are hosted

What Citrix licensing model should I choose?

• For cloud-deployment only customers, you need the DaaS Premium Plus per user license to access the Citrix DaaS.

• For customers with on-premises licenses, the on-premises licenses for Virtual Apps and Desktops, Virtual Apps and Desktops Standard for Azure, and Endpoint Management include cloud services support

• On-premises licenses can be added to your Citrix Cloud OrgID by registering the on-premises Citrix License Server with Citrix Cloud. You can also add the license using the 8-digit short code

What do I need to do to configure Citrix Workspaces from Citrix Cloud?

• Citrix Workspaces support multiple authentication providers including, Active Directory, Azure Active Directory, Citrix Federated Authentication Services, and Okta

• Citrix Workspaces does not support the legacy Program Neighborhood clients (PNAgent). If your environment includes these legacy clients, plan to use on-premises versions of StoreFront servers and Citrix Gateway or upgrade the clients to the Workspace application

• Disabling Workspace Integration prevents users from accessing the service to launch resources, but it does not disable the URL itself

• To support auto-launching of a desktop or application, verify that the Workspace URL is in the local zone or the trusted sizes zone

What is the best way to migrate my on-premises WEM infrastructure to the Azure cloud?

• The WEM Service must be provisioned in Citrix Cloud before the WEM database can be migrated

• Download the WEM migration tool from Citrix Downloads

• Before starting the migration process, configure the database maintenance on the Database Maintenance tab to reduce the database size and shorten the migration process

• The WEM migration toolkit requires .NET Framework version 4.7 or later. If not running 4.7, upgrade first to 4.7 then migrate the database to the WEM service using the toolkit

• A successful migration means that all WEM data in your current database will be lost. Make sure you back up your existing database before starting the migration

• After upgrading the database, on the VDA hosts switch to the service agent mode and point it to the Cloud Connectors then restart the VDA host.

Citrix Infrastructure Considerations

Moving Citrix workloads to the Azure cloud include more than just moving the Citrix server images. Citrix users rely on print services, while applications rely on supporting infrastructure such as databases or web servers. To provide the best experience for users, these supporting infrastructure services and workloads need to be accessible during the transition.

Here are the questions that you need to answer about the supporting infrastructure:

What information do I need to have about my infrastructure servers?

• Basic information: Host name, physical or virtual, appliance or server, the hypervisor and hardware version, server role and purpose

• VM Specs: Number of cores/vCPUs, core speed, Memory

• OS Information: OS Name and Version, End of Support Date, End of Life Date

• Networking: Number of required vNICs, IP addresses, NIC Emulation mode, VLANs Assigned Name / Number, VLANs communications flows, Inbound ports and protocols, Outbound ports and protocols, load-balancer required

• Storage: Root Device Volume (GB), other attached storage and size (GB), virtual disk format, encryption, number of volumes, disk configuration

• Backup: Backup Frequency and Type, Backup Available, Known good backup, Backup Date/Time

• Software: Required Software List, Software Versions, Vendors, File systems accessed, software verified to work in Azure Cloud

• Metrics: Have available metrics over the last 30 days for Max CPU, Steady-state CPU, Max IOPS, Steady-state IOPS, Network Bandwidth (ingress/egress), Max Memory, and Stead-state Memory

• SSL Certificate Requirements

• Migration: System Downtime window, Preferred Application (Infrastructure) Migration Strategy, Recovery Time Objective, Recovery Point Objective, Acceptable downtime during migration, Application Migration Priority, Data Criticality, Assigned Disaster Recovery Tier

• Dependencies: Uses any type of Hardware-based key (like a USB Key), other workloads dependent on this server, does this workload depend on another server or service

How do I verify the applications running on them can be hosted in Azure and determine if the server should be migrated, rebuilt, or sunset?

• Reach out to the application vendor and verify they can be virtualized in Azure. If the vendor does not know, then install and test the application in a development or test subscription

• Determine if applications are necessary to have available in Azure. If not, plan to keep them on-premises until they can be sunset

• Reach out to the application vendors to determine if all the applications in a Citrix workload are supported in Azure. If not, the applications that are not supported may need to be placed in a different Citrix workload that will be kept on premise and eventually sunset

• If the applications are supported in Azure, request information on migrating the application from the vendor

• Use the Azure Migrate: Server Migration Tool to determine if a server can be live-migrated

• If the server is virtual and is supported in Azure, but unable to be live-migrated, consider exporting the OS boot disk and importing it into Azure and creating an image in the gallery

• If the application server is supported but cannot be migrated or the OS disk copied, then plan to rebuild the server in Azure

How will printers and print servers get migrated to Azure?

• Print servers can be migrated to Azure like other infrastructure servers. Any printers that are connected to that print server need to be accessible via a routable protocol like TCP

• Print servers in the cloud will need to be managed just as they were locally

• Consider using Citrix Universal Printing

• Consider looking at centralized SaaS cloud printing services

• Test and verify any printing solutions

What are the best practices for migrating databases to Azure?

• Always keep the database as close as possible to the applications that use the database to avoid unnecessary latency or Azure egress charges

• Azure RDS supports SQL Server, PostgreSQL, and MySQL database engines. Other database engines either need to have their schema, data, and code converted to one of the RDS supported formats or be migrated to IaaS servers running the source database engine

• Always migrate to RDS versions where possible

• Consider using the Database Migration Assistant to assist with any database migrations

What about physical or virtual appliances?

• Identify the best process to patch and secure the appliances in Azure

• Physical appliances cannot be migrated to the cloud, so check with the vendor to verify that virtual versions of the appliances are available that will work in Azure

• For virtual appliances, check with the vendor to verify Azure compatibility, since in some situations you may need to change versions

When is Azure Files a good solution for application data?

• Azure Files supports Windows, macOS, and Linux clients through direct mount of SMB and NFS file shares

• Azure Files supports Active Directory Authentication

• Azure File Sync can be used to replicate SMB file shares between Azure Files and Window on-premises file servers

• Applications can share data through the File REST API interface from any location

• Use Azure Files as persistent storage volumes for stateful containers, such as user layers or profiles

Migrating supporting infrastructure

Use of Azure Migrate: Server Assessment or Movere (Microsoft SaaS offering) is recommended for planning the migration

-  Azure Migrate offers an Azure VMware Solution tool in Preview to assist you in migrating if you are currently using VMware for virtualization in your on-premises data center and plan to use VMware in Azure as well

-  Movere is available through the Microsoft Solution Assessment and Microsoft Cloud Economics Program

Use of Azure VMware Solutions provides the following advantages if you are currently using VMware virtualization in your on-premises data center:

-  Administrators are familiar with the VMware interface and are comfortable with the administration tools and interfaces

-  Migration of VMs can be accomplished through a Microsoft agentless approach that supports replicating up to 500 VMs simultaneously (similar to using vMotion)

Ancillary Infrastructure Considerations

Moving Citrix workloads to the Azure cloud include more than just moving the Citrix server images. Citrix users rely on print services, while applications rely on supporting infrastructure such as databases or web servers. To provide the best experience for users, these supporting infrastructure services and workloads need to be accessible during the transition.

Here are the questions that you need to answer about the supporting infrastructure:

What information do I need to have about my infrastructure servers?

• Basic information: Host name, physical or virtual, appliance or server, the hypervisor and hardware version, server role and purpose

• VM Specs: Number of cores/vCPUs, core speed, Memory

• OS Information: OS Name and Version, End of Support Date, End of Life Date

• Networking: Number of required vNICs, IP addresses, NIC Emulation mode, VLANs Assigned Name / Number, VLANs communications flows, Inbound ports and protocols, Outbound ports and protocols, load-balancer required

• Storage: Root Device Volume (GB), other attached storage and size (GB), virtual disk format, encryption, number of volumes, disk configuration

• Backup: Backup Frequency and Type, Backup Available, Known good backup, Backup Date/Time

• Software: Required Software List, Software Versions, Vendors, File systems accessed, software verified to work in Azure Cloud

• Metrics: Have available metrics over the last 30 days for Max CPU, Steady-state CPU, Max IOPS, Steady-state IOPS, Network Bandwidth (ingress/egress), Max Memory, and Stead-state Memory

• SSL Certificate Requirements

• Migration: System Downtime window, Preferred Application (Infrastructure) Migration Strategy, Recovery Time Objective, Recovery Point Objective, Acceptable downtime during migration, Application Migration Priority, Data Criticality, Assigned Disaster Recovery Tier

• Dependencies: Uses any type of Hardware-based key (like a USB Key), other workloads dependent on this server, does this workload depend on another server or service

How do I verify the applications running on them can be hosted in Azure and determine if the server should be migrated, rebuilt, or sunset?

• Reach out to the application vendor and verify they can be virtualized in Azure. If the vendor does not know, then install and test the application in a development or test subscription

• Determine if applications are necessary to have available in Azure. If not, plan to keep them on-premises until they can be sunset

• Reach out to the application vendors to determine if all the applications in a Citrix workload are supported in Azure. If not, the applications that are not supported may need to be placed in a different Citrix workload that will be kept on premise and eventually sunset

• If the applications are supported in Azure, request information on migrating the application from the vendor

• Use the Azure Migrate: Server Migration Tool to determine if a server can be live-migrated

• If the server is virtual and is supported in Azure, but unable to be live-migrated, consider exporting the OS boot disk and importing it into Azure and creating an image in the gallery

• If the application server is supported but cannot be migrated or the OS disk copied, then plan to rebuild the server in Azure

How will printers and print servers get migrated to Azure?

• Print servers can be migrated to Azure like other infrastructure servers. Any printers that are connected to that print server need to be accessible via a routable protocol like TCP

• Print servers in the cloud will need to be managed just as they were locally

• Consider using Citrix Universal Printing

• Consider looking at centralized SaaS cloud printing services

• Test and verify any printing solutions

What are the best practices for migrating databases to Azure?

• Always keep the database as close as possible to the applications that use the database to avoid unnecessary latency or Azure egress charges

• Azure RDS supports SQL Server, PostgreSQL, and MySQL database engines. Other database engines either need to have their schema, data, and code converted to one of the RDS supported formats or be migrated to IaaS servers running the source database engine

• Always migrate to RDS versions where possible

• Consider using the Database Migration Assistant to assist with any database migrations

What about physical or virtual appliances?

• Identify the best process to patch and secure the appliances in Azure

• Physical appliances cannot be migrated to the cloud, so check with the vendor to verify that virtual versions of the appliances are available that will work in Azure

• For virtual appliances, check with the vendor to verify Azure compatibility, since in some situations you may need to change versions

When is Azure Files a good solution for application data?

• Azure Files supports Windows, macOS, and Linux clients through direct mount of SMB and NFS file shares

• Azure Files supports Active Directory Authentication

• Azure File Sync can be used to replicate SMB file shares between Azure Files and Window on-premises file servers

• Applications can share data through the File REST API interface from any location

• Use Azure Files as persistent storage volumes for stateful containers, such as user layers or profiles

Migrating supporting infrastructure

Use of Azure Migrate: Server Assessment or Movere (Microsoft SaaS offering) is recommended for planning the migration

-  Azure Migrate offers an Azure VMware Solution tool in Preview to assist you in migrating if you are currently using VMware for virtualization in your on-premises data center and plan to use VMware in Azure as well

-  Movere is available through the Microsoft Solution Assessment and Microsoft Cloud Economics Program

Use of Azure VMware Solutions provides the following advantages if you are currently using VMware virtualization in your on-premises data center:

-  Administrators are familiar with the VMware interface and are comfortable with the administration tools and interfaces

-  Migration of VMs can be accomplished through a Microsoft agentless approach that supports replicating up to 500 VMs simultaneously (similar to using vMotion)

Business Continuity/Disaster Recovery Considerations

Every business needs to be operational to generate revenue. The longer a system is down or not functioning, the more revenue that is lost for the business. At some point in the future, if the system is down long enough, the business becomes a going concern and eventually end. For some businesses, the outage can be several months, while other business cannot survive after several days. The key to a good plan is identifying what systems are critical for the business and setting the appropriate mitigation in place.

Here are the questions that you need to answer about Business Continuity and Disaster Recovery:

What Citrix components use Availability Sets or Availability Zones?

• Not all regions support Availability Zones, if they are available in your region, prefer Availability Zones over Availability Sets

• Citrix recommends using the Availability Set or Availability Zones for the following Citrix components:

  • Cloud connector
  • Citrix Gateway
  • StoreFront
  • VDA Hosts

What information should I back up?

• Backup anything that is important. Set the backup vault frequency to match the RTO and RPO requirements for the server and set the retention based on corporate data retention policies

• Backups are not automatically copied to the paired region. This task must be done manually or scripted

• Persistent desktops nmust be backed up

• Golden image machines should be backed up

What regions should I place my resources in?

• Azure regions should be selected primarily based on data sovereignty, governance, and compliance

• Place Citrix resources in Azure regions that are close to your users

• Azure regions are paired for disaster recovery/business continuity. Take note of the regional pairs and select the pairs that work best for your users and business continuity plan

• Depending on where your users and data centers are located, one regional pair may be a better choice than another regional pair

• When planning for disaster recovery, use the paired region for backups and failover configurations to increase the probability of a successful recovery in the unlikely event of a regional failure

What information should I store in the paired region?

• Identify key servers (recovery tier 0) that absolutely must be up 100% of the time, such as a domain controller, since they are good candidates for replication

• Any servers that are using Azure Site Recovery (ASR) should be pointing to the paired sister region for

• Identify key images/backups for servers that have to be brought online quickly and do not have an automated build available

• Schedule PowerShell jobs to replicate the key images, backups, and snapshots across to the other region

• Regularly export the ARM templates for your configurations and store them in the paired region. This task can be scripted.

What is appropriate to use Azure Site Recovery Replication?

• Any core infrastructure that is considered Recovery Tier 0

• Keep at least one domain controller at the recovery site so the domain passwords are current. That allows the users doing the restore to authenticate to the ASR environment.

• Keep at least one Citrix Gateways for remote access into the system.

Workload Level Design Considerations

The most dynamic part of a Citrix virtualization system is the VDA. Remember that VDAs are where the actual work is happening. The apps and desktops you provide users on a Citrix virtualization system run from VM instances on Azure. You want to make sure you get this layer right! Do your homework up front. Set the expectation with users that the system will change over time and build simple and effective processes to handle change. With the power and flexibility of Citrix virtualization tech, managing change doesn’t have to be a major burden.

In this section, we’ve attempted to logically break the topic up such that we can dive deep without losing context. We do our best to provide the details you need in each section and call out leading practices and recommendations along the way.

Migration Considerations

Citrix servers can be hosted on various platforms, including Hyper-V, VMware vSphere, and physical servers. The most time-consuming part of moving Citrix to the cloud is the planning process. The planning process requires both discovery and analysis to determine the best path for the migration. The end result of the analysis is a document that provides a migration plan. Here are the questions that you need to answer about Citrix Workload Migrations:

How do I migrate my Citrix VDA hosts to Azure?

• Azure Migrate integrates with Azure services and supports other third-party tools. Azure Migrate helps you complete the migration process from an on-premises data center to Azure.

Are there any utilities available to help migrate applications and application data to Azure?

• Azure Migrate includes tools to help discover and assess the current environment and plan the migration of applications, servers, and data to Azure.

How do I use Azure for capacity planning?

• Use the Azure Migrate Server Assessment for planning the migration when possible.

• Movere (Microsoft SaaS offering) is another possible Microsoft tool to assist with migration assessment. Movere is only available through the Microsoft Solution Assessment and Microsoft Cloud Economics Program.

How do I move my application data to Azure?

• Azure Migrate offers an Azure VMware Solution tool in Preview to help with migrating all of your servers, including the Citrix workloads.

• Use Azure VMware Solution (AVS) integration to provision your Citrix VDA workload just like other vSphere workloads in your on-premises data center

Azure VMware Solutions

Using Azure VMware Solution (AVS) provides the following advantages if you are currently use VMware virtualization in your on-premises data center:

• Cloud Migration Support: Easily migrate desktops and applications between VMware deployments, including replicating up to 500 VMs simultaneously.

• Familiar Administration: Administrators are familiar with the VMware interface and are comfortable with the administration tools and interfaces.

• Data center extension: Use the data center extension to provide burstable capacity and support remote locations. The data center extension helps during periods of high demand, disaster recovery or business continuity by taking advantage of the rapid elasticity provided within AVS.

Scalability Considerations

Which Azure instance series are best for hosting my Citrix Virtual Apps and Desktops workload?

• Select the D/DS series instances when your applications consume a fair amount of memory. The D/DS series have a higher memory-to-CPU ratio.

• Select the F/FS series instances when you need excellent CPU response times and do not require a significant amount of memory. The F/FS series have faster processors but lower memory-to-CPU ratios than the D/DS series instance family.

• The most cost-effective instance type is the F/FS series with 8 or 16 vCPUs followed by the D/DS series with 4 vCPUs.

• Choose instance types with fewer vCPUs when you want to:

  • affect fewer users during maintenance windows or unexpected server performance issues
  • scale down quicker to take advantage of cost savings with Autoscale

• Choose instance types with more vCPUs when you want to:

  • Reduce the API calls to Azure infrastructure for operations
  • Manage fewer machines

Performance Coniderations

Should I enable Machine Creation Services I/O (MCSIO) cache?

• If Service Level Agreements (SLAs) are not required in your environment, use the less expensive standard disk for the MCSIO cache

• Consider enabling MCSIO cache when user experience is a high priority. The fast response times of the memory cache make a noticeable difference for the users.

• Using a memory cache size of 2 GiB provides the best improvement without negatively impacting user density. Always account for the extra memory when choosing an instance size for the workload

• Do not enable MCSIO cache on memory-constrained machines, such as the F/FS series where the memory to CPU ratios are low

Cost Optimization Considerations

How can I use Citrix Autoscale for both on-premises and cloud workloads?

• Use cloud workloads for burst capacity or for business continuity by tagging cloud workloads in the delivery group. Set Autoscale to power manage these tagged workloads. Use the selective power management feature to set the zone preference and failover to prefer on-premises workloads.

• Use the dynamic provisioning feature of Autoscale with low and high watermark machine counts. This approach reduces costs and still support demand under high loads. This feature is especially useful for instance types with high monthly charges.

• If usage and capacity patterns can be predicted on a consistent basis, use the schedule-based scaling feature to manage available capacity. Configure the settings to start within a 30-minute window. When paired with the capacity buffer, the scheduler prevents users from waiting for capacity to come online.

• When enabling Autoscale, prefer smaller instances over larger instances. Smaller instances drain faster and go offline sooner.

Should I use Windows 10 Multisession or Windows Server OS?

• Windows 10 Multisession has about 10% lower densities than the Windows Server operating systems

• Windows 10 Multisession allows users access to the Windows Store, which is not available on the server operating systems

• Azure Virtual Desktop (AVD) entitlement grants you the base VM price (Linux Pricing), saving a considerable amount over the normal Windows VM price.

Application Considerations

Microsoft Office and Microsoft 365 are among the most popular workloads delivered by Citrix today. Both Microsoft and Citrix have worked together to develop the best user experience when running Microsoft 365 from a Citrix session in Azure. Their collaboration created applications, processes, and guidance to help you deliver the best of breed solution. You likely have other applications hosted on the Citrix servers that must be analyzed and migrated to Azure. These applications have application data that must be accessible regardless of where the application resides. Here are the questions that you need to answer about Applications and Application Data:

How do I integrate the VDA-hosted applications with Microsoft 365?

• Use Office 365 ProPlus when installing office

• Microsoft 365 requires a plan that supports shared computer activation, which is required for any multi-user session hosts

• After installing Office on a golden image, do not open any Office applications. If you open one Office application, you must reset the image to remove the temporary key which prevents user-level activation. To reset the image, uninstall Office, reboot and then reinstall Office.

• For earlier versions of Office that use KMS licensing, such as Office 2010 and 2013, you must verify your KMS server is reachable from the Azure cloud. You can make your KMS server accesslble to your Citrix workloads by one of these methods:

  • Migrate your KMS server to the Azure cloud
  • Connect your on-premises data center to Azure using either an ExpressRoute or a Site-2-Site (S2S) Virtual Private Network (VPN)

• When using FSLogix and Office 365 containers, follow these steps to integrate it with Windows Search, rebooting between each step:

  • Configure Automatic Startup (not delayed) for the Windows Search service. This configuration should be completed before installing Office so Office sets the required hooks.
  • Install Microsoft Office.
  • Install the FSLogix agent.
  • If you do not need Windows Search, you can disable the service. Before disabling the service, go ahead and complete the install steps to save on compute resources, then disable the service. With this approach, if later it is needed, you can enable the service easily.

Should I use FSLogix with Citrix workloads?

• Use Microsoft GPOs to manage all Microsoft 365 Office settings

• Microsoft FSLogix is the recommended approach to handle Microsoft365 integration. It handles the Outlook Search, Outlook PSTs and Office activation seamlessly.

• Microsoft recommends using SSO (ADFS) with Microsoft365 Apps 1704 and above:

  • When ADFS is available, enable the "Automatically activate Office with federated organization credentials” GPO and configure the automatic logon in GPO Security Logon
  • if ADFS is not available, use FSLogix or Citrix Profile Manager to synchronize the following registry key %localappdata%\Microsoft\Office\16.0\Licensing to roam the Microsoft 365 token with the user

How should I configure Outlook? (Cached mode or online mode)?

• Use Cached Exchange Mode when the following conditions are true:

  • A profile management solution such as FSLogix or Citrix Profile Manager is available to manage the OST file and the search index
  • Users require a more responsive email system
  • Connections between the Outlook client and the mail server have high latency or are frequently disrupted

• Use Online Mode when the following conditions are true:

  • Low latency network connection is available

• Use Active Directory Group Policy to configure Outlook Exchange mode, recommended settings include the following:

  • File > Cached Exchange Mode
  • Sync Settings
  • Disable Fast Access
  • Use Cached Exchange mode
  • Cache file

What settings should I use for Microsoft 365 when using Citrix Profile Management?

• When using Citrix Profile Management use these items to provide a robust user experience and support the OST/PST storage locations and the Search Index locations

  • Use the latest version of Citrix Profile Manager. The latest version has features such as Native Outlook Search and Large File Handling which provide optimizations for Outlook.
  • Enable Large File Handling to allow storing OST/PST files on Azure Files.
  • Include these folders and registry in the Citrix Profile Management configuration:
    • %localappdata%\Microsoft\Office\16.0\Licensing
    • %localappdata%\Microsoft\Credential
    • AppData\Local\Microsoft\Credentials
    • AppData\Local\Microsoft\Windows\WebCache
    • AppData\LocalLow\Microsoft\CryptnetUrlCache
    • AppData\Local\Microsoft\Outlook
    • AppData\Local\Microsoft\Vault
    • AppData\Local\Microsoft\Office
    • AppData\Local\Microsoft\Office\*.qat
    • AppData\Local\Microsoft\Office\*.officeUI
    • AppData\Local\Microsoft\Windows\UsrClass.*
    • HKCU\Software\Microsoft\Office\16.0\Common\Identity\DisableADALatopWAMOverride

Where should I store my application data?

• Use Azure Migrate or Movere to assess and plan the application migration

• Check with the application vendors to determine if the software is supported in Azure. If planning to use PVS for streaming, also verify that the vendor supports Gen 2 VMs.

Where should my data be located relative to my application?

• Data leaving Azure incurs an egress charge. Try to keep the applications and their data as close as possible to one another. Although ideal, this configuration is not always possible. When you cannot keep your data close to the application, focus on minimizing the latency between them.

What costs should I consider when determining my data location?

• When working in a hybrid cloud environment that prevents both the application and its data from moving to Azure together, move the application first then move the application data. With this approach, the data egress charges are reduced.

How do I integrate Citrix Workspace with Microsoft 365 and Microsoft Teams?

• For multi-session hosts, install Microsoft Teams after the VDA is installed on your golden image and install it under c:\program files using the ALLUSER=1 flag

• Updates to the Microsoft Teams agent require an uninstall of the previous version before installing the new version

• Set the Citrix Microsoft Teams redirection policy to allowed

• Microsoft Teams relies on Azure Transport Relays the following ports and IP address ranges must be accessible

  • UDP 3478-3481
  • TCP 443
  • 137.106.64.0/18
  • 52.112.0.0/14
  • 52.120.0.0/14

• Use Citrix Director’s Activity Manager to monitor Microsoft Teams applications such as WebSocketAgent.exe, WebSocketService.exe, and CtxSvcHost.exe.

Image Management Considerations

The primary image management solution used in Azure is Machine Creation Services (MCS) and until recently was the most common option for Citrix image management. Citrix has been focused on improving the image management within Citrix Cloud. These improvements help ease our customer's migration to the Azure cloud and spin up any workload in a matter of minutes.

One of the new Citrix services includes the Image Portability Service (IPS). This service provides a way to port images between your on-premises data center and your Azure environment. The service uses a temporary Virtual Machine (VM) to host the Compositing Engine (CE). The CE has two modes: prepare or export. The prepare mode converts virtual disk formats used for Citrix Workloads and updates the portable properties. The export mode copies the prepared disk to the target cloud. A connector appliance controls the individual jobs, spawns the CE VM, and secures communications between Azure and your on-premises environment.

Azure supports a version of Citrix Provisioning Services (PVS) which allows you to stream a virtual disk to multiple VMs simultaneously. With PVS, you can create 2500 identical VMs within a single subscription. Most of the administration and function is the same as for an on-premises PVS environment, except for a few changes. Azure does not allow the use of the Pre-boot Execution Environment (PXE), so changes were required for the streaming and boot process. This version of PVS includes a UEFI-based Boot Disk Manager (BDM) to create UEFI boot disks. These boot disk require Azure Gen 2 VMs and cannot support 32-bit operating systems. The Citrix broker is now responsible for all power management of the PVS VMs, though the PVS console can still power off the targets. The Citrix Virtual Apps and Desktops Setup Wizard handles all the provisioning steps.

How can I use the image portability service to move my Citrix workloads?

• Deploy golden images on-premises, using either Machine Creation Services or PVS.

• The entire process for using IPS is completed through PowerShell commands from a remote workstation.

• The latest version of PowerShellGet should be installed on the remote workstation before beginning the process.

• Citrix connector appliances must be installed at each resource location where IPS is used.

• At the on-premises location, a Windows SMB File share is required for temporary data storage during export jobs. The file share should have enough free space to hold two copies of the disk to be exported.

• Automated publishing is only available with PVS on Azure deployments. Manual publishing to Azure is available for both PVS images and MCS images.

• The following machine catalog configurations have been tested with the IPS

  • Windows Server 2016, Windows Server 2019, or Windows 10 2004 or later
  • Source images provisioned by Citrix Provisioning 1912 or later
  • Citrix Virtual Apps and Desktops VDA 1912 or later

What are the limitations and requirements for deploying PVS in Azure?

• The Azure subscription must have the ReserveMacOnCreateNic feature enabled.

• At most 2500 VMs can be streamed in a single subscription.

• All provisioned VMs must be created in the same region as the hosting unit. VMs are automatically spread across all availability zones in the target region.

• Cross-region provisioning is not supported.

• Requires UEFI boot of Generation 2 Azure VMs. Generation 1 (BIOS-based) VMs are not supported. Boot images that are created using the PXE or ISO format are not supported.

• Supports only 64-bit versions of Windows 10 and Windows Server 2019 for streaming.

• Use the Citrix Image Portability Service to import an existing image.

• Active Directory support is required for machine naming using one of these methods:

  • Azure Active Directory Domain Services (AADDS) added to your Azure Active Directory (AAD) tenant
  • ExpressRoute connection to your on-premises Active Directory environment
  • Active Directory domain controllers installed in Azure and synchronized with your on-premises AD environment and the AAD tenant

• When using Azure Files Services for vDisk storage, premium storage or Azure NetApp Services is required and must be in the same region as the PVS server.

• SQL Server or SQL Server Express VM is required. Currently, using any authentication method except Windows Integrated Authentication or using an Azure SQL database is not supported.

• The Golden VM must have same disk and vGPU configurations.

• Set up a virtual network for streaming to targets and peer that network with the network used for the VM communication to Active Directory. Use the AD Domain Controller IP addresses for DNS servers.

• The PVS Server VM must have at least one vNIC on each virtual network where targets reside. The PVS server should also have at least 2 vCPUs and 8 GiB of memory.

What are the best practices for Image Management in Azure?

• Always make a copy or snapshot of your golden image and use the copy or snapshot for machine catalog images. This practice allows for easy image updates and provides protection against image corruption.

• Point existing image-based machines (PVS and MCS) to the cloud connectors by modifying the ListOfDDCs registry key on the golden images. The registry key can be found at HKLM\Software\Citrix\VirtualDeliveryAgent. After modifying the registry key, take a snapshot of the image. Update the machine catalog with the new snapshot when ready for the images to register with Citrix Cloud.

• Use the Citrix Group Policy (Computer Configuration > Citrix Policies > Controllers) to point the other Citrix workload servers to the FQDN of the cloud connector and set the Enable auto update of Controllers to “allowed”.

• Use Managed disks for the golden images, unless you are using App Layering.

• Be sure to keep copies of golden images is different regions.

• Automate the build of the Golden Image

  • PowerShell SDKs can be used for full automation of the Golden Image build

    • PowerShell v5.x
    • Azure PowerShell Module
    • Citrix Cloud Remote PowerShell SDK

  • Use Azure PowerShell to do the following:

    • Create a new Azure VM
    • Configure the Windows Firewall
    • Remove the Azure Public IP address (if present)
    • Automate domain join
    • Automate software installs such as the VDA
    • Seal the image
      • Reset Log files
      • Reset Office Licensing
      • Clear GPO cache
      • Remove user profiles used for build process
    • Copy golden image to business continuity region

  • Install any other software manually that cannot be automated.

  • Use the Citrix Cloud Remote PowerShell SDK to do the following:

    • Update the Machine Catalog

  • Do not store passwords unencrypted in any scripts or change any stored passwords (such as the local admin password) immediately after building.

Should I do anything different from on-premises when managing MCS images in Azure?

• Use the Azure shared image gallery for storing MCS images to reduce the time required to create and hydrate the OS disks and improves application performance.

How do I manage images across geographic regions?

• Use Azure shared image gallery to provide multiple replicas in different regions.

How do I enable MCSIO in Azure?

• MCSIO can be enabled on MCS catalogs in Azure by using PowerShell to create a new Provisioning Scheme and Catalog
  • Use VDA version 1912 or later for best results since not all earlier versions are supported.
  • Do not forget to pre-create a Resource group in Azure for the MCS catalog.
  • Install Citrix Remote PowerShell SDK to access Citrix Cloud configurations.
  • In Azure, the write-back cache is stored on non-persistent media.

Image Layering Considerations

Use App Layering inside Azure for the same use cases that you would use it in your on-premises data center:

• Manage a significant number of Machine Creation Service (MCS) images
• Provide persistent desktops for users using non-persistent VDA hosts with MCS
• Limit the rebooting of the Virtual Delivery Agent (VDA) hosts

NOTE: App Layering requires Gen 1 VMs and Provisioning Services (PVS) requires Gen 2 VMs so the two services are currently incompatible.

App Layering works almost the same way in Azure as on-premises. Here are some of the questions that you may have about App Layering.

What are the differences between using Citrix App Layering on-premises and using it in the Azure cloud?

Each application along with its related software should be installed in its own layer. These guidelines help you plan the layers.

Base Operating System (OS) Layer

Start with a new OS image and use only a single OS layer and choose Resource Manager from the listed deployment models

• Do not select the “Use managed disks” option. In Azure, Layering requires a storage account.

• Verify that the OS is set to use DHCP for IP addressing.

• If using an Azure VM that has the Page file on the D: drive, move it back to the C: drive before capturing the OS layer. With this change, the image will still deploy correctly in production. The requirement is just temporary during the single disk OS image capture process.

• Do not use or include an UNATTEND.TXT file, since the Layering process removes it automatically.

• Use ngen.exe to pre-compile .NET executables.

• Set Built-in Administrator to “Password Never Expires”.

• For server OS builds, set the PowerShell Execution policy to unrestricted and enable PSRemoting.

• Install the App Layering Services on the OS Layer.

• Install App Layer OS Machine Tools and follow the instructions for KMS scripts if using KMS Licensing. The Citrix App Layering OS Machine Tools include special scripts to automatically handle the complexities of Microsoft Licensing and prevent any misconfigurations.

Platform Layer

The platform layer consists primarily of software not included in the base OS layer that connects to other infrastructure.

• Join the Active Directory domain and verify that the user name is in the format DOMAIN\Username. Ignore the default request for just the user name.

• Install the provisioning software, Citrix VDA, and Citrix Workspace Environment Manager into this layer.

Publishing the Image

Create an image template and use that to publish an image for MCS.

• The new image appears as a VHD in the Storage account’s container citrix-al-images.

• If your App Layering version is earlier than 4.15, attach the image to an Azure VM and boot the VM to let Sysprep complete its tasks.

• If your App Layering version is 4.15 or later, use the Azure Connector for MCS, since it does not Sysprep the image.

• Choose the disk file in the storage account as the Golden image when creating or updating the machine catalogs.

What permissions are required for using App Layering in Azure?

• Use Accelerated networking for your Enterprise Layer Manager (ELM) virtual appliance to improve performance.

• The ELM appliance uses the Azure Service Principal to access Azure resources. Both the service principal and the user installing ELM must have at least contributor permissions on the resource groups used by App Layering.

• Use Azure premium storage for packaging machines and image layers to reduce packaging time.

How do I support Microsoft 365 and KMS licensing with App Layering in the Azure cloud?

• Place Microsoft Office in its own layer with all the Office Add-ons
  • Install Office into the default location.
  • Do not open any Office applications during the installation and packaging process.
  • In the Optimize script, be sure to enable “Activate MS Office via KMS” and select ONLY the versions of your installed Office products. The script will only run successfully if Microsoft Office is installed in the default location.
  • Run the Office2013Windows81_PREP.cmd for all versions of Microsoft Office starting with Office 2013, this includes Microsoft 365.
• Use larger layer sizes if users can store large files in the application layer. Increasing layer sizes later to support large PST and OST files is difficult.

Where are the Elastic Layers stored in the cloud?

• Elastic layers are mounted dynamically when a user logs on and provide access for applications. Elastic layers are normally stored on network file shares.

• Elastic layers must be available 100% of the time. If the elastic layer is unavailable, even for a short time, all connection layers fail and the VDA host must be rebooted to fix the issue. Options for storing elastic layer files on always available storage include:

  • Scale Out File Server for Application Data
  • Azure Files (premium storage recommended)

User Level Design Considerations

In this section you will find a list of items to help you focus on the appropriate design decisions specific to users and their data.

Authentication Considerations

Active Directory Domain Services (AD DS): This service is the traditional on-premises Active Directory infrastructure that supports GPOs, Kerberos authentication, and domain joins. A new AD DS can be created and hosted on virtual machines in the cloud. Alternatively, an existing AD DS infrastructure can become a hybrid model with some controllers in Azure and some on-premises. In both deployment scenarios, the AD DS domain can be synchronized to Azure Active Directory (AAD) using Azure AD Connect.

Azure Active Directory (Azure AD): This service is Azure’s authentication cloud-based identity and mobile device management service that provides user authentication. Azure AD does not support device authentication, domain joins or group policy objects (GPOs). However, Azure AD can be paired with Azure Active Director Domain Services (AAD DS) to provide the minimum level of support needed for Citrix in Azure.

Azure Active Directory Domain Services (Azure AD DS): This service is a managed domain service hosted in the cloud. This service supports GPOs, Kerberos authentication, and domain joins. The difference between Azure AD DS and AD DS is that the AD domain controllers are managed by Microsoft rather than you. Azure AD DS integrates directly with Azure AD and is a great option for cloud-based Citrix deployments.

Here are the questions you need to answer regarding Active Directory infrastructure:

Should I continue to use only my on-premises Active Directory infrastructure?

• Easy to deploy by just installing Cloud Connectors and joining them to the on-premises domain

• Citrix Cloud can be configured to use only the on-premises AD Domain

• Using or synchronizing to Azure AD is not required, leaving Azure AD to be solely used as identity management for Azure administration

• To prevent latency introduced by domain authentication, Citrix recommends placing domain controllers near the Citrix Virtual Delivery Agent (VDA) hosts and the Cloud Connectors

• Approval from information security (infosec) may be required to place domain controllers in Azure

• Not recommended for deployments that use Microsoft 365 and that have users logging into Azure AD for that service

Should I extend on-premises Active Directory using Hybrid Mode with Azure AD Connect?

• Microsoft recommends this design when you have an existing on-premises AD infrastructure and need either of these features:

  • schema extensions
  • account-based Kerberos constrained delegation

• At least one Active Directory domain controller should always be available for the Cloud Connectors and VDAs. This design prevents any authentication bottlenecks or latency during the group policy processing, domain joins, and authentication events

• Citrix recommends this model when you have Citrix workloads that are still on-premises

• Citrix recommends this model if Citrix Cloud services such as Endpoint Management will be used

• Place at least two domain controllers in Azure and use Azure AD Connect to synchronize AAD with AD DS over ExpressRoute or VPN

• Using Windows 10 under a Hybrid Use Benefit license requires computer accounts and user accounts be in the same Azure Active Directory

• Approval from information security (infosec) may be required to place domain controllers in Azure

• If using smart cards, Kerberos must be enabled on a domain controller

Should I establish a new Azure Active Directory(AAD)?

• Microsoft recommends this model when you are using a cloud-only deployment or when you do not have an existing on-premises AD infrastructure

• Citrix recommends this design when all your Citrix workloads are in Azure and using one of these services:

  • Citrix DaaS

• Plan to use Azure AD Connect to synchronize Azure AD with Azure AD DS and enable password hash synchronization

• If using smart cards, Kerberos must be enabled for Azure AD DS

• Using Windows 10 under a Hybrid Use Benefit license requires computer accounts and user accounts be in the same Azure Active Directory

• Azure AD DS does not support schema extensions, one-way trusts or account-based constrained delegation for Kerberos

• Azure AD DS does not support Domain or Enterprise Admin privileges

Should I use Azure AD as the Citrix Cloud Identity provider?

• Citrix Cloud supports both Azure AD and AD DS for authentication

• When using Azure AD as the Citrix Cloud Identity provider you maintain control of password policies and can easily disable accounts

• Using Azure AD provides multifactor authentication (MFA) to increase the security posture for Citrix Cloud

• When Azure AD is branded, Citrix Cloud has a branded sign-in page

• Azure AD extends Citrix Cloud to support federated identity options such as Okta, Ping or ADFS

• Requires Global Admin role for consent to allow Citrix Cloud to connect to Azure AD. If access to this role is not available, consider using other identity providers such as on-premises AD or the default Citrix identity provider.

Should I enable Multifactor Authentication (MFA) on the Azure Active Directory Accounts?

• Multifactor authentication is always recommended for any resource that is accessed over the internet. Multifactor authentication decreases the available attack vectors and increases the security posture of the system.

• Azure AD makes enabling MFA simple and integrates easily with the Citrix Cloud identity provider

• If not using Azure AD for MFA, consider other identity providers such as Okta to provide this additional security.

Migration and Management Considerations

Most end-users are connecting from outside of the Azure cloud when accessing cloud resources using their own devices or devices from your enterprise. The user and device characteristics influence greatly the design and architecture of the cloud environment and the recommended migration paths. How you manage your environment today imposes certain requirements if that management system is moving into Azure.

Users are managed primarily through the directory services. If you are using a cloud-only deployment of users, you would start by creating an Azure AD tenant. Then you link that tenant to Azure AD and create the users and groups directly within Azure AD. If you have an existing deployment, you can use Azure AD Connect to synchronize your AD users and groups to Azure AD automatically.

Usually, installing and configuring Azure AD Connect to synchronize with Azure AD is a bit of a time-consuming process. The time spent setting up Azure AD Connect is worth it because users are able to access resources easier. Implementing Azure AD is recommended as the best long-term cloud strategy for authentication.

Here are the questions you need to answer regarding User Management:

Which identity provider do I need for Citrix Cloud?

• Citrix cloud supports the folloing identity providers natively:

  • On-premises Active Directory
  • Azure Active Directory
  • Citrix identity provider
  • Over 20 third-party federated providers such as Okta or Ping

• Select the identity provider that makes the most sense for your Citrix Cloud deployment. Do not forget to consider all existing applications and their requirements to integrate with cloud services

• Determine if using a federation established with an on-premises deployment is a requirement for user identities. Examples of potential federations include Kerberos-based SSO, SAML, or MFA with smart cards or hardware tokens like RSA SecurID.

How do I move my existing on-premises AD users and groups to Azure AD?

• Review available Microsoft documentation to determine what design works best for your business requirements

• Verify that the licensing model for your Azure AD supports the features and number of users for your environment.

• Microsoft recommends installing a domain controller in Azure to synchronize with your on-premises domain controllers over Azure ExpressRoute or VPN. Having a domain controller in Azure improves the Azure AD Connect synchronization performance.

• Install and configure Azure AD Connect and allow it to synchronize your users and group memberships over to Azure AD

• A single Azure AD Connect server is limited to a single forest for synchronization. Using Azure AD Connect is more complicated when multiple AD domains within the same forest are involved in the synchronization process.

• Depending on the hybrid identity required, different options may be enabled:

  • Password hash synchronization (PHS)
  • Pass-through authentication (PTA)
  • Multifactor Authentication (cloud-based only)
  • Single sign-on with Federated Services (smart cards, password expiry notifications, on-premises MFA)

• If not all the users must be synchronized to Azure AD, Azure AD Connect supports filtering at the domain, organizational unit, attribute, or group level.

• Filter the AD scope to only include objects that need to be in Azure AD

• Large directories take a considerable time to import. Currently, Azure AD throttles write operations to 84,000 per hour so you need to allow adequate time for a full sync to occur.

• Monitor and configure alerts using the Azure AD Connect Health portal in Azure.

Profiles and User Data Considerations

Profile management solutions are designed to make a user’s local profile portable so that it can be accessed from any session or device. Both Citrix User Profile Management (UPM) and Microsoft FSLogix improve on the traditional roaming profile model used in data centers. Both solutions improve the response time for users and store the user profile using Azure Files. The benefits of Citrix Profile Management are outlined below.

Citrix User Profile Management

• Integrates with the following products:
  • Citrix Virtual Apps and Desktops (Citrix DaaS)
  • Citrix Workspace Environment Management (WEM) service
  • Azure Files
• Virtualizes user profiles so the user settings can be applied to the user desktop or application
• Streams profile data so that it is not downloaded until needed
• Offers large file handling which allows large files to be redirected individually providing a native (local) file experience
• Supports profile exclusions to reduce bloat
• Supports multiple concurrent file accesses for multi-session users
• Supports profile containerization
• Improves logon speed
• Implements containerization through redirection of users profiles to a virtual hard disk
• Supports profile containers and Microsoft Office containers
• Maintains user data for non-persistent environments, such as Citrix session or Azure Virtual Desktop
• Reduces logon times by mounting a VHD instead of copying user profile data across the network
• Supports profile exclusions to reduce bloat
• Provides a native (local) profile experience for users
• Integrates with OneDrive and Azure Files

Profile Design Considerations

Some applications are not designed with roaming users in mind and rely on local file caches and indexes that do not roam between sessions. Microsoft Outlook is one of the more popular applications with this behavior. Both Citrix User Profile Manager and Microsoft FSLogix can provide an improved user experience with these types of applications. Other design considerations include:

• Users accessing their data from multiple sessions simultaneously require solutions that support that level of file access.

• Keep user data as close as possible to the user’s session. When users are accessing from both on-premises and cloud-based sessions, choose the cloud when possible.

• With both profile management solutions, permissions for profile stores must be configured manually. Support for multi-session simultaneous access requires extra configuration.

  • Always combine profile management with folder redirection to reduce the amount of data copied locally
  • Always configure profile folder exclusions to reduce bloat, since they are not configured by default
  • Always enable large file handing for Citrix User Profile Management so that large files, such as PSTs or OSTs, are not copied down

• Antivirus exclusions are required for both FSLogix and Citrix User Profile Management profile solutions because they implement system-level drivers for redirection.

• When using Microsoft FSLogix (or Citrix Profile Containers), you must exclude VHD(X) containers from AV scanning when hosted internally on traditional file shares.

• Azure Files sync can replicate containers quickly and easily for staged deployments.

User Data Challenges

One of the biggest challenges with migrating to Azure includes how to manage user profiles and access to personal and department data. Users require their data to perform their job and they need to access it from any device they are using. This section provides guidance for the challenges associated with user data and considerations that influence the design.

The goals for user data are:

• provide access to the data securely
• provide access to it always from any location
• provide access with the lowest latency possible

Meeting these goals is a challenge with a hybrid environment where some Citrix workloads are in the cloud and some remain on-premises. The user’s data cannot be in both places at once without creating data collision opportunities. Selecting a single location introduces security, latency, or access concerns. This dilemma is true for both the user data and the shared department data.

Windows Profiles

Windows still relies on the concept of profiles to store user data. The loading of those profiles can significantly impact a user’s logon experience, especially when the user’s desktop contains a large amount of data. The logon experience is made worse when the user's session has a significant amount of latency between the profile store and the session host. Several technologies, such as Citrix Profile Manager and Microsoft FSLogix, have been developed to help remove these pain points. The information below helps you select which technology is best for your users.

When should I use the traditional file server technologies for hosting user data?

The traditional file server technologies are file sharing solutions that are used in data centers today. Often these technologies use Distributed Files System Replication (DFS-R) or Distributed File System - Namespaces (DFS-N) to make file shares highly available across multiple locations. Accessing these file shares from an on-premises location typically introduces high latency because of routing and protocol latency. The different file server technologies along with their benefits and drawbacks are provided below.

• Standalone File Servers: Windows Server configured as file servers

  • Requires management and maintenance
  • Has potential cost advantages when hosted in the cloud compared to other server-based technologies
  • Compatible with familiar backup/restore products
  • The standalone server is a single point of failure since it has downtime during updates that force the server to reboot

• Storage Replicas: Windows Server technology that enables synchronous replication of volumes between servers or clusters

  • Requires management and maintenance
  • Supports block-level replication (synchronous or asynchronous)
  • Supports the SMB 3.0 protocol which includes security enhancements such as encryption
  • Has only minimal downtime during manual failover between replicas

• Storage Spaces: Windows Server technology that allows drive pooling in a RAID-type configuration and can be clustered across multiple server nodes for high-availability

  • Requires management and maintenance
  • Supports SMB 3.1 which includes transparent failover mechanisms
  • Has a multi-node topology that can scale up/out as necessary,
  • Has a transparent failover,
  • Uses 3 times more disk space than a traditional file server
  • Not always supported by third-party backup/restore products

• Traditional file servers work best in a data center where the Citrix workloads have direct access to the file share.

• Traditional file servers support the installation of governance and security software, such as:

  • Data loss prevention (DLP)
  • Antivirus (AV)
  • Backup
  • Encryption software
  • Host-based Intrusion Prevention System (HIPS)
  • Host-based Security System (HBSS)

• Some traditional file server deployment configurations result in lower overall costs compared to the PaaS shares.

• Use traditional file servers when your organization needs complete control over the data. With a traditional file server, governance and legal ownership is easy to maintain and data classification is easier to implement.

• Traditional file server technologies are used when applications need nearby for compute or when an extensive amount of read/writes are expected on the data.

• Traditional file server technologies represent less durable data storage when compared to cloud-based alternatives such as Azure Files.

• Look at the scalability path for meeting demand, scale out or scale up with current hardware.

When should the PaaS file shares be used?

These cloud-based file services were built specifically as a service instead of an application and optimized to operate over the internet.

• Azure Files: File shares as a service backed by Azure storage

  • Platform as a Service (PaaS)
  • Supports SMB 3.1/NFS 4.1 protocols
  • No server maintenance, Microsoft handles all maintenance
  • Mountable in Azure VM and Windows Server 2012 and later
  • Can be mounted from on-premises hosts
  • Supports different performance tiers: hot, cold, and high performance
  • Supports NTFS permissions and ACLs
  • Costs vary based on storage performance requirements

• Azure NetApp Files: NetApp Filers as a service backed by Azure Files

  • Platform as a Service (PaaS) using NetApp Filers
  • No server maintenance, Microsoft handles all maintenance
  • Mountable in Azure VM and Windows Server 2012 and later
  • Can be mounted from on-premises hosts
  • Includes Extreme Performance compared to Azure Files options
  • Supports NTFS permissions and ACLs
  • Increased cost compared to Azure Files

• PaaS file shares have unlimited highly-durable data storage.

• PaaS file shares have limits on performance, throughput, and protocol support.

• PaaS file shares are more complex to setup with Active Directory NTFS permissions.

• PaaS file shares can be mounted from most operating systems.

• PaaS file shares integrate with other cloud-bases services such as logging and metrics.

• PaaS file shares are best when the user workloads are also in Azure. Using PaaS file shares reduces the egress data charges and saves on monthly charges.

• PaaS file shares work best for sharing data internally across departments when user workloads are in Azure.

• PaaS file shares do not work as well for sharing files externally because permissions are tied to the Azure AD users.

• When using PaaS file shares, verify that users accessing their data shares from on-premises are receiving acceptable response times.

• Select the lowest performance tier that meets the user's expectations.

• Governance and legal requirements are imposed based on the region hosting the data.

• Cloud-based file services do not support the installation of third-party software such as DLP, AVS or encryption software.

• Backups can be easily configured using Azure Backup.

When should I use Azure NetApp Files vs Azure Files?

This decision is based on what your users consider acceptable. Generally speaking, when you have less than 100 users accessing the file share simultaneously, an Azure Files, Transactional performance level works best. With workloads of between 100 and 2000 users, depending on the frequency of the file updates, consider Azure Files Premium performance level. With workloads over 2000 users, consider using the Azure NetApp Files. To reduce the traffic on the file share, consider using Citrix User Profile Management with profile streaming and large file handling enabled. You can also reduce traffic on the file share by using Microsoft FSLogix containers or Citrix Profile Management Containers.

When should the File Sharing Collaboration solutions be used?

File Sharing and collaboration services are designed to make share files accessible using the HTTPS protocol over the internet. These services allow not only individuals to store and retrieve files from the service, but also support collaboration between departments and even outside entities. They have security built in and provide a single point of access. These solutions are best for storing data securely that must be shared both internally and externally. Though they have been adapted to work as personal storage locations for users, they don’t always work well for storing user profiles.

• ShareFile: Secure file-sharing cloud-based service hosted by ShareFile.

  • File sharing repository accessible over HTTPS/CIFS
  • Light management required around user security
  • Limited maintenance of local StorageZone controllers. ShareFile maintains the rest of the infrastructure
  • Data owners can easily grant permissions to internal and external entities
  • Integrates with Active Directory
  • ShareFile-managed StorageZones are protected by durable cloud storage (in Azure)
  • Customer-managed StorageZones allows use of local customer data centers
  • ShareFile handles all the backups, antivirus, and indexing operations
  • All files stored encrypted with AES-256
  • Integrates with SharePoint and OneDrive
  • Supports mobile access to network shares
  • ShareFile Sync client can synchronize local user data with ShareFile storage
  • Includes document management, workflow management, content collaboration, and e-signing capabilities
  • Costs vary depending on functionality selected

• SharePoint in Microsoft 365: Cloud-based SharePoint service hosted by Microsoft

  • Limited management of user security and access
  • Microsoft maintains all servers
  • File-sharing repository accessible over HTTPS/CIFS
  • Light-weight version of SharePoint server
  • Data owners can easily grant permissions to internal and external entities
  • Integrates with Active Directory
  • Supports mobile access to network shares
  • Includes document management, workflow management, and content collaboration capabilities
  • Costs vary depending on functionality selected
  • Integrates with OneDrive to synchronize content

• OneDrive: OneDrive provides synchronization of user data between a local Windows workstation and a back end data storage location

  • Local agent client installed and configured to synchronize user data with SharePoint or ShareFile
  • Can be configured to automatically backup the Documents, Desktop, and Pictures folders to OneDrive in Microsoft 365
  • Included with Microsoft 365 licenses

Cloud Storage Design Considerations

Cloud storage technologies have changed the landscape of personal data storage expectations. Fortunately, most users now treat the extra latency as an acceptable tradeoff for being able to access their data securely from anywhere. Other design considerations when using these collaborative file shares include:

• Cloud-based file sharing and collaboration services have unlimited data storage.

• With collaborative file sharing services, highly durable data storage is used and backups are included in the cost of the service.

• Cloud-based file sharing and collaboration services do not support the installation of third-party data protection software. You are expecting the vendor to provide the protection against data loss, viruses, and loss of confidentiality.

  • These technologies are preferred when a need exists to share the files externally, such as with other businesses or third parties.
  • Using the ShareFile Sync agent with ShareFile or the OneDrive with SharePoint provides an excellent user data backup solution for their local device files.
  • Collaborative file shares are excellent for remote users that keep a large number of documents locally on their assigned device.
  • When collaborative file shares are used from non-persistent sessions, use the Session Linger setting so data can be synchronized before the session terminates.

• When using SharePoint

  • Keep the top-level parent portals to minimum to improve security, usability, navigation and adoption.
  • Avoid using deep hierarchies with unique permissions to improve performance.
  • Do not bury content or keep stale content as it impacts usability and deters users from using the site.
  • Use standard groups first (Members, Visitors, Owners) followed by AD Groups or SharePoint groups next, and direct user access last.
  • Take advantage of permission inheritance.

• OneDrive clients interoperate with GPOs and support folder redirection.

• OneDrive clients integrate with profile management solutions.

What if my users are accessing their data from both on-premises and in the Azure cloud?

• Collaborative file services work for sharing data internally and externally and also function as user data file repositories

• PaaS file services can be integrated directly with Windows since they can be mounted and accessed like internal file shares

• Augment PaaS file services with profile management solutions that virtualize the file system. This approach reduces the amount of data on the wire and reduces the monthly charges from outbound Azure Files data.

• Using PaaS file services prepares the way for complete cloud adoption while providing an improved experience for users accessing their data out of the cloud

What data methods work best together?

• Different methods are acceptable for different user groups based on their data access requirements

• To reduce costs, avoid combinations that store the same data in multiple locations, for instance do not use Citrix ShareFileSync and Citrix User Profile Manager with Azure Files

• For cloud-based Citrix workloads, combining a profile management solution with cloud-based file service has proven to be a good combination

  • Citrix User Profile Management with large file handing enabled on an Azure Files share
  • Using Microsoft FSLogix with Azure Files
  • Citrix ShareFileSync backed by Citrix ShareFile with Microsoft FSLogix

• For on-premises Citrix workloads, combine a profile management solution with the traditional file server technologies

  • Citrix User Profile Management with large file handing enabled on a traditional file server share
  • Using Microsoft FSLogix with a traditional file server share.

Device Management Considerations

The main challenge with device management is enforcing policies at the device level. Device policies can be enforced through GPOs or through endpoint management software.

For instance, GPOs applied through domain memberships are used to administer the devices by setting policies such as screen saver timeouts to improve security. Azure AD does not support device level management directly. However, GPOs for device management are available when Azure AD is used with an on-premises Active Directory or Azure AD DS.

Both Citrix and Microsoft provide solutions for managing mobile devices that can apply policies to iOS, Android and Windows 10 devices. Citrix provides the Endpoint Management service in Citrix Cloud while Microsoft offers Endpoint Manager, which includes Intune. Windows 10 includes modern features for managing devices and removes the legacy dependencies on Active Directory GPOs. You can choose your method of policy enforcement depending on how extensively device management is used within your organization. Here are the questions that you need to answer about Device Management:

Do I still need GPOs for my devices?

• If all of your user devices are running Windows 10 or later, GPOs may be replaced by Microsoft Intune policies

• If legacy applications require settings that cannot be deployed via Intune, then a traditional Active Directory DS is required

• Carefully review all existing GPOs in place, you may not need them any more

• Citrix VDA hosts still use GPOs

What are the requirements for Citrix Endpoint Management?

• Citrix Endpoint Management requires a Citrix Cloud Connector for directory synchronization

• Citrix Endpoint Management should be set to use the Citrix Identity provider through Secure Hub so Endpoint Management can authenticate directly to Azure AD

• Citrix Endpoint Management integrates with Azure AD as long as the users are not using local accounts

• Citrix Endpoint Management integrates with Microsoft Endpoint Manager so you can wrap your own line of business (LOB) applications with Intune and provide a micro-VPN

• Citrix Endpoint Management requires that enrollment invitations use LDAP authentication instead of Azure AD

• Citrix Endpoint Management requires user names, email addresses, and groups match between Active Directory DS and Azure AD

• Citrix Endpoint Management requires a Citrix Gateway (v12.1 or later) installed at your resource location for micro-VPN access, mobile productivity apps, or integration with Microsoft Endpoint Manager

• Citrix Endpoint Management requires a local StorageZone controller to support Citrix Files with private data storage

• Citrix Endpoint Management needs certificate-based authentication configured on the Citrix Gateway to provide a single sign-on experience

• Citrix Endpoint Management requires enrollment profiles for Android Enterprise and ΓÇ£Allow users to decline device managementΓÇ¥ set to off.

• Citrix Endpoint Management supports using the Citrix Cloud service to authenticate managed devices on the following platforms:

  • Apple iOS
  • Android BYOD
  • Android Legacy Device Administration mode

• To manage the Citrix Cloud Endpoint Management Service use the console found under My Services

What are the requirements for Microsoft Endpoint Manager?

• Microsoft Endpoint Manager does support both cloud and on-premises deployments

• Microsoft Intune requires Azure AD Global Administrator or Intune Service Administrator permissions to deploy

• Microsoft Intune does not have a hierarchy for applying settings to determine if one policy clearly has precedence. If two policies exist for the same setting within Intune, then a conflict results.

• Microsoft Endpoint Manager supports iOS, Android, Windows Mobile, and Windows 10

• Microsoft Intune can be licensed in one of three ways:

  • Standalone Azure service
  • Enterprise Mobility + Security (EMS)
  • Microsoft 365

• Azure AD Premium licenses are required for the following features:

  • Some AD join operations
  • Windows AutoPilot
  • MFA device settings
  • Conditional access
  • Dynamic device groups

• Manage Microsoft Intune through the Azure Intune console.

Network Level Considerations

In this section you will find a list of items to help you focus on the appropriate design decisions specific to the Citrix App Delivery Controller (ADC).

Licensing Considerations

Citrix Application Delivery Controller (ADC) on Microsoft Azure is a L4-L7 virtual networking appliance. The Citrix ADC provides organizations secure access to applications and assets deployed in Azure. Citrix ADC on Azure provides a foundation for the network infrastructure without any physical limitations. Citrix ADC on Azure comes in two models: VPX (virtualized) or CPX (containerized). Citrix also provides an Ingress Controller based on Kubernetes Ingress. The Ingress Controller can automatically configure the VPX and CPX models based on a defined configuration.

To ensure enterprise-grade reliability and security, Citrix ADC uses advanced traffic management, observability, and comprehensive security features. Selecting the correct model and feature set is beneficial when it comes to planning your architecture. Some questions to answer about model selection and features might include the following:

What use cases are best for the VPX Virtual Appliance?

• You use virtual appliances on your hypervisor instead of physical appliances

• You need high SSL performance with no hardware acceleration

• You have a hybrid cloud scenario

• You need load-balancing on-premises and in public or private clouds

• You are replacing MPX or other hardware load-balancers with virtual appliances

• You need a multitenant infrastructure with full isolation

What use cases are best for the CPX Containerized Appliance?

• You need to support Kubernetes or OpenShift containerized applications

• You require load-balancing for microservices traffic within a Kubernetes cluster

• You want load-balancing as part of a DevOps application development pipeline

What instance sizes and prerequisites are recommended for the Citrix ADC VPX virtual appliance?

• The compatible networking models with Microsoft Azure are Citrix ADC VPX 10, 200 and 1000. Any Citrix ADC VPX licenses work, including Standard, Advanced, and Premium edition licenses.

• Models VPX1000 and higher require version 13.0 build 76.x or later AND Accelerated networking be enabled to reach the wanted performance level

• VPX virtual appliances can be deployed on any instance type that has two or more Intel VT-X cores and more than 2 GB memory. Currently, Citrix ADC supports only Intel processors with the following instance size recommendations:

  • Standard D2s v4 for VPX10 or VPX200
  • Standard D4s v4 for VPX1000 or VPX3000
  • Standard D8s v4 for VPX5000
  • Standard D16s v4 for VPX10000

Do I need a Citrix Ingress Controller?

• Citrix ADC CPX and Citrix Ingress Controllers are deployed from the Azure Marketplace and used for microservices deployments

• Azure Kubernetes Engine (AKS), supports deploying a Citrix ADC CPX as an Ingress Controller with either basic or advanced (CNI) networking

• Citrix Ingress Controllers are used for microservice communication with a Citrix ADC CPX

• Citrix Ingress Controller can be deployed in a standalone pod as

  • a Tier 1 ADC device to proxy North-South traffic, which supports traffic outside the AKS cluster to microservices inside the cluster
  • a sidecar container to an ADC CPX to load-balance North-South or East-West traffic, which supports microservices traffic inside the AKS cluster

• Citrix ADC CPX Express is a 20 Mbps container-based ADC that can run on a Docker container and supports up to 250 SSL connections simultaneously

• Citrix Ingress Controller is freely licensed and has no usage fees, you only pay for the Azure costs

ADC Licensing

Review your licensing options before you choose a particular deployment model so you are aware of the options up front. In some situations, you can run a Citrix ADC for only the costs of the Azure infrastructure. Some ADC licensing questions might include the following:

Can I use the Citrix ADC VPX as an ICA Proxy without buying a license?

• Citrix ADC in basic mode has the ICAOnly VPN virtual server parameter set to ON and works fully on an unlicensed VPX instance

• Citrix ADC in Smart-Access mode has the ICAOnly VPN virtual server parameter set to OFF and only supports 5 AAA session users on an unlicensed VPX instance

• Apply a Premium license to the Citrix ADC VPX instance to license more than 5 AAA sessions

• Citrix ADC VPX Express version 12.0.56.30 or later does not require a license file

• Citrix ADC CPX Express is a freely licensed CPX, you only pay the associated Azure costs

How is Citrix ADC licensed in the cloud?

• Citrix ADC on Azure is available with pay-as-you-go licensing through the Azure Marketplace subscription or using your own perpetual licenses

• Using your own perpetual license is referred to as Bring Your Own License (BYOL)

• BYOL requires the MyCitrix licensing portal to generate a valid license for Azure

• BYOL is the only licensing model available on Azure if you are not using the Azure Marketplace subscription

• License activation requires access to the public domain internet

Does Citrix ADC support check-in/check-out licensing model under the Citrix Application Delivery Management (ADM) service?

• Citrix ADC supports Check-in/Check-out licensing from Citrix Application Delivery Management (ADM), which has an automated license provisioning system

• Requires Citrix ADC VPX running 12.0 or later

• Requires Citrix ADM running 12.0 or later

• All licenses must be rehosted to Citrix ADM

• When Citrix ADC instances are removed or destroyed, licenses are automatically returned for reuse

Occasionally, the Citrix ADC VPX may come online with a default ADC license unexpectedly. To resolve this issue, do a warm restart before making any configuration changes to the ADC VPX instance to allow the Azure Instance Metadata Service (IMDS) to correct the licensing.

Scalability Considerations

Designing your ADC architecture and planning the deployment are the two key activities for the transition. Selecting the correct features and the best architecture model for your ADC deployments can be both time consuming and challenging. This section provides guidance about the Citrix ADC features and functionality to help you choose the best model.

What types of deployments are available and what are the best practices when deploying that type?

• Use multi-NIC and multi-IP design when you are deploying into production where high-availability requirements for redundancy or security exist

  • Using Citrix Solution Templates in the Azure marketplace is the recommended deployment method
  • Citrix recommends deploying the multi-NIC architecture using the “Citrix ADC” Citrix solution template from the Azure Marketplace\ Choosing this Citrix solution template gives you the following software plan options:
    • Citrix ADC VPX Bring Your Own License
    • Citrix ADC VPX Subscription License
    • Citrix ADC HA (Availability Zone) - SL
    • Citrix ADC HA (Availability Set) BYOL
    • Citrix ADC HA (Availability Set) - SL
    • Citrix ADC HA (Availability Zone) BYOL
    • Citrix ADC FIPS HA (Availability Zone) BYOL
    • Citrix ADC FIPS Standalone BYOL
    • Citrix ADC FIPS HA (Availability Set) BYOL
  • Integrates with Citrix ADM for GSLB (Traffic management) and Licensing
  • Best for the following use cases
    • Isolation of data and management traffic
    • Improved scale and performance of the ADC
    • Where applications require more than 1 Gbps of throughput
    • Web Application Firewall (WAF) deployments

• Use the single-NIC, multi-IP design for production environments with a single subnet or for non-production environments, such as testing

  • With a single NIC, you have 3 IP configurations:

    • ipconfig1 is management
    • ipconfig2 is client-side traffic
    • ipconfig3 is back-end server traffic

  • Ipconfig3 should not have a public IP address associated with it

  • Add IP addresses for all the configurations in the Azure portal first before configuring them in the Citrix ADC

  • Create an untagged VLAN for each data interface on the ADC VPX and bind the primary IP of the NIC. This procedure helps prevent MAC moves and interface changes in Azure from unexpectedly impacting your ADC.

• Use the single-NIC, single-IP for a Citrix ADC in standalone mode.

  • All functions, NSIP, SNIP, and VIP are tied to a single Citrix ADC IP address
  • Configure the resource group, network security groups, and virtual network before you provision the Citrix ADC VPX VM so the network information is available before provisioning
  • Only available in Azure and on Azure stack

• When deploying High Availability using Availability Sets (recommended)

  • The ADC VPX needs an HA independent Network Configuration (INC)
  • The Azure Load Balancer must be configured in Direct Server Return (DSR) mode

• When deploying High Availability using Availability Zones

  • Use the “Citrix ADC” Citrix Solution template in the Azure Marketplace with a software plan where "(Availability Zone)" is included in the name
  • Currently, not all Azure regions support Availability Zones, so check in your region before deploying this Solution template

What are the benefits of using Azure accelerated networking?

• Accelerated networking is not available on all instance types and the VMs must be stopped to before enabling accelerated networking on a NIC

• You must perform all configuration changes from the Citrix ADC VPX PV interface. Use the ADC show interface command to determine which physical interface is bound to PV

• Citrix recommends not performing any operations on the Citrix ADC VPX VF interface. If you must perform operations on the VF interface, Citrix only allows the clear stats or enable, disable, and *reset interface operations. VLAN binding is unavailable.

What methods are available for deploying Citrix ADC?

• Deploy through the Azure Marketplace. The Citrix ADC VPX virtual appliance is available as an image in the Microsoft Azure Marketplace.

• Deploy using the Citrix ADC Azure Resource Manager (ARM) json template available on GitHub.

• Deploy using Citrix ADM service.

Traffic Distribution

With DNS-based autoscaling, DNS is the layer that decides where the traffic is routed. The traffic manager uses DNS to direct the client traffic to the appropriate Citrix ADC instance that is available in the Citrix Application Delivery and Management autoscaling group. Azure traffic manager resolves the FQDN to the VIP address of the Citrix ADC instance.

With Azure Load-Balancer (ALB) as the traffic manager, inbound traffic goes first to the ALB and it decides where the traffic is routed. ALB manages the client traffic and distributes it to Citrix ADC VPX clusters. ALB sends the client traffic to Citrix ADC VPX cluster nodes that are available in the Citrix Application Delivery and Management autoscaling group across availability zones.

With both traffic distribution options, the Citrix Application Delivery and Management triggers the scale-out or scale-in action at the cluster level. When a scale-out is triggered, the registered virtual machines are provisioned and added to the cluster. Similarly, when a scale-in is triggered, the nodes are removed and de-provisioned from the Citrix ADC VPX clusters.

How do you deploy Citrix ADC VPX on Azure with Global Server Load Balancing (GSLB) and use Azure DNS Private Zones?

• When using DNS-based traffic management, each Citrix ADC instance in the Citrix Application Delivery and Management Autoscale group requires a public IP address.

• For DNS-based autoscaling, Application Delivery and Management waits for the specified Time-To-Live (TTL) period. After the TTL expires, it waits for existing connections to drain before initiating node de-provisioning.

• When using ALB-based traffic management, the public IP address is allocated to Azure Load Balancer. Citrix ADC VPX instances do not require a public IP address.

• The Citrix ADC requires either a DNS virtual server or a nameserver configured which is used by the Azure Load balancer for resolution

• For a Hybrid GLSB configuration (multi-cloud/data center)

  • A SNIP address or GLB Site IP address must be configured on each Citrix ADC node for metrics exchange between the nodes
  • The ADNS or ADNS-TCP service must be set up on the Citrix ADC nodes to process the DNS traffic
  • The Azure cloud security groups and firewalls must allow traffic on ports 53 and 3009
  • Support for GSLB Load-balancing solutions other than Citrix ADC is limited
  • Use the Multi-cloud GLB StyleBook for configuration of Global Load Balancing

Autoscale Guidance

An Autoscale group is a group of Citrix ADC instances that load-balance applications as a single entity. The number of instances in the ADC Autoscale group is based on the configured parameters, such as CPU usage. The Azure infrastructure (ALB or Azure traffic manager) sends the client traffic to a Citrix Application Delivery and Management autoscaling group in the availability set. Citrix Application Delivery and Management triggers the scale-out or scale-in action at the cluster level.

What are the requirements for integrating Citrix ADC with Azure Autoscale?

• Using Autoscale with Azure virtual machine scale sets (VMSS) with multi-IP deployments enabled for high-availability minimizes costs. Citrix recommends using Autoscale to reduce the amount of configuration and overhead necessary to monitor the server performance across VNets.

• An Azure Active Directory (AAD) application and service principal with contributor role on the affected resources are required to implement Autoscale

• With auto-scaling an IP set is created on clusters in every availability zone. After which, the domain and instance IP addresses are registered with the Azure traffic manager or ALB. When the application is removed, the domain and instance IP addresses are deregistered from the Azure traffic manager or ALB. Then, the IP set is deleted.

Citrix Application Delivery Management (ADM) Service

The Citrix Application Delivery Management Service (ADM) within Citrix Cloud provides a centralized location to manage your Citrix ADC deployments. These deployments include Azure cloud or on-premises versions of the following: Citrix ADC MPX, Citrix ADC VPX, Citrix ADC SDX, Citrix ADC CPX, Citrix Gateway, and Citrix Secure Web Gateway appliances. Citrix ADM is a cloud-based solution that manages, monitors, and assists with troubleshooting your entire application delivery infrastructure. Citrix ADM includes all the necessary capabilities to deploy, automate, and license Citrix ADC within an easy to navigate cloud-based console.

How does the Citrix ADM service work?

• Deploy through the Azure Marketplace. The Citrix ADC VPX virtual appliance is available as an image in the Microsoft Azure Marketplace.

• Deploy using the Citrix ADC Azure Resource Manager (ARM) json template available on GitHub

• Deploy using Citrix ADM service

StyleBooks

The most complex part of deploying an ADC is configuring it to work with your authentication system and your applications. Citrix offers StyleBooks to help ease the configuration experience. StyleBooks offer a way to simplify the complex task of Citrix ADC configurations. A StyleBook is a pre-configured template that users can use to create or manage Citrix ADC configurations. StyleBooks exist for most of the common applications and configurations. For instance, The SSO Office 365 StyleBook allows you to enable SSO for Microsoft Office 365 through Citrix ADC instances.

What are the Citrix ADM StyleBook applications templates and how do I use them?

• We recommend using StyleBooks for initial configurations if one is available. StyleBooks for Microsoft 365, Skype, Exchange, SharePoint, and ADFS are available

• Microsoft SharePoint StyleBook requires the following:

  • Sharepoint 2016 or later
  • Citrix ADM v12.0 or later
  • Citrix ADC v10.5 or later

• Microsoft SharePoint StyleBook supports the Load Balancing, Content switching, Responder, Rewrite, Compression, and Integrated Caching features of Citrix ADC

• When using SSL with the SharePoint StyleBook, verify that the Rewrite configuration parameter is enabled in the SharePoint Advanced Settings section of the StyleBook

• Citrix recommends you first select Dry Run to view the configuration objects that the StyleBook creates on the target Citrix ADC instances, If acceptable, then go ahead and execute the actual configuration.

Load-balancing with Azure Tag

For Citrix ADC VPX standalone and high-availability instances deployed on the Azure Cloud, now you can create load-balancing service groups associated with an Azure tag. The VPX instance constantly monitors Azure virtual machines (back-end servers) and network interfaces (NICs), or both, with the respective tag and updates the service group accordingly. The VPX instance creates the service group that load balances the back-end servers using tags. Whenever a VM or NIC with the appropriate tag is added or deleted, the ADC detects the change and updates the service group automatically.

How do I configure Load Balancing to use Azure Tags?

• Tags must be assigned to the VM instance or the VM’s NIC

• When using the Azure CLI to propagate tags, the secondary (standby) Citrix ADC must terminate the rain_tags process after a warm restart. This behavior prevents the old information from being used inadvertently

• The ADC VPX needs to be able to reach the tagged IP Address for the back-end server. For a tagged VM, this is the primary IP address, for a tagged NIC, it is the NIC’s IP address. If the VM is on a different VNet, then VNet Peering must be enabled.

• Save all configurations so they persist between VM restarts.

High Availability Considerations

Within the Azure cloud, the Citrix ADC virtual and containerized appliances have reduced feature sets. Some features, such as VLAN tagging, are no longer necessary because Azure performs the functionality at the infrastructure level. Understanding the limitations and the requirements are key to planning your migration. Using GSLB and Azure for the HSM Key Vault have other requirements that you should be aware of.

Azure Key Vault

Citrix ADC integrates with Azure Key Vault and stores its private keys in the Key Vault, which increases the security protection of the keys. Using Azure Key Vault simplifies the storage and management of keys. Azure Key Vault provides a central key management location for all enterprise ADC appliances across both Azure and the on-premises data centers.

Some questions to answer during the planning stages might include the following:

How does the ADC application integrate with Azure Key Vault and what are its limitations?

• Citrix ADC integration with Azure Key Vault requires the use of the TLS 1.3 protocol

• FIPS 140-2 level 2 compliance requires Azure Key Vault Premium pricing tier and the use of hardware security module (HSM) backed keys

• The ADC will access the Key Vault for each SSL handshake

• Access to the Azure Key Vault requires an Azure Enterprise application and service principal

• Citrix ADC use of Azure Key Vault has the following limitations:

  • Azure Key Vault limits the number of concurrent calls and the limits vary by request type and key type
  • Elliptic-curve cryptography (ECC) keys are not supported
  • HDX Enlightened Data Transport (EDT) and Datagram Transport Layer Security (DTLS) protocols cannot be used to communicate with Azure Key Vault
  • Clustering and admin partitions are not supported
  • The Azure application, Azure Key Vault, and HSM certificate-key pair cannot be updated in Azure after adding them to the Citrix ADC appliance
  • HSM certificate bundles are not supported
  • An HSM key cannot be bound to a DTLS virtual server
  • Neither the SSL Service or Online Certificate Status Protocol (OCSP) requests can use a certificate-key pair created with the HSM key
  • No error is generated when an HSM key and certificate mismatch occurs

GSLB

As businesses transition their workloads to the Azure Cloud, they need a hybrid model that allows DNS resolution in a secure manner. The Azure DNS Private Zone service is the key to this transition. With Private DNS zones, businesses can create a hybrid model that allows DNS resolution for both on-premises and Azure-based servers. The Azure servers can be connected to the on-premises data center via an ExpressRoute or VPN tunnel. Citrix ADC provides a seamless way for distributing traffic across both the on-premises and Azure workloads at a global scale. The Global Server Load Balancing (GSLB) feature provides that global scale and relies on the ADNS service within the Citrix ADC console.

This GSLB feature supports business goals including: migrating from on-premises to the Azure cloud, DNS-based failover, and blue-green environment testing. Both Round Robin and Location-based (static proximity) server routing methods are available. GSLB can be used for any service or host resolution, including StoreFront.

What are the requirements and limitations of using Citrix ADC for GSLB across both my on-premises and Azure cloud hybrid deployment?

• The ADNS service is a DNS server that runs on the Citrix ADC appliance. ADNS supports delegation of DNS name spaces, such that the Citrix ADC is the authoritative name server for the zone and all hosts within it

• Support for GSLB Private DNS zones is implemented using Citrix ADC appliances in the Azure cloud running the ADNS service

• Plan to use DNS forwarders for both virtual networks and data center networks

• All DNS queries are routed first to the local DNS forwarder to provide the best user experience

• GSLB DBS Service requires the following:

  • Citrix ADC version 12.0.57 or later and Microsoft Azure Load Balancer instances
  • Citrix ADC GSLB Service Group Feature Enhancements
  • GSLB Service Group entity: Citrix ADC version 12.057 or later
  • DBS feature components must be bound to the GSLB service group

What are the limitations of running Citrix ADC VPX instances on Azure?

• A secure tunnel between Azure and the on-premises data center must exist, typically across an ExpressRoute or VPN connection

• Assign a static Internal IP address to the Citrix ADC virtual machine to avoid issues caused by the IP address changing after a VM deallocation

What data center Citrix ADC functionality is not available in the Azure Citrix ADC?

• High availability does not work if the Public IP (PIP) address is associated with the VPX instance instead of an Azure Load Balancer

• The Azure architecture does not support the following Citrix ADC features:

  • Clustering, unless deployed via the Citrix ADM Autoscale feature
  • IPv6
  • Gratuitous ARP (GARP)
  • L2 Mode (Bridging); however, Transparent virtual servers with MAC rewrite (L2) will work for servers on the same subnet as the ADC’s SNIP
  • Tagged VLAN
  • Dynamic Routing
  • Virtual MAC
  • USIP
  • Jumbo Frames

• Public IP addresses do not support protocols where the port mapping is opened dynamically, such as passive FTP or ALG.