Xi Frame on Nutanix AHV

Bring Your Own Nutanix AHV Cluster to Frame

You can host your applications, desktops, and user data within your own private cloud using Nutanix AHV infrastructure with Frame. To use your Nutanix AHV cluster, you will deploy a Cloud Connector Appliance (CCA) which enables Frame Platform to communicate with Prism Central and Prism Element on your Nutanix AHV cluster. Frame Platform leverages Nutanix AHV infrastructure to enable a rich of enterprise grade features:

  • High Availability (HA) configuration

  • Proxy server (authenticated and unauthenticated) support

  • Instance definition including vGPU configurations

  • Multiple gold/template images supporting Windows and Linux

  • Enterprise Profiles

  • Personal Drives

Announcement

Support for Cloud Connector Appliance (CCA) 2.x will be deprecated as of October 31, 2020. Existing CCA 2.x customers will need to upgrade to CCA 3.0 as soon as possible. Instructions for upgrading the CCA can be found here.

The ISO file for CCA 3.0 can be downloaded from the Nutanix Downloads Portal.

As part of the upgrade process, customers will also need to configure new firewall rules and proxy servers to allow for the new FQDNs required by CCA 3.0. A complete list of required FQDNs for CCA 3.0 can be found in our network requirements documentation.

Requirements

  • A Nutanix cluster running Acropolis Hypervisor (AHV), Acropolis Operating System (AOS), and Prism Central version 5.11.2 or newer

  • If vGPU instance types are required, NVIDIA GRID vGPU drivers, license manager, and licenses installed on the AHV cluster*

  • Frame Guest Agent (which you can find at https://portal.nutanix.com/#/page/XiFrame)

  • A valid Xi Frame subscription

  • Compliance with the Network Configuration Requirements in order for the Nutanix Frame components in the AHV cluster to communicate with Frame Platform.

* Supported vGPUs can be found here.

Attention

The automated deployment of Cloud Connector Appliance for Frame on AHV is supported for standard installations of Prism Central 5.11.2 or higher, and the latest CCA version found on Nutanix downloads portal. If you have a custom/older version of either component, please follow the Xi Frame on AHV (Manual CCA) instructions.

Setup Overview

Adding your AHV cluster to your Frame customer or organization entity and creating a Frame account on the AHV cluster requires the following 4 steps:

Please make sure to read the requirements before getting started.

Prepare your Nutanix Cluster

Navigate to your Prism Central and Prism Element Management console to prepare your cluster for Xi Frame on AHV. You will want to start by creating Prism Central and Prism Element user accounts to be used by Frame for provisioning and infrastructure management.

  1. Go to your Prism Central settings by clicking on the gear menu in the top right corner of the management console.

    ../_images/ahvprep1.png
  2. Click on “Local User Management” from the menu on the left-hand side. Click the “New User” button.

    ../_images/ahvprep2.png
  3. Fill out the “Create User” form that appears.

    ../_images/ahvprep3.png
  4. Ensure that both “User Admin” and “Prism Central Admin” boxes are checked before submitting the information.

    ../_images/ahvprep4.png
  5. Save your Prism Central user name and password – you will need these later on in the process.

  6. Now, create a user account from the Prism Element of the Nutanix AHV cluster which will be used for Frame desktops and applications.

  7. Go to Prism Element and access the Prism Element settings by clicking on the gear menu in the top right corner of the management console.

  8. Click on “Local User Management” from the menu on the left-hand side. Click the “New User” button.
    ../_images/ahvprep5.png
  9. Fill out the “Create User” form that appears.

  10. Ensure that both “User Admin” and “Cluster Admin” boxes are checked before submitting the information.

    ../_images/ahvprep5-1.png
  11. Save your Prism Element user name and password – you will need these later on in the process.

  12. Click on the hamburger menu icon in the upper left portion of your Prism Central interface.

  13. Expand the “Virtual Infrastructure” menu item and select “Categories.”

  14. Click on the “New Category” button.

../_images/ahvprep6.png
  1. Fill in the form with the following values:

    ../_images/ahvprep7.png
    • Name: FrameRole

    • Purpose: Create and Manage Frame Instances

    • Values: Instance, Template, and MasterTemplate (Add each value on separate lines using the plus symbol.)

  2. Click “Save.”

Prepare a Gold Image

Before the AHV cluster can be added to your Frame customer or organization entity, you need to create at least one template or “gold” image. This template image will be used to create the Sandbox VM disk of each Frame account you create once the AHV cluster is added to Frame.

Considerations

Please consider the following before preparing your template image:

  • Nutanix Guest Tools cannot be installed onto your template image, as this could cause communication issues between the Frame backplane and workload instances. If your image already has Nutanix Guest Tools installed, you must install VirtIO drivers before uninstalling Nutanix Guest Tools. If you attempt to remove Nutanix Guest Tools without first installing VirtIO drivers, your virtual machine will not boot.

  • When creating a new template image or adding an existing disk to a VM in Prism Central or Prism Element, you must use SCSI as the bus type. Do not use a volume group as the disk type. IDE bus types and volume groups disks are not supported as Frame template image VM disks.

  • Due to technical specifics of how the Frame display driver works, once the Frame Guest Agent is successfully installed the VM can no longer be accessed via the built-in AHV VNC console. Enable Microsoft RDP in your template image before installing the Frame Guest Agent to access the template image VM.

  • We recommend disabling automatic Windows updates on the template image. Automatic updates should also be disabled on non-persistent images locally and via any applied GPOs if linked to production pool VMs. Leaving Windows updates enabled could potentially impact user performance in certain use cases.

  • Make sure to set the timezone in the template image VM to UTC. When users start a Frame session after a Frame account is created, the workload VM will be set to the correct timezone based on the user’s browser locale.

  • When setting up the template image, it can be useful to have the Windows Firewall disabled. Firewall can be customized and re-enabled later in the Sandbox for a Frame account. To do so, run the following command in Powershell.

    Set-NetFirewallProfile -Profile Domain, Public, Private -Enabled False
    
  • If you plan on setting up Domain Join for your account, we strongly recommend that you do not domain join the template image VM since the sysprep procedure to generalize the template image VM will remove the template image VM from the domain.

  • If you require HTTPS traffic from your workload VMs to go through a proxy server to reach the Internet (and Frame Platform), you will need to configure the proxy setttings in the Windows registry of the gold image. Frame Guest Agent supports authenticated (username/password) and unauthenticated proxy servers, provided they can handle HTTPS and Secure WebSocket traffic. Refer to our Proxy Server documentation page for further details.

Procedure

Let’s create a Windows template image. If you plan on using a Windows 10 image, there are certain conditions that could cause sysprep to fail. Please reference this article from Microsoft for additional details.

  1. First, starting with a Windows 10, Windows Server 2016, or Windows Server 2019 image, create a VM in the cluster you are going to use for Xi Frame. When configuring your image, you must use a Windows OS user account with local administrative privileges.

Note

Windows 10 base images must be a 64-bit version of the operating system.

Warning

The Windows user account names “Frame” and “FrameUser” are reserved. Please use a different user name when setting up your Windows local administrator account to prevent issues accessing your image.

  1. Access the VM via RDP. Do not use the built-in VNC client. Once the Frame Guest Agent is installed, the Frame video driver will prevent you from accessing the VM using VNC (you will see a frozen Windows OS loading screen). To enable RDP on your VM, run the following Powershell command as an Administrator.

    Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -Name "fDenyTSConnections" -Value 0
    

Attention

Before moving on to step 3, we strongly suggest that you create a snapshot of your template image to use if needed.

  1. Confirm the timezone is set to UTC.

  2. Install the Frame Guest Agent using either the standalone executable or using an ISO image with the executable. We recommend using the ISO image since it can be uploaded to Prism and mounted to the virtual CD-ROM.

  3. If a proxy server is required for all outbound traffic to the Internet from your private network, you will need to configure Frame Guest Agent using the FrameProxyHelper tool. Refer to our Proxy Server documentation page for further details.

  4. Once you have configured the image the way you like, launch Powershell as an Administrator.

  5. Run the following command to generalize your image using Microsoft Sysprep:

    Start-Process -FilePath "C:\Windows\System32\Sysprep\Sysprep.exe" -ArgumentList "/oobe /shutdown /generalize /unattend:C:\ProgramData\Frame\Sysprep\Unattend.xml" -Wait -NoNewWindow
    
  6. Verify that sysprep succeeds by examining the sysprep logs in %WINDIR%System32SysprepPanther.

  7. If you mounted the Frame Guest Agent ISO file as a CD-ROM, eject the CD-ROM device on the VM.

  8. Stop the VM.

  9. Navigate to your VM list and click on your VM. Put the VM in the FrameRole category you created earlier with the value MasterTemplate by opening the “More” drop-down menu and selecting “Manage Categories.”

Warning

Make sure the MasterTemplate VM does not get deleted.

You have successfully created a template image to use for your Xi Frame workloads. You may prepare additional gold images if you wish to register additional gold images.

If you run into any issues, check out the Troubleshooting section at the end of this guide.

Connect your AHV Cluster to Frame

Connecting your AHV cluster to Frame is the final task required to get Frame running on AHV.

  1. From your home drop-down menu on the left, click “Settings.”

  2. Under the “Setup” section of the “Settings” menu, click “Connect to Frame.”

    ../_images/ahvCCA_1.png
  3. On this page, specify the following:

    ../_images/ahvCCA_2.png
    • Select Frame Service: This is your Frame service deployment type. Most customers select “Commercial” for Xi Frame. Select “Government” for Xi Government Services.

    • Dedicated AHV cluster: Select the cluster where the CCA VM will be deployed from the drop-down menu.

    • Network connected to Prism Central: Specify the VLAN where the CCA VM will be installed from the drop-down menu. The CCA VM must be able to access Prism Central and Prism Element from this VLAN.

Note

DHCP must be configured for the “Network connected to Prism Central” in order for CCA to be automatically provisioned and functional. If you do not have DHCP configured, you will need to connect to the CCA console (once the CCA VM has been configured) and configure a static IP address for the CCA. Please refer to Configuring CCA/WCCA for Static IP Addresses.

  1. Click “Next.” The image for the current CCA version will be downloaded from Nutanix and deployed on the Nutanix AHV cluster specified in the previous step. The progress can be monitored in the status bar. Once complete, click on the “Configure” button.

    ../_images/ahvCCA_3.png

Note

If the progress bar stops at 25%, then Prism Central was unable to download the CCA image. Please verify your network connectivity from Prism Central to the Internet.

Note

If the progress bar stops at 75%, then the CCA VM was successfully provisioned but does not have an IP address. Verify your DHCP server for the VLAN or manually assign the CCA VM with a static IP address.

Note

Each time “Connect to Frame” is clicked, Prism Central will attempt to provision a new CCA VM.

Note

The CCA wizard can also be accessed using its IP address as seen in the Prism Central. For example https://192.168.2.X.

  1. Login with the Prism Central user account that you created earlier for Xi Frame and enter the Prism Central URL.

    ../_images/ahvconnect1.png
  2. CCA can communicate via an HTTPS proxy server to the Frame backplane. If a proxy server is not required, just click Continue. Enable “Use proxy” slider if a proxy server is required.

    ../_images/ahvconnect2-1.png

Specify the proxy server URL which the CCA will use to reach the proxy server. If the proxy server requires a service username and password, specify the service username and password required for the CCA to authenticate to the proxy server. Otherwise, leave the proxy username and password blank. Use the “Verify” button to validate your proxy server configuration.

../_images/ahvconnect2-2.png
  1. If you are creating a new AHV cloud account, choose the option Connect Appliance to Frame. If you have an AHV cloud account already and want to set up high availability (two or more CCA VMs) or migrate your AHV cloud account to a later CCA software version, please see Adding the CCA to an Existing Frame AHV Cloud Account.

    ../_images/ahvconnect2-3.png
  2. Specify the AHV cluster, Prism Element credentials of the user account created for Xi Frame, and workload VLAN (as defined within Prism Central). Provide a name for the AHV Cloud Account which will be displayed in the Frame Control Panel.

    ../_images/ahvconnect2-4.png

Note

The cloud account name will default to the cluster name. It is important to choose a Cloud Account name which is unique to easily identify the cloud account, especially if more than one cloud account will be set up in the same cluster on Xi Frame.

  1. In order to create Enterprise Profiles and/or Personal Drives, check the designated check box. Choose the storage container to store the data for Personal Drives and/or Enterprise Profiles.

    ../_images/ahvconnect2-5.png

Note

The decision to use (or not use) enterprise profiles and/or personal drives can be changed by creating a new CCA VM and attaching the appliance to Frame. In addition, for persistent data like the profile disk, personal drive and persistent VMs, you should use a container that has compression and dedup enabled. For non-persistent data and VMs, choose a container with compression only and no dedup.

  1. Click on Add Instance Type to define and add the VM instance types to use for your AHV-hosted Xi Frame account.

    ../_images/ahvconnect3.png

    When defining instance types, the AHV Cluster Administrator can now specify instance types with vGPU profiles.

    ../_images/ahvconnect3-1.png
  2. After creating the desired instance type(s), you now need to choose gold images (or template images) for the Sandbox VMs that will be created when you create Frame accounts.

    ../_images/ahvconnect4.png

Note

If the CCA does not list the template VMs you expect, then verify that the FrameRole category and MasterTemplate value has been set on the template image VM in Prism Central. You can refresh this page once the template VMs have been properly categorized in Prism Central.

Note

To ensure a smooth setup, please make sure you logged in to your Frame customer entity or organization entity (either through “My Nutanix” or your identity provider) in a separate browser window tab before proceeding to the next step. If you do not see your Customer or Organization entity in Step 4, login to https://console.nutanix.com/ in a separate tab. Then, switch to the CCA wizard in your other browser tab, return to Step 3 using the Back button, and then go to Step 4. Your Customer or Organization entity should be visible as you have been authenticated and authorized to your Customer or Organizatiion entity.

  1. Select the Customer or Organization entity you want the AHV cluster to be associated with in Xi Frame. Click “Connect”.

    ../_images/ahvconnect5.png
  2. The wizard should inform you that your cluster has been connected successfully.

    ../_images/ahvconnect6.png

After the CCA has been added to Frame and the AHV Cloud Account is successfully registered, the administrator can go to Frame Control Panel and create the first Frame account.

  1. Log in to your Xi Frame account to check the cloud account creation status. The “C” status indicates that the account is still being created.

    ../_images/ahvconnect9.png

Note

You may need to refresh your browser page to update the AHV Cloud Account status. If your account status hasn’t changed after an hour, please create a ticket through your MyNutanix support portal and provide your cloud account’s display name. You can locate the cloud account’s display name by navigating to the Customer or Organization entity’s “Cloud Accounts” tab.

  1. The account status should change from “C” (Creating) to “R” (Ready).

    ../_images/ahvconnect10.png

Creating your first Frame Account

  1. Once the cloud account is ready, you can create a Frame account using your AHV infrastructure. From the account creation page in Frame, click “Nutanix” to use your AHV infrastructure. Select the Cloud Account Name you created earlier and choose the Network for Frame desktops. Click “Next”.

    ../_images/ahvconnect7.png
  2. Create the sandbox by specifying the OS version and the Instance Type.

    ../_images/ahvconnect8.png
  3. Navigate to your freshly-created account. On the Sandbox page of your Dashboard, you will see that your Sandbox is “under maintenance.” The most recent Xi Frame server components are likely being installed at this time. When the Sandbox VM ready for use, the status will update to “Running.”

  1. Click the “Start Session” button to enter your Sandbox.

    ../_images/ahvconnect14.png

You can now manage your Xi Frame account in the Dashboard.

Configuring CCA/WCCA for Static IP Addresses

By default, CCA and WCCA VMs are assumed to obtain their IP address from a DHCP server. If a static IP address is required for either of these two Frame components, then the IP address of the VM will need to be manually configured.

  1. Open console of the CCA or WCCA VM from Prism Central, and login as netconfig user (no password).

    ../_images/netconfig-static-IP.png
  2. Edit the IP address, gateway, DNS servers, and search domain and Save.

Set up WCCA to use Proxy Server

A Workload CCA (WCCA) is an appliance that is installed by the Frame Platform automatically on the AHV cluster as part of creation of an AHV Cloud Account. The WCCA forwards orchestration information from Frame Platform to all Frame-managed workload VMs (Sandbox, Production instances, and Utility Servers). Without this appliance, end users will not be able to connect to the workload VMs.

The following step is required only if you want the WCCA to use a proxy server.

  1. Open console of the WCCA VM from Prism Central, and login as proxyconfig user (no password).

    ../_images/ahvconnect15.png
  2. Please provide the PROXY_URL in the dialog box in the format https://username:password@1.2.3.4:8080 where 1.2.3.4 would be the IP address to the proxy server and 8080 is the proxy server port. If the proxy server requires no username/password for authentication, then the URL has a pattern of https://1.2.3.4:8080. Note: https or http can be specified, depending on the requirements of the proxy server.

    ../_images/ahvconnect16.png
  3. Click on the OK button, which redirects the user back to the start screen.

After this configuration is applied, the WCCA will start using the proxy server to establish a connection to Frame Platform.

Adding the CCA to an Existing Frame AHV Cloud Account

This section describes the workflow if you have an existing AHV Cloud Account and would like to perform any of the following operations:

  • Setup a highly available CCA configuration

  • Update the version of a CCA VM

  • Changed the AHV cluster’s IP address and need to create a new CCA VM

  • Enable Enterprise Profiles, Personal Drives, or proxy server for an existing AHV Cloud Account

  • Modify settings for Enterprise Profiles, Personal Drives, or an existing proxy server configuration

First, follow Steps 1-5 in Connect your AHV Cluster to Frame to create a new CCA VM and login to the CCA setup wizard with your Prism Central credentials. Once you have logged in, you can use the following workflow to update your proxy server settings (Steps 1 and 2 below) or make additional configuation changes (Steps 3 through 7).

  1. If you want to update the CCA configuration for your existing AHV Cloud Account to use a proxy server, then click on the slider to Use Proxy.

    ../_images/ahvconnect2-1.png
  2. Specify the proxy server URL which the CCA will use to reach the proxy server. If the proxy server requires a service username and password, specify the service username and password required for the CCA to authenticate to the proxy server. Otherwise, leave the proxy username and password blank. Use the “Verify” button to validate a valid proxy server configuration.

    ../_images/ahvconnect2-2.png
  3. Since you are adding a CCA instance to an existing AHV cloud account, choose the Attach Appliance to Frame option.

    ../_images/ahvconnect2-3.png
  4. Specify the AHV cluster and its Prism Element credentials. At this step, you can update the Enterprise Profiles and Personal Drives settings as well for the existing AHV Cloud Account.

    ../_images/ahvconnect2-6.png
  5. Select the Customer or Organization entity you created earlier in the Xi Frame account setup. Choose the AHV Cloud Account for which high availability configuration is required and Click “Attach.”

    ../_images/ahvconnect2-7.png
  6. The wizard should inform you that your cluster has been connected successfully.

    ../_images/ahvconnect6.png
  7. Once you have attached the new version of the CCA to the AHV cloud account, you can power off the CCA VM running the older version of CCA and terminate the VM. You can perform the above steps again to add additional CCA VMs of the new version for high availability and scalability.

Deleting an AHV Cloud Account

Xi Frame administrators can deactivate an AHV Cloud Account from Frame when they wish to disassociate or delete an AHV Cloud Account registration from their Customer or Organization entity. Once the AHV Cloud Account is deactivated, the AHV cluster administrator can terminate the corresponding Cloud Connector Appliance (CCA) VM(s) and Workload Cloud Connector Appliance (WCCA) VMs through Prism Central.

In order to disassociate an AHV Cloud Account, please follow the following steps:

  1. Terminate any Frame Accounts that are using the AHV Cloud Account which needs to be deleted. This can be done by clicking the ellipsis and click Update to Terminate a Frame account. When a Frame account is terminated, the workload VMs including the Sandbox, Utility Server(s), production VMs, and backups will be terminated.

../_images/deleteahvcloudaccount1.png ../_images/deleteahvcloudaccount2.png
  1. After deleting the Frame accounts, access the AHV Cloud Account (from Organization or Customer entity wherever it is associated) and click on the ellipsis. The Deactivate menu item can now be selected to deactivate the AHV Cloud Account on the Customer or Organization entity.

../_images/deleteahvcloudaccount3.png ../_images/deleteahvcloudaccount4.png
  1. The AHV Cloud Account will disappear from the Cloud Account list after successful deactivation.

  2. If there are no other Cloud Accounts created under the Organization, then the Organization can be removed by clicking the ellipsis (3 vertical dots) and clicking “Update” and then click “Terminate” from the menu bar at the top. (If there are any Accounts still associated with this Organization, then the option to Terminate will not be available. Click “Terminate” to remove the Organization.

../_images/deleteahvcloudaccount5.png
  1. Now, access Prism Central of the associated AHV cluster and delete the Cloud Connector Appliance (CCA) VMs and Workload CCA VMs.

  2. Under Images, delete any images which were added for Cloud Connector Appliance (CCA).

Updating Cloud Account for Network configuration, Instance types, Template image

After the initial setup is done, you can update your AHV Cloud Account from the Frame portal to add or update the following:

  • VLANs for Frame-managed VMs

  • Template images

  • Instance types

  1. You start by accessing the AHV Cloud Account associated with the Organization or Customer entity.

    ../_images/ahvupdate1.png
  2. Next click Update on the Cloud Account which needs to be updated

    ../_images/ahvupdate2.png
  3. You can now add additional Virtual Networks (VLANs), template images, and instance types or update/delete any existing configurations.

    ../_images/ahvupdate3.png

Updating Prism Central/Prism Element Credentials

To update Prism Central or Prism Element credentials, launch the CCA dashboard using CCA’s IP address. Click Profile in the top right menu and launch the update password wizard. The CCA wizard allows you to change either Prism Central or Prism Element username and password. You can also update both usernames/passwords at the same time.

../_images/ahvupdate4.png

Troubleshooting

To troubleshoot CCA issues, admins can review logs or access network configuration via AHV’s VNC console. If the CCA page does not display the “Prism Central URL” field, verify the following:

  • Ensure that there is a 0.1 GB attached disk on the CCA.

  • Ensure that a “volume group” is not attached to the VM.

  • CCA 3.0 Dashboard can provide details on connection between CCA and Frame or CCA and Prism Central. It also shows the latest API requests to Prism Central and a summary of the Prism Central API calls since the CCA 3.0 VM was last powered on. This can be accessed by going to https://<CCA VM IP address>/ and logging in using your Prism Central service username/password.

    ../_images/ahv30dashboard.png

Supported vGPUs

The table below outlines which vGPUs Frame on AHV supports.

Supported vGPUs

NVIDIA GPU Name

P100

P40

P4

T4

M60

M10

M6

P6

GPUs/Board (Architecture)

1 (Pascal)

1 (Pascal)

1 (Pascal)

1 (Turing)

2 (Maxwell)

4 (Maxwell)

1 (Maxwell)

1 (Pascal)

CUDA Cores

3,584

3,840

2,560

2,560

4,096 (2,048 per GPU)

2,560 (640 per GPU)

1,536

2,048

Memory Size

16 GB HBM2

24 GB GDDR5

8 GB GDDR5

16 GB GDDR5

16 GB GDDR5 (8 GB per GPU)

32 GB GDDR5 (8 GB per GPU)

8 GB GDDR5

16 GB GDDR5

vGPU Profiles (GB)

1, 2, 4, 8, 16

1, 2, 3, 4, 6, 8, 12, 24

1, 2, 4, 8

1, 2, 4, 8, 16

0.5, 1, 2, 4, 8

0.5, 1, 2, 4, 8

0.5, 1, 2, 4, 8

1, 2, 4, 8, 16