Jump to content
Welcome to our new Citrix community!
  • 0

Deploy VMs using Ansible


Bojan Vitnik1709158530

Question

Hi guys,

 

Sorry if this looks like shameless self promotion but for a few weeks now I've been working on Ansible module for managing XenServer VMs - first of it's kind if I'm not mistaken. The module is in line with what Ansible offers for VMWare and there is a pull request to mainline it. What I would love is for anyone willing to help, and already familiar with Ansible, is to spare some time to test the module since the only way to test it is against real hardware. Of course, the module should not be tested in production environments.

 

UPDATE: 2019-03-01

 

Some of my Ansible modules have been merged upstream and are currently available in development version of Ansible. Ansible 2.8 will be the first official release to include these modules.

 

My work is currently involved around these three modules:

 

xenserver_guest - upstreamed - used for deployment of new VMs from templates and reconfiguration of existing VMs.

xenserver_guest_facts - upstreamed - used for getting VM facts (useful XenServer VM params).

xenserver_guest_powerstate - upstreamed - used for controlling VM power state (running/halted/suspended), graceful shutdown and reboot etc.

 

For anyone interested in using/testing these modules, upstreamed modules can be acquired by cloning Ansible repo from GitHub (or downloading the ZIP archive) and running Ansible from source. Instructions can be found here:

 

https://docs.ansible.com/ansible/devel/installation_guide/intro_installation.html#running-from-source

 

To use the modules, you will also need XenAPI.py from here:

 

https://raw.githubusercontent.com/xapi-project/xen-api/master/scripts/examples/python/XenAPI.py

 

Copy the file to your Python site-packages (e.g. /usr/lib/python2.7/site-packages/ on CentOS 7).

 

You can get module documentation by running this command:

$ ansible-doc <module_name>

or from official Ansible docs (upstreamed modules only):

 

CHANGELOG:

  • Fixed a bug on XenServer 7.1 with Cumulative Update where a version could not be properly detected, causing an exception.

 

Thanks.

 

Link to comment

Recommended Posts

  • 0

Bojan,

 

I was testing this out on XenServer 7.1 CU2 and I was getting some errors.

Not sure if its a problem with my version on XenAPI (installed via pip).

I was running everything in a python virtual environment with ansible dev 2.8


 

The full traceback is:
Traceback (most recent call last):
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py", line 113, in <module>
    _ansiballz_main()
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py", line 105, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py", line 48, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py", line 170, in load_source
    module = _exec(spec, sys.modules[name])
  File "<frozen importlib._bootstrap>", line 618, in _exec
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/ansible_xenserver_guest_payload__55mxcr6/__main__.py", line 416, in <module>
  File "/home/johntho-dev/ansible-dev/lib/python3.6/site-packages/XenAPI.py", line 64
    except Exception, exn:
                    ^
SyntaxError: invalid syntax

fatal: [10.8.47.11 -> localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549491729.9147422-10901829864249/AnsiballZ_xenserver_guest.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py\", line 235, in load_module\n    return load_source(name, filename, file)\n  File \"/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py\", line 170, in load_source\n    module = _exec(spec, sys.modules[name])\n  File \"<frozen importlib._bootstrap>\", line 618, in _exec\n  File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/tmp/ansible_xenserver_guest_payload__55mxcr6/__main__.py\", line 416, in <module>\n  File \"/home/johntho-dev/ansible-dev/lib/python3.6/site-packages/XenAPI.py\", line 64\n    except Exception, exn:\n                    ^\nSyntaxError: invalid syntax\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

 

Let me know if you are still checking this thread. Thanks.

Link to comment
  • 0

Hi johntho. Version of XenAPI.py from pypi (pip) is just too old. It's not compatible with Python 3.x. As far as I can tell, pip package is not updated since 2011. You should use XenAPI.py either from XenServer SDK or latest version from GitHub:

 

https://raw.githubusercontent.com/xapi-project/xen-api/master/scripts/examples/python/XenAPI.py

 

Just copy the file you your site-packages:

 

/home/johntho-dev/ansible-dev/lib/python3.6/site-packages/

 

I guess I should really try to contact Citrix and ask if they could update the pip package. There is an unofficial pip package called "xenapi-python" but it's imported differently (documentation for it is wrong/missleading) so it's not direct replacement for XenAPI package. I'll consider supporting both packages in the future but for now copy the XenAPI.py file manually to your site-packages.

Link to comment
  • 0

Any idea why it would be hanging at this step?
Where can I look for logs?

 

TASK [Create a VM from a template] **********************************************************************************************************************************
task path: /home/johntho-dev/ansible-dev/create-xen-vm.yml:4
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: johntho-dev
<localhost> EXEC /bin/sh -c 'echo ~johntho-dev && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847 `" && echo ansible-tmp-1549635006.9388394-253983018632847="` echo /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847 `" ) && sleep 0'
Using module file /home/johntho-dev/ansible-dev/lib/python3.6/site-packages/ansible/modules/cloud/xenserver/xenserver_guest.py
<localhost> PUT /home/johntho-dev/.ansible/tmp/ansible-local-27238s36du_e/tmpscg1czgz TO /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/ /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py && sleep 0'
<localhost> EXEC /bin/sh -c '/home/johntho-dev/ansible-dev/bin/python /home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py && sleep 0'
  
  
  The full traceback is:
Traceback (most recent call last):
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py", line 113, in <module>
    _ansiballz_main()
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py", line 105, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py", line 48, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py", line 170, in load_source
    module = _exec(spec, sys.modules[name])
  File "<frozen importlib._bootstrap>", line 618, in _exec
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py", line 1901, in <module>
  File "/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py", line 1886, in main
  File "/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py", line 556, in deploy
  File "/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py", line 457, in gather_params
  File "/tmp/ansible_xenserver_guest_payload_iuf4g3ld/ansible_xenserver_guest_payload.zip/ansible/module_utils/xenserver.py", line 411, in gather_vm_params
ValueError: invalid literal for int() with base 10: '1 CU2'

fatal: [10.8.47.11 -> localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/johntho-dev/.ansible/tmp/ansible-tmp-1549635006.9388394-253983018632847/AnsiballZ_xenserver_guest.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py\", line 235, in load_module\n    return load_source(name, filename, file)\n  File \"/home/johntho-dev/ansible-dev/lib64/python3.6/imp.py\", line 170, in load_source\n    module = _exec(spec, sys.modules[name])\n  File \"<frozen importlib._bootstrap>\", line 618, in _exec\n  File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py\", line 1901, in <module>\n  File \"/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py\", line 1886, in main\n  File \"/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py\", line 556, in deploy\n  File \"/tmp/ansible_xenserver_guest_payload_iuf4g3ld/__main__.py\", line 457, in gather_params\n  File \"/tmp/ansible_xenserver_guest_payload_iuf4g3ld/ansible_xenserver_guest_payload.zip/ansible/module_utils/xenserver.py\", line 411, in gather_vm_params\nValueError: invalid literal for int() with base 10: '1 CU2'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
        to retry, use: --limit @/home/johntho-dev/ansible-dev/create-xen-vm.retry

 

Link to comment
  • 0

Ah. That would be a bug. Thanks for spotting it. My code expects product_version_text_short to contain only numerics but in case of XS 7.1 with CU there are some extra non numeric characters (CU2). I've only tested 7.1 without any CU.

 

This would have to be patched. Give me some time to think how to resolve this. In the mean time, can you send me the output of this CLI command:

# xe host-param-get uuid=your_host_uuid param-name=software-version

Do you have access to any other version of XS without LTSR? They have no extra characters in version number so the module should work with them.

 

Link to comment
  • 0

No worries.  Its a lab/test environment.  The info you requested is below.

 

Xenserver 7.1 CU2

 

product_version: 7.1.2; product_version_text: 7.1 CU2; product_version_text_short: 7.1 CU2; platform_name: XCP; platform_version: 2.2.0; product_brand: XenServer; build_number: release/havana/master/46; hostname: 0469b939b66a; date: 2018-12-03; dbv: 2017.0214; xapi: 1.9; xen: 4.7.6-1.26; linux: 4.4.0+2; xencenter_min: 2.6; xencenter_max: 2.6; network_backend: openvswitch; db_schema: 5.109

 

Xenserver 7.1 CU1

 

product_version: 7.1.1; product_version_text: 7.1 CU1; product_version_text_short: 7.1 CU1; platform_name: XCP; platform_version: 2.2.0; product_brand: XenServer; build_number: XS71ECU1/150; hostname: 81c2a8c27cbc; date: 2017-08-31; dbv: 2017.0214; xapi: 1.9; xen: 4.7.4-1.14; linux: 4.4.0+2; xencenter_min: 2.6; xencenter_max: 2.6; network_backend: openvswitch; db_schema: 5.109

 

Link to comment
  • 0

Also... officially I think citrix xenserver team only supports their current version and 1 to 2 LTSR versions, if I remember right.

That is if I'm reading this correctly:

https://www.citrix.com/support/product-lifecycle/milestones/citrix-hypervisor.html

 

So it may be a waste of your time to try and get 7.2, 7.3, 7.4, 7.5 to work?

 

Current version is 7.6

Link to comment
  • 0

One more time.  Here is 7.5.  I don't have any 7.6's at the moment.

 

Xenserver 7.5

 

product_version: 7.5.0; product_version_text: 7.5; product_version_text_short: 7.5; platform_name: XCP; platform_version: 2.6.0; product_brand: XenServer; build_number: release/kolkata/master/29; hostname: localhost; date: 2018-07-26; dbv: 2018.0509; xapi: 1.20; xen: 4.7.5-5.6; linux: 4.4.0+10; xencenter_min: 2.10; xencenter_max: 2.10; network_backend: openvswitch; db_schema: 5.142


 

Link to comment
  • 0

Module already works with version 7.2. That's what I primarily use. I expect the module to work with any version starting with 5.6 up to 7.6. There is no much effort to support new versions unless I want to support some newly introduced functionality. I'm using XenAPI calls that havent changed since version 5.6 and changes in XenAPI that break backward compatibility should be extremely rare.

 

Basicaly, 7.1 with CU is black sheep here. I wasn't expecting it to contain non numeric characters in version number. That was my bad assumption. With software_version reports you provided here, I can now see that I should have used product_version instead of product_version_text_short. This will be an easy fix. 

Link to comment
  • 0

Here is a patched version of xenserver module util:

 

https://raw.githubusercontent.com/bvitnik/ansible/xenserver-7-1-cu-bugfix/lib/ansible/module_utils/xenserver.py

 

Put it here:

 

/home/johntho-dev/ansible-dev/lib/python3.6/site-packages/ansible/module_utils

 

and overwrite existing file.

 

The module should now work with XenServer 7.1 with CU.

 

I will try to upstream this bugfix soon.

Link to comment
  • 0

So I tested this once and the new patch worked.  However, it seems to take a very long time to create a machine.

Its been over 7 minutes, and my second run, usually stops here with the -vvv option.

 

TASK [Create a VM from a template] **********************************************************************************************************************************
task path: /home/johntho-dev/ansible-dev/create-xen-vm.yml:4
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: johntho-dev
<localhost> EXEC /bin/sh -c 'echo ~johntho-dev && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539 `" && echo ansible-tmp-1549923962.8106399-30361546781539="` echo /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539 `" ) && sleep 0'
Using module file /home/johntho-dev/ansible-dev/lib/python3.6/site-packages/ansible/modules/cloud/xenserver/xenserver_guest.py
<localhost> PUT /home/johntho-dev/.ansible/tmp/ansible-local-6554tnr9ne7q/tmpwk1sjadi TO /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539/AnsiballZ_xenserver_guest.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539/ /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539/AnsiballZ_xenserver_guest.py && sleep 0'
<localhost> EXEC /bin/sh -c '/home/johntho-dev/ansible-dev/bin/python /home/johntho-dev/.ansible/tmp/ansible-tmp-1549923962.8106399-30361546781539/AnsiballZ_xenserver_guest.py && sleep 0'

 

I assume it will succeed this second time, but, its strange that its waiting this long to finish creating one machine.  Thoughts?

 

UPDATE:

The second time did work, but, it took 15 min. to create the vm.

Maybe I have something wrong in the yaml instructions?

hosts: xenservers



tasks:

- name: Create a VM from a template

xenserver_guest:

hostname: 0.0.0.0 (not showing ip here)

username: root

password: xenroot

validate_certs: no

#folder: /home/testvms

name: CCConn-0002

state: poweredon

template: W2K16_RTM_64_EN_ans

disks:

- size_gb: 100

name: CCConn-0002

sr: MyVol

hardware:

num_cpus: 4

num_cpu_cores_per_socket: 2

memory_mb: 8192

cdrom:

type: iso

iso_name: guest-tools.iso

networks:

- name: vlan40

wait_for_ip_address: no

delegate_to: localhost

register: deploy

 

Link to comment
  • 0

By default, module will make a full copy of any disk contained in template. That's probably why it takes so long. There is an option linked_clone which is equivalent of "Use storage-level fast disk clone" option in XenCenter which should work much faster.

 

Unfortunately, there is no any progress indication in XenCenter but you can use:

# xe task-list

to see the progress. Take a look at task list and notify me if you don't see any progress or if you see that some task is hanged.

 

UPDATE:

 

Your playbook looks good to me. Since your template is Windows 2016 I suspect it has system disk of around 30 GB minimum so it takes some time to copy. If it's 100 GB like specified in playbook, then it's even worse. So a speed of your SR is a limiting factor here. Use linked_clone if supported by SR to make it faster. Also, linked_clone will not work if your template is on one SR and your VM is on another. Disks have to be copied fully in that case.

Link to comment
  • 0

Ok, now for a harder question.

 

Is it possible to somehow give windows templates a new hostname when or after you create them?
If you use a template that is syspred, it always generates a random hostname.

 

If I could somehow change the hostname and let the dns server pickup the new hostname, then ansible could access a large amount of machines after creation.

 

Otherwise I have to manually, change all the hostnames to get them ready for an ansible inventory...

 

Thoughts?

Link to comment
  • 0

Unfortunately, configuration of OS parameters including hostname is outside of responsibility of the module. For a module to change OS parameters there has to be some kind of agent inside VM that module can communicate with through XenAPI. There is an agent for Windows included in XenServer Guest Tools and some XenAPI support but it is limited to configuration of IP, netmask and gatweway only. There is no way to configure hostname.

 

On the other hand, Ansible offers enough functionality to accomplish that without direct support in agent. Here are some examples:

 

1) You start by making an inventory of your soon to be deployed VMs. Inventory should contain hostname and ansible_host=<ip address>. You also need VM template with preinstalled XenServer Guest Tools. Deploy your VMs and use "ip" and "gateway" suboptions of "networks" to set VM IP address. Also use wait_for_ip_address option to wait for VM to setup it's networking. After that, you can use win_hostname module to connect to VM and set a hostname.

 

2) You start by making an inventory of your soon to be deployed VMs. Inventory should only contain hostname. Again, you need  VM template with preinstalled XenServer Guest Tools. Deploy your VMs and put primary network interface to some network/vlan that has DHCP server. Use wait_for_ip_address option to wait for VM to acquire an IP address. The module will return acquired IP address in it's facts (use register: some var). Look up for instance.networks.0.ip key in VM facts. Use that value to dynamically add VM IP to in memory inventory using add_host module. Now you can use win_hostname module to connect to VM and set a hostname.

 

3) Make some script inside VM that will be called on first boot. The script should read VM name from xenstore using WMI interface or possibly some xenstore tools and set hostname accordingly.

 

...

 

I'm sure there are other ways but I think you get the idea.

Link to comment
  • 0

I'm also struggling to find the best way to provision VMs at scale. By now, I've implemented three completely different ways to do that. I can't share them here but I can give you some tips:

 

1) Don't use with_* loop controls. They will be deprecated. Use plain loop.

 

2) For a var containing hierarchy of VMs, I'd use something like this:

    Machines:
      Connectors:
        - hostname: CCConn-0001
          num_cpus: 4
          num_cpu_cores_per_socket: 2
          memory_mb: 8192
        - hostname: CCConn-0002
          num_cpus: 4
          num_cpu_cores_per_socket: 2
          memory_mb: 8192
      Storefronts:
        - hostname: SFPrinci-0001
          num_cpus: 4
          num_cpu_cores_per_socket: 2
          memory_mb: 8192

So I lost one dict level and used lists. Now you can do something like this:

  tasks:
  - name: Create VMs from a template
    xenserver_guest:
      hostname: 10.8.47.11
      username: 
      password: 
      validate_certs: no
      #folder: /home/testvms
      name: '{{ item['hostname'] }}'
      state: poweredon
      template: W2K16_RTM_64_EN_ans
      disks:
      - size_gb: 100
        name: ''
        sr: XenRTVol
      linked_clone: yes
      hardware:
        num_cpus: '{{ item['num_cpus'] }}'
        num_cpu_cores_per_socket: '{{ item['num_cpu_cores_per_socket'] }}'
        memory_mb: '{{ item['memory_mb'] }}'
      cdrom:
        type: iso
        iso_name: guest-tools.iso
      networks:
      - name: vlan40
      wait_for_ip_address: no
    delegate_to: localhost
    register: deploy
    loop: '{{ Machines['Connectors'] + Machines['Storefronts'] }}'

For a simple use case, that should do it.

 

3) Learn how to use jinja filters that manipulate lists and dicts. The most powerful one of them is json_query (you'll have to learn some jmespath to use it efficiently).

 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...