Standard Test Interface Roles

This repository contains the shared Ansible roles for the Ansible based Standard Test Interface as described at InvokingTests

Style suggestions, conceptual approaches

• Ansible YAML is not imperative. It does not focuse on describing how a program operates. It's a declarative syntax, "statement of fact". It's far easier to stick any needed logic into scripts, and keep playbooks very matter-of-fact, or declarations-of-state.
• Do all the subject creation by script, before the main testing-related play(s) run.
• Host creation and management logic is kept in inventory and out of playbooks. Playbooks need only declare desired idempotent-state from one task to the next.
• No Ansible inventory population during playbook runs. Banning that will cut down on the number of failure-modes quite a bit.
• There is some caution against using role-dependencies.
• You should use include_role instead of import_role when utilizing roles from this project.

Standard Inventory scripts

The inventory scripts are about taking the subjects produced by a CI system and transforming them into something testable. That includes launching a QCow2 image into a virtual machine, or installing an RPM.

Tests are not required to use these scripts but they provide useful, usable defaults for this task.

Flexible Metadata Format for default provisioner(s)

You can find information for FMF here, and for supported FMF attributes here.

Initialize a new metadata tree

To add desired parameters for default provisioners first of all make sure that your /tests directory has initialized metadata tree. Directory .fmf must be present. If it is not, run next command:

fmf init

Usage

Make sure fmf is initialized as described above. Put in /tests a file with name provision.fmf:

---

standard-inventory-qcow2:
  qemu:
    m: 3G
    net_nic:
      model: e1000
    drive:
    - size: 11000000000
      path: /home/vader
    - size: 2000000000
    usb_drive:
    - path: /tmp
    - size: 2000000000
      path: /tmp

standard-inventory-docker:
  dumb_option: dumb_parameter

Supported parameters

standard-inventory-qcow2

• qemu.m - RAM size in megabytes. Optionally, a suffix of M or G.
• qemu.net_nic.model - Use qemu-system-x86_64 -net nic,model=help for a list of available devices.
• drive - has to contain dict of additional drive(s)
• drive.size - allows to specify additional drive with size in bytes (default size: 2G)
• drive.path - allows to specify additional drive with custom directory path to its backing file Filename for image will be auto-generated. (default path: /tmp)
• usb_drive - has to contain dict of additional usb drive(s)
• usb_drive.size - allows to specify additional usb drive with size in bytes (default size: 1G)
• usb_drive.path - allows to specify additional usb drive with custom directory path to its backing file Filename for image will be auto-generated. (default path: /tmp)

You can open a RFE ticket to extend supported parameters according to https://qemu.weilnetz.de/doc/qemu-doc.html

LOCK_ON_FILE for standard-inventory-qcow2

This inventory script launches a virtual machine via qemu. The VM image should be specified via command line or an environment variable TEST_SUBJECTS.

By default, the virtual machine is killed when the process (your shell) it invoked the script, is gone. This behavior may not be desirable when invoking the inventory script from an Ansible playbook (since Ansible spawns a long hierarchy of processes). For that purpose you can use environment variable LOCK_ON_FILE: the VM will be killed once the file specified by this variable is missing. Let's look at an example:

---
- hosts: localhost
  vars:
    # path to the VM image to use
    vm_image: "./Fedora-Cloud-Base-27-1.6.x86_64.qcow2"
    # inventory name of the VM
    vm_name: "awesome-vm"
    # which python interpreter should be used by ansible (e.g. Fedora doesn't have /usr/bin/python)
    vm_python_interpreter: "/usr/bin/python3"
    # path to the script which will provision the VM
    vm_provisioning_script: /usr/share/ansible/inventory/standard-inventory-qcow2
  tasks:
  - name: create lock file to synchronize on the VM
    tempfile:
      prefix: inventory-cloud
      suffix: .lock
      state: file
    register: tmp_lock_file
  - name: provision the VM
    command: "{{ vm_provisioning_script }} {{ vm_image }}"
    register: vm_provision_data
    environment:
      LOCK_ON_FILE: "{{ tmp_lock_file.path }}"
  - name: prepare inventory data for add_host
    set_fact:
      inventory_data: '{{ (vm_provision_data.stdout | from_json)._meta.hostvars[vm_image] }}'
  - add_host:
      name: "{{ vm_name }}"
      ansible_ssh_common_args: "{{ inventory_data.ansible_ssh_common_args }}"
      ansible_host: "{{ inventory_data.ansible_host }}"
      ansible_ssh_pass: "{{ inventory_data.ansible_ssh_pass }}"
      ansible_port: "{{ inventory_data.ansible_port }}"
      ansible_ssh_private_key_file: "{{ inventory_data.ansible_ssh_private_key_file }}"
      ansible_user: "{{ inventory_data.ansible_user }}"
      ansible_python_interpreter: "{% if vm_python_interpreter != '' %}{{ vm_python_interpreter }}{% else %}/usr/bin/python2{% endif %}"

TEST_EXTRA_SSH_ARGS for standard-inventory-qcow2

If you are working with very old releases, like EL6, you may have to pass in additional SSH arguments to allow newer systems to use older cipher suites. For example:

TEST_EXTRA_SSH_ARGS="-o PubkeyAcceptedKeyTypes=+ssh-rsa" TEST_SUBJECTS=/path/to/el6.qcow2 ...

HOSTALIAS for standard-inventory-qcow2

By default, standard-inventory-qcow2 will use the full path and filename of the image.qcow2 file as the host key in the inventory. If you would rather provide a different name for the host that corresponds to the file, you can use either the --hostalias cli flag or the TEST_HOSTALIASES environment variable. You can also use the special keyword BASENAME if you want the hostalias of the image to use the basename of the image. For example:

TEST_SUBJECTS="/path/to/fedora-34.qcow2 /path/to/f-33.qcow2" TEST_HOSTALIASES="sut1 BASENAME" ....

will result in 2 inventories that look like this:

all:
  children:
    localhost:
      hosts: &id001
        sut1:
          ansible_host: 127.0.0.3
...
all:
  children:
    localhost:
      hosts: &id001
        f-33.qcow2:
          ansible_host: 127.0.0.3
...

If you want all of the hostaliases for all images to be the basename, use --use-basename or TEST_USE_BASENAME=true. NOTE: If you do this, make sure that the basenames of all of the images are unique.

TEST_SUBJECTS="/path/to/fedora-34.qcow2 /path/to/f-33.qcow2" TEST_USE_BASENAME=true ....

will result in 2 inventories that look like this:

all:
  children:
    localhost:
      hosts: &id001
        fedora-34.qcow2:
          ansible_host: 127.0.0.3
...
all:
  children:
    localhost:
      hosts: &id001
        f-33.qcow2:
          ansible_host: 127.0.0.3
...

If the number of hostaliases does not match the number of subjects, you will get an error.

TEST_SSHD_USEDNS_NO

Some EL7 systems have sshd configured with UseDNS yes by default. This will cause terrible performance with ssh and especially with Ansible. You can use --sshd-usedns-no or set TEST_SSHD_USEDNS_NO=True to configure the VM to use UseDNS no instead.

TEST_SKIP_MISSING_DEVICE

If you are using the same provision.fmf on multiple systems, the qemu on some systems may not support all of the specified devices. For example, qemu on many platforms does not support NVMe. By default, the script will issue an error if you attempt to use an unsupported device. Use this flag to skip missing devices, with a warning.