Here's an example output of fmf show for the workflow example:

/ci/test/build/smoke
discover: {u'filter': u'tier: 1', u'tool': u'fmf'}
execute: {u'tool': u'restraint'}
prepare: {u'ansible': [u'setup.yml']}
provision: {u'arch': [u'x86_64'], u'memory': u'1 GB'}
report: {u'contact': u'email@address.org'}
summary: Basic smoke test (Tier1)
/ci/test/build/features
discover: {u'filter': u'tag: functional', u'tool': u'fmf'}
execute: {u'tool': u'restraint'}
prepare: {u'ansible': [u'setup.yml']}
provision: {u'arch': [u'x86_64'], u'memory': u'1 GB'}
report: {u'contact': u'email@address.org'}
summary: Verify important features
/ci/test/update/basic
discover: {u'filter': u'tier: 1, 2, 3', u'tool': u'fmf'}
execute: {u'tool': u'restraint'}
prepare: {u'ansible': [u'setup.yml']}
provision: {u'arch': [u'x86_64', u'ppc64', u's390x'], u'memory': u'1 GB'}
report: {u'contact': u'email@address.org'}
summary: Run all Tier1, Tier2 and Tier3 tests
/ci/test/update/integration
discover: {u'filter': u'tag: integration', u'tool': u'fmf'}
execute: {u'tool': u'restraint'}
prepare: {u'ansible': [u'integration.yml']}
provision: {u'arch': [u'x86_64', u'ppc64', u's390x'], u'memory': u'4 GB'}
report: {u'contact': u'email@address.org'}
summary: Integration tests with related components
/ci/test/update/security
discover: {u'filter': u'tag: security', u'tool': u'fmf'}
execute: {u'tool': u'restraint'}
prepare: {u'ansible': [u'setup.yml']}
provision: {u'arch': [u'x86_64', u'ppc64', u's390x'], u'memory': u'1 GB'}
report: {u'contact': u'email@address.org'}
summary: Security tests (extra job to get quick results)

It nicely demonstrates how fmf features like inheritance and
attribute merging can prevent unnecessary data duplication.

+1 - the examples follow the ideas and the structure we discussed, looks good to me.

I'd just like to point out to other readers that these examples are not final - e.g. fields like provision will need a bit more work to support constraints users put on the testing systems these days. I've spent few hours digging into schema Beaker machine filtering provides, and I believe we're going to need a bit more tools than just "memory: 1 GB" :) But that doesn't invalidate the overall structure of the examples.

Bear with me, please - I am aware this is just an example, I just want to make you aware of what's running through my head these days :)

I was checking out a way Beaker handles "machine filtering" - a complex set of XML rules, including conditions (<and>, <or>), and a vast list of dimensions and properties one can check. I also checked how this works in Travis, for example (it doesn't, one size fits all...). Another input into our thinking would be possible backends and what they are able to deliver when asked for details, possibly leading to a small common set of properties based on the most common use cases.

For example, both OpenStack and beaker can provide machines with a different number of CPUs and amounts of RAM. I guess this would be true for many other provisioners. It'd make sense to start with these, and support general minimal/maximal/absolute constraints for these two dimensions, e.g.

  ...
  provision:
    # the box must have at least 2 CPUs and at least 4 GB RAM
    memory:
      min: 4 GB
      # or "max: 8GB", or both of them to specify range, or "equal"...
    cpu:
      min: 2
      # or "max" or ...
  ...
  provision:
    # I'm happy with a box with 4 GB RAM, or with a box with at least 2 CPUs
    any:
      memory:
        min: 4 GB
      cpu:
        min: 2

My idea is that whatever code stands between this configuration and the actual provisioner, translates these rules into something the provisioner understands. When asking beaker for machines, this would become an XML snippet for <hostFilter>, when dealing with OpenStack, the code would pick an instance flavor, when running QEMU, the code would translate these rules into standard-inventory-qcow2 options, when... I'd really like to support a generic way, separated from the actual provisioning backed - the idea of typing "give me a box with at least 4 GB of RAM" in ten different ways for ten different backends my test might be executed on gives me goosebumbs :)

There are many other properties of cpu in Beaker (e.g. one can request machines of specific CPU family or a machine with specific CPU flag) and other objects as well (e.g. disk and its properties) - when in an extensible way, these less frequent properties don't really have be supported from the beginning, focusing on common use cases would be better, leaving door opened for the additions.

@happz I like the any attribute idea :) sounds reasonable to me about proposing min/max values instead of some exact value

@psss +1 to me, let's hit this

Thanks for the review, guys. I've updated the workflow example with memory attribute supporting min and max values. As for the any keyword I'd leave that for some future discussion (more detailed and backed up by tangible use cases). I'd like to keep the initial set of examples as simple as possible. Going to merge.

@psss Sure, any, min, max and so on were just to illustrate a concept.