From ffe1b7670146fbd1cfbc8371d90a0f0af7c9faa7 Mon Sep 17 00:00:00 2001
From: Petr Šplíchal <psplicha@redhat.com>
Date: Jul 25 2019 07:56:42 +0000
Subject: Merge #19 `Define testsets, gates and isolate`


---

diff --git a/l2/README.md b/l2/README.md
index ad548e8..2951d56 100644
--- a/l2/README.md
+++ b/l2/README.md
@@ -27,6 +27,13 @@ Specifies the artifact to be tested. Currently defined artifacts:
 Based on the artifact it is possible to detect some environment
 constraints such as architecture or base compose.
 
+
+## Testsets
+
+For each artifact it is possible to define one or more testsets.
+Each testset represents group of tests with a unique identifier.
+The `summary` attribute can be used to describe testset purpose.
+
     /test:
         /pull-request:
             /pep:
@@ -40,6 +47,50 @@ constraints such as architecture or base compose.
                 summary: Verify important features
 
 
+## Gates
+
+Multiple gates can be defined in the process of releasing a change.
+Currently we define the following gates:
+
+* merge-pull-request ... block merging a pull request into a git branch
+* add-build-to-update ... attaching a build to an erratum / bodhi update
+* add-build-to-compose ... block adding a build to a compose
+* release-update ... block releasing an erratum / bodhi update
+
+Each testset can define one or more gates it should be blocking.
+Here is an example of configuring multiple gates
+
+    /test:
+        /pull-request:
+            /pep:
+                summary: All code must comply with the PEP8 style guide
+                # Do not allow ugly code to be merged into master
+                gate:
+                    - merge-pull-request
+            /lint:
+                summary: Run pylint to catch common problems (no gating)
+        /build:
+            /smoke:
+                summary: Basic smoke test (Tier1)
+                # Basic smoke test is used by three gates
+                gate:
+                    - merge-pull-request
+                    - add-build-to-update
+                    - add-build-to-compose
+            /features:
+                summary: Verify important features
+        /update:
+            # This enables the 'release-update' gate for all three testsets
+            gate:
+                - release-update
+            /basic:
+                summary: Run all Tier1, Tier2 and Tier3 tests
+            /security:
+                summary: Security tests (extra job to get quick results)
+            /integration:
+                summary: Integration tests with related components
+
+
 ## Hierarchy
 
 There are several levels of test execution data configuration:
@@ -157,6 +208,9 @@ Example config:
 
     execute:
         how: restraint
+        isolate: true
+
+Optional boolean attribute `isolate` can be used to request a clean test environment for each test.
 
 ### Report