| |
@@ -0,0 +1,92 @@
|
| |
+ :toc:
|
| |
+
|
| |
+ = Test Case Relevancy =
|
| |
+
|
| |
+ == Motivation ==
|
| |
+
|
| |
+ Sometimes a test case is only relevant for specific environment.
|
| |
+ Test Case Relevancy allows to filter irrelevant test cases out.
|
| |
+
|
| |
+ == Implementation ==
|
| |
+
|
| |
+ Test case relevancy is function which takes environment parameters and returns `True`, `False` or a list of environment variables.
|
| |
+
|
| |
+ === Syntax ===
|
| |
+
|
| |
+ Test case relevancy is defined by one or more `condition: decision` rules.
|
| |
+
|
| |
+ Allowed operators are: `+= == != < <= > >= &&+`.
|
| |
+ Anything beyond a `#` sign is considered to be a comment and will be ignored.
|
| |
+
|
| |
+ Everything is case insensitive.
|
| |
+ First matching rule wins (terminates immediately the relevancy evaluation, the rest of the rules will be ignored).
|
| |
+
|
| |
+ === Defaults ===
|
| |
+
|
| |
+ If there is no rule specified, test case relevancy defaults to `True`, that is test case is relevant and should be executed.
|
| |
+
|
| |
+ === Environment ===
|
| |
+
|
| |
+ Acceptable parameters defining the environment are:
|
| |
+
|
| |
+ * product — product name (rhel rhel-5 rhel-5.6 rhdts rhscl)
|
| |
+ * distro — distribution (rhel-6 rhel-6.3 rhscl-1.0.0 f-28)
|
| |
+ * variant — distro variant (Client Desktop Server Workstation)
|
| |
+ * arch — architecture (i386 ppc64 s390x x86_64)
|
| |
+ * collection — software collection (python27 python33 perl516...)
|
| |
+ * component — component to be tested (php, apache, ...)
|
| |
+
|
| |
+ While `distro` is always used to define the operating system the software is supposed to run on, `product` is used to describe the target product subscribed and consumed by the customer (can be a layered product on top of RHEL such as RHSCL, or RHEL itself if the component is included directly in the operating system).
|
| |
+
|
| |
+ Distro comparison operates in two modes:
|
| |
+
|
| |
+ * Major mode — When comparing against a major version such as `distro < rhel-6` other major versions are considered (matches any of `rhel-3 rhel-4 rhel-5`).
|
| |
+ * Minor mode — If minor version is provided as well, for example `distro < rhel-6.3`, comparison is performed only within the given major (matches `rhel-6.0 rhel-6.1 rhel-6.2`).
|
| |
+
|
| |
+ Rules which contain environment parameters which are not known at the time of evaluation will be skipped.
|
| |
+
|
| |
+ === Decision ===
|
| |
+
|
| |
+ The `decision` part of the rule can contain following values:
|
| |
+
|
| |
+ * `True` — test case is relevant for given environment
|
| |
+ * `False` — test case is not relevant for this environment
|
| |
+ * `A=X B=Y C=Z` — test case is relevant for the modified environment
|
| |
+
|
| |
+ The last option above allows to adjust the environment in which the test case is to be executed by providing the list of environment variables which will be passed to the test.
|
| |
+
|
| |
+ The decision value may be omitted.
|
| |
+ In such case `True` is used by default.
|
| |
+ So these two lines define identical relevancy:
|
| |
+
|
| |
+ distro > rhel-7: True
|
| |
+ distro > rhel-7
|
| |
+
|
| |
+ == Examples ==
|
| |
+
|
| |
+ mod_wsgi relevant for RHEL6 and newer:
|
| |
+
|
| |
+ distro < rhel-6: False
|
| |
+
|
| |
+ busybox not available for s390x on RHEL6:
|
| |
+
|
| |
+ arch = s390x && distro = rhel-6: False
|
| |
+
|
| |
+ perl-Config-General not present in the `Client` variant:
|
| |
+
|
| |
+ variant = Client: False
|
| |
+
|
| |
+ Run python unit tests under valgrind on suitable archs only:
|
| |
+
|
| |
+ arch = ia64: PHASES=novalgrind
|
| |
+ arch = s390x && distro < rhel-6: PHASES=novalgrind
|
| |
+
|
| |
+ New component python-ctypes added in rhel-5.8:
|
| |
+
|
| |
+ distro < rhel-5: False
|
| |
+ distro < rhel-5.8: False
|
| |
+
|
| |
+ Exim present solely in RHEL5:
|
| |
+
|
| |
+ distro != rhel-5: False
|
| |
+
|
| |
ph0zzy
commented 5 years ago
add test case relevancy page