From 0cbafc1b274da1257ce0e83a1d16275b6ceaa32a Mon Sep 17 00:00:00 2001 From: Fabian Affolter Date: Mar 17 2019 19:53:48 +0000 Subject: Add Scripts API docs --- diff --git a/README b/README index 070935b..2182a02 100644 --- a/README +++ b/README @@ -77,7 +77,7 @@ Extending As of 0.4, fedora-review has two plugin interfaces: - * The native python interface used by modules in the src/checks + * The native Python interface used by modules in the src/checks directory. This is the most complete, and is required e. g., when defining a new group of tests for a new language. This is descibed in the project wiki, at @@ -85,8 +85,8 @@ As of 0.4, fedora-review has two plugin interfaces: * The script interface used by scripts in the scripts/ directory. This is a small interface designed to make it possible to write - simple tests in plain bash. This is covered in the Wiki, at - https://fedorahosted.org/FedoraReview/wiki/ScriptsApi + simple tests in plain bash. This is covered at + https://pagure.io/FedoraReview/blob/master/f/SCRIPTS_API The JSON-based interface which existed in earlier versions has been retired. diff --git a/SCRIPTS_API b/SCRIPTS_API new file mode 100644 index 0000000..83f4926 --- /dev/null +++ b/SCRIPTS_API @@ -0,0 +1,311 @@ +fedora-review Scripts API +========================= + +As of 0.3, fedora-review ha a new, experimental mechanism where small +shell scripts can implement tests. This API is utterly unstable. + +This as an easy way to expand the functionality for yourself or for +your community. A useful script can be shorter than 10 lines!. + +The script API is best explained through short tutorial, based +on a real example. + +WARNING! WARNING! This has been written out of the TOP OF MY HEAD, +it has NOT BEEN TESTED and NOT BEEN REVIEWED. + +Preparations +------------ + +For this tutorial to work, you should remove the existing script +scripts/check-bundled-jars.sh. + +Creating a manual test +---------------------- + +Create a file check-bundled-jars.sh like the following. Make it executable and +store it in ~/.local/share/fedora-review/scripts + + #!/bin/bash + # @text: Bundled jar/class files should be removed before build. + exit $FR_PENDING + +Then run fedora-review. + + $ ./try-fedora-review -b 787713 + +You should see a new line in the report like: + + [ ] Bundled jar/class files should be removed before build. + +If you test a java package instead, this line will be repeated twice. We'll +handle that later. + +Only running in Java packages +----------------------------- +So far, so good. But we don't want this line in the report unless the package +is a Java package. +So, lets try next version: + + #!/bin/bash + # @text: Bundled jar/class files should be removed before build. + # @group: Java + exit $FR_PENDING + +Adding the @group tag means that the script only will be invoked when +checking a Java package. + +The list of available groups can be obtained using + + $ ./try-fedora-review -d | grep Group: + +Note that definition of what constitutes package of any group is unstable. +For example java packages have to contain "*.jar" or "*.pom" for the Java +check to be run. If you want to decide if the check is applicable yourself, +you can do it by setting group to Generic and checking applicability yourself. +If your check is not applicable you can call `exit $FR_NOT_APPLICABLE` to +omit check from review output. + +Now, try using fedora-review on a Java and non-Java package. Since you +already have built the FreeSOLID package, use shortcuts to avoid rebuilding: + + $ ./try-fedora-review --no-build --bug 787713 + $ ./try-fedora-review --bug 805487 + +You should see this line twice in the Java package 805487-logback, but not +at all in the 787713-FreeSOLID review. + +Deprecating the generic test +---------------------------- + +Of two lines when testing a Java package one comes from the script, and one +from the test in the Python code. Basically, we want to override this test. +First find out the name of the test by grepping for the first part of the +text in the -d output: + +$ ./try-fedora-review -d | grep 'If source tarball includes bundled' + CheckBundledJars: If source tarball includes bundled jar/class + +So, the name of the test is CheckBundledJars. Then we get our new version: + + +#!/bin/bash +# @group: Java +# @deprecates: CheckBundledJars +# @text: Bundled jar/class files should be removed before build. + +exit $FR_PENDING + +Run fedora-review on the java package, avoiding rebuild: + + $ ./try-fedora-review --no-build --bug 805487 + +If you look into the report in 805487-logback directory, you should see +this line only once. You can also see that the deprecation is in effect: + + $ ./try-fedora-review -d | grep -A 1 Deprecations + Deprecations: check-bundled-jars.sh: check-bundled-jars.sh + CheckBundledJars + + +Automating the test +------------------- +Now, lets try to run this automatically instead. To do this you need to +understand the environment: + +- The review dir, which contains a lot of info at fixed paths. See the + manpage for this. +- The file review-env.sh, which fedora-review creates before running a + script. By looking into this file, you know which environment variables + and functions fedora-review has setup. + +After reading the manpage and review-env.sh, we write the following: + + #!/bin/bash + # @group: Java + # @deprecates: CheckBundledJars + # @url:http://fedoraproject.org/wiki/Packaging:Java#Pre-built_JAR_files + # @text: Bundled jar/class files should be removed before build. + + cd BUILD/* + jars=$( find . -name \*.jar -o -name \*.class) + + test -z "$jars" && exit $FR_PASS + echo "Jar/class files in source: $jars" + exit $FR_FAIL + +here, we access the sources under BUILD and checks for .jar or .class files. +Fedora-review guarantees that the sources are available under a single topdir +in BUILD. We have also added a @url: tag which is used when the test fails: +Depending on the outcome, we will get: + + $FR_PASS: [x] Bundled jar/class files should be removed... + $FR_FAIL: [!] Bundled jar/class files should be removed... + Note: Jar/class files in source: + /a/long/path/ex1.jar + /a/long/path/ex2.jar + See: http://fedoraproject.org/wiki/Package... + +Handling pitfalls +----------------- + +There are always pitfalls. When accessing sources, we cannot use the sources +under BUILD if user have not installed mock (and thus uses --prebuilt). In +this case mock havn't run and there is nothing under BUILD. We could handle +this is several ways: + +- By looking at ${FR_SETTINGS!['prebuilt']} to find out if this setting is + active. +- To try to unpack the upstream sources using the unpack_sources function, which + puts sources under upstream-unpacked (but fails without mock). +- Or, just leave this to the user. + +Since we want it simple, we stick to the third alternative: + + + #!/bin/bash + # @group: Java + # @deprecates: CheckBundledJars + # @type: MUST + # @url:http://fedoraproject.org/wiki/Packaging:Java#Pre-built_JAR_files + # @text: Bundled jar/class files should be removed before build. + + if [ ! -d BUILD/${FR_NAME}-* ]; then + echo "Cannot find sources under BUILD (mock command not available?)" + exit $FR_PENDING + fi + + cd BUILD/${FR_NAME}-* + jars=$( find . -name \*.jar -o -name \*.class) + + test -z "$jars" && exit $FR_PASS + echo "Jar/class files in source: $jars" + exit $FR_FAIL + +It will write snippet below if it fails to find the sources: + + [ ] Bundled jar/class files should be removed + Note: Cannot find sources under BUILD (mock command not available?). + +Reference +--------- + +This is a very small part of the current interface which should be reasonably +stable. The complete story is the review-env.sh file! + +stdout, stderr, exit codes +-------------------------- + +Any message written to stdout is appended to the output prefixed with 'Note:' +(see examples above). + +Any message written to stderr marks the test as failed, dumping stdout+stderr +to the test output. + +Test should return one of the four legal exitcodes: + FR_PASS:: test passed i. e., a '[x]' prefixed message. + FR_FAIL:: test failed i. e., a '[!]' prefixed message. + FR_PENDING:: manual test i. e., a '[ ]' prefixed message. + FR_NOT_APPLICABLE:: Nothing printed in report. + +Any other exit code marks test as failed, handled as stderr output. + +Tags +---- + + @text:: The text printed in the report for this text. Long texts could be + split into multiple @text tags which are concatenated in the report. + + @group:: Which kind of packages this test should be applied to. Setting + @group to 'Generic' means that test is applied to all packages. Groups + are listed using './try-fedora-review -d | grep Group:'. The exact definition + of what goes into a group is defined in a python plugin, see the "Write a + language plugin" page. + + @deprecates:: List of tests which should not run. + + @type:: 'MUST'|'SHOULD'|EXTRA', defaults to 'MUST'. The header test in printed + under. + + @name:: Can be used to give the test a name. By default, the + filename (basename) is used as name. + + @url:: Printed with a 'See: ' prefix if test fails. + +Variables +--------- + + FR_NAME:: The Name: tag. + FR_VERSION:: The Version: tag. + FR_URL:: The Url: tag. + +Functions +--------- + + unpack_rpms:: + unpack all generated rpms under the directory rpms-unpacked. + Might fail in some situations. Typical usage: + + if unpack_rpms; then + cd rpms-unpacked + for rpm in *; do + # do something + done + test 'something' && exit $FR_PENDING + exit $FR_FAIL + else + exit $FR_PENDING + fi + + attach :: + Creates an attachment with title and sort key + from stdin. The sort_key is an integer 1 <= key <= 10 which governs where + the attachment should br printed; lowest goest first. Typical usage: + + echo "$jars" | attach 8 "Jar and class files in source" + +Testing +------- +The simple way to test a script after editing it: + + $ cd <review-directory> + $ bash + $ source review_env.sh + $ source /path/to/script + +The script must be sourced, otherwise arrays like FR_SETTINGS are unavailable, +bash doesn't export arrays. + +Accessing Bash Arrays +--------------------- + +Some values in review-env.sh are stored in bash associative arrays, which maps +rather close to a python (java, perl...) hash. One example is FR_FILES, +which reflects the %files sections. A quick crash course: + +In review-env.sh we have expressions like: + + declare -A FR_FILES + FR_FILES[%files examples]="%doc LICENSE.txt + %{_datadir}/%{name}-%{version} + " + ... + + +All the keys in FR_FILES i. e., all %files sections: + + $ for files in "${!FR_FILES[@]}"; do echo $files; done + %files devel + %files javadoc + %files examples + %files + +The content of a section: + + $ echo ${FR_FILES['%files javadoc']} + %doc LICENSE.txt %{_javadocdir}/%{name} + +The content of all sections: + + $ echo ${FR_FILES[@]} + %{_javadocdir}/%{name} %doc LICENSE.txt %{_datadir}/%{name}-%{version} %doc LICENSE.txt README.txt docs/* %{_javadir}/%{name} %{_mavendepmapfragdir}/%{name} %{_mavenpomdir}/*.pom +