From 154e9a3210753def95233e4adc8bbc371172ac45 Mon Sep 17 00:00:00 2001 From: Dan Callaghan Date: Apr 09 2018 04:34:12 +0000 Subject: [PATCH 1/5] docs: remove STOMP messages from roadmap It's implemented now. --- diff --git a/docs/todo.rst b/docs/todo.rst index aaf36d8..fc003c4 100644 --- a/docs/todo.rst +++ b/docs/todo.rst @@ -4,12 +4,6 @@ Roadmap This document describes some plans for the future of Greenwave. Items are listed in approximate order of importance. -Support for STOMP messaging ---------------------------- - -Like WaiverDB, we want to use Greenwave in environments which have -a STOMP-compatible message bus (#107). - Per-package policies -------------------- From aa21f23cb80a1736d57ee3cf0d158a2792206f61 Mon Sep 17 00:00:00 2001 From: Dan Callaghan Date: Apr 10 2018 07:21:05 +0000 Subject: [PATCH 2/5] docs: replace Contribution Guidelines with Development Guide I am much more interested in helping contributors get started developing Greenwave, and less in prescribing what they must do if they want to contribute. This patch changes the Contribution Guidelines doc to a Development Guide instead, and updates its advice to match the current state of Greenwave. The sections are ordered from most important to least important, for someone who wants to write a patch. This also matches the approach we have in the WaiverDB docs, which already has a Development Guide. --- diff --git a/docs/contributing.rst b/docs/contributing.rst deleted file mode 100644 index 4fc92f0..0000000 --- a/docs/contributing.rst +++ /dev/null @@ -1,45 +0,0 @@ -======================= -Contribution Guidelines -======================= - -Please follow the following contribution guidelines when contributing a pull -request. - - -Code style -========== - -We follow the `PEP 8`_ style guide for Python. The test suite includes a test -that enforces the required style, so all you need to do is run the tests to -ensure your code follows the style. If the unit test passes, you are good to go! - - -Unit tests -========== - -.. automodule:: greenwave.tests - - -Documentation -============= -Greenwave uses `sphinx `_ to create its documentation. -New packages, modules, classes, methods, functions, and attributes all should be -documented using `"Google style" `_ -docstrings. - -Python API documentation is automatically generated from the code using Sphinx's -`autodoc `_ extension. -HTTP REST API documentation is automatically generated from the code using the -`httpdomain `_ extension. - - -Development Environment -======================= - -Set up a Python virtual environment and then install Greenwave:: - - $ pip install -r dev-requirements.txt - $ pip install -e . - - -.. _PEP 8: https://www.python.org/dev/peps/pep-0008/ diff --git a/docs/dev-guide.rst b/docs/dev-guide.rst new file mode 100644 index 0000000..8b6a4d8 --- /dev/null +++ b/docs/dev-guide.rst @@ -0,0 +1,89 @@ +================= +Development Guide +================= + +If you would like to write a patch for Greenwave, this document will help you +get started. + + +Quick development setup +======================= + +Install dependencies: + +.. code-block:: console + + $ sudo dnf builddep greenwave.spec + +Create a local configuration file: + +.. code-block:: console + + $ cp conf/settings.py.example conf/settings.py + +Run the server: + +.. code-block:: console + + $ DEV=true python run-dev-server.py + +The server is now running at and API calls can be sent to +. + + +Running the tests +================= + +You can run the unit tests, which live in the ``greenwave.tests`` package, with +the following command: + +.. code-block:: console + + $ TEST=true py.test greenwave/tests/ + +There are also functional tests in the :file:`functional-tests` directory. The +functional tests will start their own copy of the `ResultsDB`_, `WaiverDB`_, +and Greenwave servers and then send HTTP requests to them. You can run the +functional tests like this: + +.. code-block:: console + + $ TEST=true PYTHONPATH=. py.test functional-tests/ + +The functional tests assume you have ResultsDB and WaiverDB git checkouts in +:file:`../resultsdb` and :file:`../waiverdb` respectively. You can tell it to +find them in a different location by passing ``RESULTSDB`` or ``WAIVERDB`` +environment variables. + + +Building the documentation +========================== + +The documentation is built using `Sphinx`_. If you've made changes to the +documentation, you can build it locally and view it in your browser: + +.. code-block:: console + + $ cd docs + $ make html + $ firefox _build/html/index.html + + +Code style +========== + +We follow the `PEP 8`_ style guide for Python. You can check your code's style +using flake8: + +.. code-block:: console + + $ flake8 + +Additionally, we follow the `"Google style" for docstrings +`_. + + +.. _PEP 8: https://www.python.org/dev/peps/pep-0008/ +.. _ResultsDB: https://pagure.io/taskotron/resultsdb +.. _WaiverDB: https://pagure.io/waiverdb +.. _Sphinx: http://www.sphinx-doc.org/ diff --git a/docs/index.rst b/docs/index.rst index 7ced87e..c239755 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -16,7 +16,7 @@ gating points in a software delivery pipeline, based on test results stored in :caption: Contents: api - contributing + dev-guide release-notes todo From 72b34dd5b1a791f5daff86b09af4c846cec33ea2 Mon Sep 17 00:00:00 2001 From: Dan Callaghan Date: Apr 10 2018 07:25:04 +0000 Subject: [PATCH 3/5] trim down README to remove redundant info Most of this was redundant with the Development Guide in the Sphinx docs. This keeps the README short and tight, and it avoids having the same development instructions in two places which then inevitably get out of sync. --- diff --git a/README.md b/README.md index 751191d..c3c476c 100644 --- a/README.md +++ b/README.md @@ -2,72 +2,18 @@ ![logo of Greenwave](https://pagure.io/greenwave/raw/master/f/logo.png) -## What is Greenwave - Greenwave is a service to decide whether a software artifact can pass certain gating points in a software delivery pipeline, based on test results stored in [ResultsDB](https://pagure.io/taskotron/resultsdb) and waivers stored in [WaiverDB](https://pagure.io/waiverdb). -## Quick development setup - -Set up a python virtualenv: - - $ sudo dnf install python-virtualenv - $ virtualenv env_greenwave - $ source env_greenwave/bin/activate - $ pip install -r requirements.txt - $ pip install -r dev-requirements.txt - -Install the project: - - $ python setup.py develop - -Run the server: - - $ DEV=true python run-dev-server.py - -The server is now running at and API calls can be sent to -. - -## Adjusting configuration - -You can configure this app by copying `conf/settings.py.example` into -`conf/setting.py` and adjusting values as you see fit. It overrides default -values in `greenwave/config.py`. - -## Running test suite - -You can run the unit tests, which live in the `greenwave.tests` package, with -the following command: - - $ TEST=true py.test greenwave/tests/ - -To test against all supported versions of Python, you can use tox:: - - $ sudo dnf install python3-tox - $ tox - -There are also functional tests in the `functional-tests` directory. -The functional tests will start their own copy of the -[ResultsDB](https://pagure.io/taskotron/resultsdb), -[WaiverDB](https://pagure.io/waiverdb), and Greenwave applications and then -send HTTP requests to them. You can run the functional tests like this: - - $ TEST=true PYTHONPATH=. py.test functional-tests/ - -The functional tests assume you have ResultsDB and WaiverDB git checkouts in -`../resultsdb` and `../waiverdb` respectively. You can tell it to find them in -a different location by passing `RESULTSDB` or `WAIVERDB` environment -variables. - -## Building the docs +## Documentation -You can view the docs locally with:: +The latest version of Greenwave's documentation is available at: +https://docs.pagure.org/greenwave/ - $ cd docs - $ make html - $ firefox _build/html/index.html +To get started developing Greenwave, check out the +[development guide](docs/dev-guide.rst). ## Copyright and license From 909c5ea44213f62a765ef72abd8a7d4adfac2b20 Mon Sep 17 00:00:00 2001 From: Dan Callaghan Date: Apr 10 2018 07:25:57 +0000 Subject: [PATCH 4/5] docs: describe how to write policies Fixes #150. --- diff --git a/docs/index.rst b/docs/index.rst index c239755..6fa7d49 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -15,6 +15,7 @@ gating points in a software delivery pipeline, based on test results stored in :maxdepth: 2 :caption: Contents: + policies api dev-guide release-notes diff --git a/docs/policies.rst b/docs/policies.rst new file mode 100644 index 0000000..bc46071 --- /dev/null +++ b/docs/policies.rst @@ -0,0 +1,96 @@ +================ +Writing Policies +================ + +When you ask Greenwave for a decision, it checks all the configured policies +to find which ones are applicable to the subject of the decision. It then +evaluates all the rules in each applicable policy and makes a decision based +on whether they are *all* satisfied. + +Policies are YAML files, loaded from the directory given by the +``POLICIES_DIR`` configuration setting (by default, +:file:`/etc/greenwave/policies`). + +The YAML format allows you to write one or more "documents" in a file. +Greenwave considers each YAML document to be a policy. + +Here is an example policy: + +.. code-block:: yaml + :linenos: + + --- !Policy + id: taskotron_release_critical_tasks + decision_context: bodhi_update_push_stable + product_versions: + - fedora-26 + - fedora-27 + rules: + - !PassingTestCaseRule {test_case_name: dist.rpmdeplint} + - !PassingTestCaseRule {test_case_name: dist.upgradepath} + blacklist: + - qt + - mariadb + +On line 1, the ``---`` YAML document header marks the beginning of a new +document. + +The top-level document has the YAML tag ``!Policy`` to indicate that this is a +Greenwave policy. Greenwave expects each YAML document to be tagged this way. + +The document is a map (dictionary) with the following keys: + +``id`` + This is an arbitrary string identifying this policy. Each policy in the + configuration must have a distinct id. Greenwave does not assign any + meaning to this identifier, but it appears in Greenwave's decision API + responses so that you can map it back to the configuration where it is + defined. + +``decision_context`` + This is an arbitrary string identifying the "context" of the decisions + where this policy is applicable. In other words, if Greenwave is making + decisions at gating points in a pipeline, this is how we identify which + gate we are talking about. + + Greenwave does not enforce anything about this identifier. It should be + chosen in coordination with the tool asking Greenwave for a decision. In + this example, the identifier is ``bodhi_update_push_stable``. `Bodhi`_ + passes this value when it asks Greenwave to decide whether a Bodhi update + is ready to be pushed to the stable repositories. + +``product_versions`` + A policy applies to one or more "product versions". When you ask Greenwave + for a decision, you must tell it which product version you are working + with, and it only selects policies which are applicable for that product + version. + + This mechanism makes it possible to enforce different rules across + different versions of a product. For example, the policy for Fedora could + become increasingly stricter across versions as the quality and coverage of + tests improves. + + The "product version" strings used here (and in the Greenwave decision API) + are expected to match the product version identifiers used in `Product + Definition Center`_ (see the `/product-versions + `_ endpoint), + although Greenwave does not enforce this. + +``rules`` + A list of rules which this policy enforces. Each item in the list is a YAML + map, tagged with the rule type. + + Currently only one rule type is defined, ``PassingTestCaseRule``. The + ``test_case_name`` key in the map identifies the name of the test case. For + this rule to be satisfied, there must be a result in ResultsDB for the + given test case with an outcome of ``PASS``, *or* there must be a + corresponding waiver in WaiverDB for the given test case. + +``blacklist`` + A list of binary RPM package names which are exempted from this policy. + + The blacklist only takes effect when Greenwave is making a decision about + subjects with ``"item": "koji_build"``. + +.. _Bodhi: https://github.com/fedora-infra/bodhi +.. _Product Definition Center: https://github.com/product-definition-center/product-definition-center \ No newline at end of file From 8e167260df32cb24489960d221d22201a0fe9655 Mon Sep 17 00:00:00 2001 From: Dan Callaghan Date: Apr 11 2018 05:56:36 +0000 Subject: [PATCH 5/5] docs: add instructions for how to test your policy changes Fixes #150. --- diff --git a/docs/policies.rst b/docs/policies.rst index bc46071..b815f4b 100644 --- a/docs/policies.rst +++ b/docs/policies.rst @@ -93,4 +93,30 @@ The document is a map (dictionary) with the following keys: subjects with ``"item": "koji_build"``. .. _Bodhi: https://github.com/fedora-infra/bodhi -.. _Product Definition Center: https://github.com/product-definition-center/product-definition-center \ No newline at end of file +.. _Product Definition Center: https://github.com/product-definition-center/product-definition-center + + +Testing your policy changes +=========================== + +If you're writing a new policy, you can use the Greenwave dev server to try it +out and experiment with how if affects Greenwave's decisions. + +First, follow the steps in the :doc:`dev-guide` to get the dev server running +locally. + +Then, add your new or modified policy in the :file:`conf/policies/` directory +of your source tree. Note that Greenwave currently loads policies once at +startup, it doesn't reload them at runtime. Therefore you should restart the +dev server whenever you make a change to the policies. + +Now, you can use :program:`curl` or your favourite HTTP client to ask +Greenwave for a decision: + +.. code-block:: console + + $ curl http://localhost:5005/api/v1.0/decision \ + --header 'Content-Type: application/json' \ + --data '{"product_version": "fedora-27", + "decision_context": "bodhi_update_push_stable", + "subject": [{"item": "akonadi-calendar-tools-17.12.1-1.fc27", "type": "koji_build"}]}'