| |
@@ -9,24 +9,24 @@
|
| |
User submits module build request
|
| |
---------------------------------
|
| |
|
| |
- There is the MBS frontend, which provides a REST API (See `views.py`). A user sends a POST request
|
| |
+ There is the MBS frontend, which provides a REST API (See ``views.py``). A user sends a POST request
|
| |
with JSON describing the module to build. There is mainly the URL to the git repository (called
|
| |
- `scmurl`) and branch name (called `branch`). The `scmurl` points to the git repository containing
|
| |
+ ``scmurl``) and branch name (called ``branch``). The ``scmurl`` points to the git repository containing
|
| |
the modulemd file defining the module.
|
| |
|
| |
- This JSON data is handled by `views.SCMHandler`, which validates the JSON and calls
|
| |
- `utils.submit.submit_module_build_from_scm(...)` method. This goes down to
|
| |
- `submit_module_build(...)`.
|
| |
+ This JSON data is handled by ``views.SCMHandler``, which validates the JSON and calls
|
| |
+ ``utils.submit.submit_module_build_from_scm(...)`` method. This goes down to
|
| |
+ ``submit_module_build(...)``.
|
| |
|
| |
|
| |
Module Stream Expansion (MSE)
|
| |
-----------------------------
|
| |
|
| |
- The first thing done in `submit_module_build(...)` is Module Stream Expansion (MSE).
|
| |
+ The first thing done in ``submit_module_build(...)`` is Module Stream Expansion (MSE).
|
| |
|
| |
The submitted modulemd file might have buildrequires and requires pairs defined in an ambiguous way.
|
| |
- For example the module can buildrequire `platform:[f28, f29]` modules, which means it should
|
| |
- be built against the `f28` and `f29` streams of `platform` module.
|
| |
+ For example the module can buildrequire ``platform:[f28, f29]`` modules, which means it should
|
| |
+ be built against the ``f28`` and ``f29`` streams of ``platform`` module.
|
| |
|
| |
The process of resolving these ambiguous buildrequires and requires is called Module Stream
|
| |
Expansion.
|
| |
@@ -35,31 +35,31 @@
|
| |
Output of this process is the list of multiple modulemd files with all the ambiguous
|
| |
buildrequires/requires resolved.
|
| |
|
| |
- This all happens in `utils.mse.generate_expanded_mmds(...)` method.
|
| |
+ This all happens in ``utils.mse.generate_expanded_mmds(...)`` method.
|
| |
|
| |
At first, this method finds out all the possible buildrequires/requires for the input module.
|
| |
- This is done using `DBResolver` which simply finds out the modules in the MBS database.
|
| |
- In our case, it would list all the `platform:f28` and `platform:f29` modules.
|
| |
+ This is done using ``DBResolver`` which simply finds out the modules in the MBS database.
|
| |
+ In our case, it would list all the ``platform:f28`` and ``platform:f29`` modules.
|
| |
|
| |
- It then uses the `MMDResolver` class to find all the possible combinations of buildrequires/requires
|
| |
+ It then uses the ``MMDResolver`` class to find all the possible combinations of buildrequires/requires
|
| |
against which the input module can be built.
|
| |
|
| |
In our case, it would generate two expanded modulemd files (one for each platform stream) which
|
| |
would be identical to the input modulemd file with only the following exceptions:
|
| |
|
| |
- The buildrequires/requires pairs from the input modulemd files will be replaced by the particular
|
| |
- combination returned by `MMDResolver`
|
| |
- - The `xmd` section of generated modulemd files will contain `buildrequires` list which lists all
|
| |
+ combination returned by ``MMDResolver``
|
| |
+ - The ``xmd`` section of generated modulemd files will contain ``buildrequires`` list which lists all
|
| |
the modules required to build this expanded modulemd file. Requirements are traversed to produce
|
| |
- a transitive list that includes all `requires` of each `buildrequires` entry. This is used later
|
| |
+ a transitive list that includes all ``requires`` of each ``buildrequires`` entry. This is used later
|
| |
by MBS.
|
| |
- The context is computed and filled for each expanded modulemd file. It is based on the
|
| |
- expanded buildrequires and requires pairs. See `models.ModuleBuild.contexts_from_mmd(...)`.
|
| |
+ expanded buildrequires and requires pairs. See ``models.ModuleBuild.contexts_from_mmd(...)``.
|
| |
|
| |
Such expanded modulemd files are then added to database as the next step in
|
| |
- `submit_module_build(...)` and are handled as separate module builds later in MBS.
|
| |
+ ``submit_module_build(...)`` and are handled as separate module builds later in MBS.
|
| |
|
| |
- The `submit_module_build(...)` then moves the module builds to "init" state and sends a message on
|
| |
+ The ``submit_module_build(...)`` then moves the module builds to "init" state and sends a message on
|
| |
the configured message bus.
|
| |
|
| |
|
| |
@@ -67,9 +67,9 @@
|
| |
------------------------------------------------
|
| |
|
| |
When module build is moved to the "init" state, the backend handles that in the
|
| |
- `scheduler.handlers.modules.init(...)` method.
|
| |
+ ``scheduler.handlers.modules.init(...)`` method.
|
| |
|
| |
- This method calls `utils.submit.record_component_builds` which reads the modulemd file
|
| |
+ This method calls ``utils.submit.record_component_builds`` which reads the modulemd file
|
| |
stored in the database by the frontend and records all the components (future RPM packages) in the
|
| |
database.
|
| |
|
| |
@@ -83,16 +83,16 @@
|
| |
------------------------------------------------
|
| |
|
| |
When the module build is moved to the "wait" state, the backend handles that in the
|
| |
- `scheduler.handlers.modules.wait(...)` method.
|
| |
+ ``scheduler.handlers.modules.wait(...)`` method.
|
| |
|
| |
At first, this method uses KojiModuleBuilder to generate the Koji tag in which the components will be
|
| |
built. The Koji tag reflects the buildrequires of the module by inheriting their Koji tags. In our
|
| |
- case, the Koji tag would inherit just the `platform:f28` or `platform:f29` Koji tag, because that's
|
| |
+ case, the Koji tag would inherit just the ``platform:f28`` or ``platform:f29`` Koji tag, because that's
|
| |
the only buildrequired module we have.
|
| |
- The list of modules buildrequired by the currently building module is determined by the `buildrequires` list in
|
| |
- the `xmd` section of the expanded modulemd file.
|
| |
+ The list of modules buildrequired by the currently building module is determined by the ``buildrequires`` list in
|
| |
+ the ``xmd`` section of the expanded modulemd file.
|
| |
|
| |
- Once the Koji tag is ready, it tries to build a synthetic `module-build-macros` package. This
|
| |
+ Once the Koji tag is ready, it tries to build a synthetic ``module-build-macros`` package. This
|
| |
package contains special build macros which for example, define the dist-tag for built RPMs, ensure
|
| |
that filtered packages are not installed into the buildroot and so on.
|
| |
|
| |
@@ -102,17 +102,17 @@
|
| |
Module-build-macros package is built
|
| |
------------------------------------
|
| |
|
| |
- Once the `module-build-macros` package is built, Koji sends a message on the message bus, which is
|
| |
- handled in the `scheduler.handlers.components.complete(...)` method.
|
| |
+ Once the ``module-build-macros`` package is built, Koji sends a message on the message bus, which is
|
| |
+ handled in the ``scheduler.handlers.components.complete(...)`` method.
|
| |
|
| |
This method changes the state of that component build in the MBS database to "complete".
|
| |
|
| |
It then checks if there are any other unbuilt components in the current batch. Because the
|
| |
- `module-build-macros` package is the only component in batch 1, it can continue tagging it
|
| |
- into the Koji tag representing the module, so the `module-build-macros` can be later
|
| |
+ ``module-build-macros`` package is the only component in batch 1, it can continue tagging it
|
| |
+ into the Koji tag representing the module, so the ``module-build-macros`` can be later
|
| |
installed during the build of the next batch of components and can influence them.
|
| |
|
| |
- Note that the `module-build-macros` package is the only package in the course of a module build that
|
| |
+ Note that the ``module-build-macros`` package is the only package in the course of a module build that
|
| |
*only* gets tagged into the build tag. All other builds are tagged both into the build tag (to
|
| |
influence subsequent component builds) and into the destination tag (to be delivered as a component
|
| |
in the module).
|
| |
@@ -121,7 +121,7 @@
|
| |
Module-build-macros package is tagged into the Koji tag
|
| |
-------------------------------------------------------
|
| |
|
| |
- Once the module-build-macros package is tagged by Koji, the `scheduler.handlers.tags.tagged(...)`
|
| |
+ Once the module-build-macros package is tagged by Koji, the ``scheduler.handlers.tags.tagged(...)``
|
| |
method is called.
|
| |
|
| |
This simply waits until all the components in a currently built batch are tagged in a Koji tag.
|
| |
@@ -135,17 +135,17 @@
|
| |
------------------------------
|
| |
|
| |
Once the Koji repository containing packages from the currently built batch is regenerated,
|
| |
- the `scheduler.handlers.repos.done(...)` method is called.
|
| |
+ the ``scheduler.handlers.repos.done(...)`` method is called.
|
| |
|
| |
This verifies that all the packages from the current batch (just module-build-macros for now)
|
| |
really appear in the generated repository and if so, it starts building the next batch by calling
|
| |
- `utils.batches.start_next_batch_build(...)`.
|
| |
+ ``utils.batches.start_next_batch_build(...)``.
|
| |
|
| |
|
| |
Building the next batch
|
| |
-----------------------
|
| |
|
| |
- The `start_next_batch_build(...)` increases the `ModuleBuild.batch` counter to note that it
|
| |
+ The ``start_next_batch_build(...)`` increases the ``ModuleBuild.batch`` counter to note that it
|
| |
is going to build the next batch with the next component builds.
|
| |
|
| |
It then generates the list of unbuilt components in the batch and tries to reuse some from
|
| |
@@ -172,14 +172,14 @@
|
| |
-----------------------
|
| |
|
| |
Once the last component is built and the repository is regenerated, the
|
| |
- `scheduler.handlers.repos.done(...)` method moves the module build to the "done" state.
|
| |
+ ``scheduler.handlers.repos.done(...)`` method moves the module build to the "done" state.
|
| |
|
| |
|
| |
Importing the module build to Koji
|
| |
----------------------------------
|
| |
|
| |
- The "done" state message is handled in the `scheduler.handlers.modules.done(...)` method.
|
| |
+ The "done" state message is handled in the ``scheduler.handlers.modules.done(...)`` method.
|
| |
|
| |
- This method imports the module build into Koji using the `KojiContentGenerator` class.
|
| |
+ This method imports the module build into Koji using the ``KojiContentGenerator`` class.
|
| |
The module build in Koji points to the Koji tag with the module's components and also contains the
|
| |
final modulemd files generated for earch architecture the module is built for.
|
| |
merlinm
commented 5 years ago
This PR fixes up some incorrect RST markup in doc files so they render properly. No actual content change.