fedmod modularity tools

fedmod provides tools for working with Fedora's modulemd metadata format that aren't related to actually building them (for build commands, see fedpkg and mbs-build).

Currently, this consists of:

• python -m fedmod rpm2module: generates a draft modulemd file based on the given RPM name (multiple RPM names can be given, but the resulting draft module will lack any descriptive metadata in that case)
• python -m fedmod fetch-metadata: download the F27 Modular Server repository metadata needed to generate draft module definitions

Project status

fedmod is not yet released, but can already generate basic draft modules from a local development checkout:

$ pipenv run python -m fedmod rpm2module samba

This will generate a draft modulemd file called samba.yaml in the current directory.

See the local development instructions below for info on setting up the execution environment with pipenv.

Interim alternative

fedmod aims to generate draft module definitions based on the same repository level metadata that dnf module uses to actually install and manage modules.

While this is the most appropriate approach for generating further modules once we actually have an initial Fedora Modular Server release available with multiple modules defined (since it will ensure that the module metadata used is accurate and up to date), it has some limitations while that initial release is still being assembled.

In the meantime, https://github.com/fedora-modularity/dependency-report-scripts allows for new module definitions to be generated based on the expected contents of the initial module set. While modulemd files created that way are still reasonable for now, they're expected to lose accuracy over time as module maintainers focus on dist-git and the actual module build service, and stop updating the now redundant metadata files in https://github.com/modularity-modules.

For now, a decent approach is to run both modulemd generators, and then pick whichever one you think looks better as your starting point.

Modulemd creation

python -m fedmod rpm2module [RPM NAMES] creates a modulemd file from the given package names. Output is written as <input-package-name>.yaml when a single package name is given, and modulemd-output.yaml otherwise.

The following metadata is currently used as input to the draft module generation process:

• Module definitions and package dependencies are pulled from the modular Fedora Bikeshed repository, with the metadata being downloaded for local use via the python -m fedmod metadata command

• Descriptive metadata is taken from the system running fedmod. Due to this, fedmod currently only supports Fedora 26+. (This will be fixed to use the same repository metadata as everything else)

Module dependencies currently err on the side of making the generated modules smaller by permitting generated modules to depend on packages that aren't listed as part of the public API of other modules. This reflects the fact that those transitive dependencies are typically the reason for the lower level modules appearing in the dependency set in the first place, as well as the fact that true dependency isolation will start being enforced once modules begin including opaque container images, such that only the client libraries are installed into shared environments.

Other limitations in generated modulemd files:

• components are only given a name and rationale, relying on the default settings for everything else
• the stream for module level dependencies is currently hardcoded to f27. This isn't right, but we can't set anything better until the mechanism for depending on multiple streams without naming them specifically is defined.

Local development

Runtime dependencies

The currently preferred means of dependency management is with pipenv:

$ pipenv --three --site-packages
$ PIP_IGNORE_INSTALLED=1 pipenv install --dev

This will pull in the required Python level dependencies from PyPI.

Some dependencies aren't currently available from PyPI, and hence need to be installed system-wide:

$ sudo dnf install python3-dnf python3-solv

How to run tests

To start a shell that's correctly configured to run the tests with the library and all necessary dependencies installed:

$ pipenv --three --site-packages
$ PIP_IGNORE_INSTALLED=1 pipenv install --dev
$ pipenv shell

The PIP_IGNORE_INSTALLED=1 setting means that everything available to pip will be installed into the virtual environment based on Pipfile.lock, and only components that aren't installable with pip will be used from the system Python installation.

See the next section for additional preparation currently needed to run the tests.

The tests can then be run in the launched subshell with:

$ pytest tests

Note: the tests will currently fail. Fixing that is a work in progress :)

Reviewing dependencies

To see the Python level dependencies graph:

$ pew toggleglobalsitepackages
$ pipenv graph
$ pew toggleglobalsitepackages

(If you don't turn off global site-packages access first, you'll get the dependency graph of all the installed system Python components as well)