#85 Updated the local module building guide
Merged 3 years ago by mcurlej. Opened 3 years ago by mcurlej.
fedora-docs/ mcurlej/modularity localhost  into  master

@@ -1,23 +1,23 @@ 

- = Building Fedora modules locally

+ = Building modules locally

  

  This page walks you through building modules locally on your machine.

  

  == Prerequisites

  

- You need to install the `module-build-service` as well as `fedpkg`:

+ Before you can start building modules locally, you have to install the dependencies below:

  

  ----

- $ sudo dnf install module-build-service fedpkg

+ $ sudo dnf install module-build-service fedpkg rpkg

  ----

  

- and add yourself to the `mock` group:

+ If you are not running the build commands as root you have to add your user to the `mock` group:

  

  ----

  $ sudo usermod -a -G mock USERNAME

  $ newgrp

  ----

  

- == Step 1: Build your module

+ == Build your module from distgit

  

  Local builds are started using `fedpkg` from within your dist-git repository.

  
@@ -40,7 +40,84 @@ 

  

  where MODULE is "module-NAME-STREAM-VERSION", so for example "module-testmodule-master-20180612122805".

  

- == Step 2: Test your module

+ == Build your module with other SCM

+ 

+ WARNING: This feature is experimental use it at own risk. If you can use distgit for building your packages.

+ 

+ Building modules from other SCM (github, gitlab etc.) is more complicated then disgit. The installation prerequisities are the same as for disgit.

+ 

+ When all the prerequisities where installed we need to configure MBS and.

+ 

+ === MBS configuration

+ 

+ We need to set a configuration option in main MBS configuration file so it will accept other SCM urls not only disgit. In the file `/etc/module-build-service/config.py` we add the `DISTGITS` property to the `BaseConfiguration` class. 

+ 

+ ----

+ class BaseConfiguration(object):

+     .

+     .

+     .

+     DISTGITS = {"<scm_url_you_want_to_build_from>": (<cmd_that_clones_your_repo>, <cmd_which_gets_your_package_source>)}

+     .

+     .

+     .

+ ----

+ 

+ The defaults for the `DISTGITS` property can be found https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/common/config.py[here^]. The `DISTGITS` repository is a python dictionary. Each `key` of the `DISTGITS` property represents an SCM url. When MBS resolves where from to download the spec files and sources for the build process, it will look into the `DISGITS` property and try to match package urls with the keys of the dictionary. When a match is found then MBS knows this SCM is enabled.

+ 

+ The `value` part of each `key` in the `DISTGIT` property represents a python tuple with 2 elements. First element represents a bash command which clones your package repository into the package buildroot. Second element represents a bash command which downloads sources of your package into the SOURCE dir in the package buildroot at build time.

+ 

+ For example if the packages are hosted on github and we use rpmbuild built-in feature to download sources, the `DISTGITS` would look like this:

+ 

+ ----

+ class BaseConfiguration(object):

+     .

+     .

+     .

+     DISTGITS = {"https://github.com/user/package": ("git clone {repo_path}", None)}

+     .

+     .

+     .

+ ----

+ 

+ The `{repo_path}` will be replaced during runtime with url you provided in your modulemd yaml file when defining packages, but only for the first element of the tuple. The second element is set to `None` as it won't be downloading package sources through MBS but through rpmbuild. Although this needs to be enabled in your spec files. If you don't want to modify your spec files, then you have to define a bash command which will download your package sources.

+ 

+ === Spec file configuration (Optional)

+ 

+ To enable downloading package sources through rpmbuild, you have to modify the spec file of your RPM packages. By default rpmbuild does not download your sources from `Source0`. To enable this feature you need to define the `_disable_source_fetch` property to 0 somewhere before the `Source0` directive:

+ 

+ ----

+ %define _disable_source_fetch 0

+ Source0:        <url_to_your_source>

+ ----

+ 

+ === MBS Mock configuration

+ 

+ MBS has its own configuration file for Mock. The file can be found in `/etc/module-build-service/mock.cfg`. To be able to download package sources from the buildroot chroot environment we need to enable networking for mock. Just add this into the `/etc/module-build-service/mock.cfg`:

+ 

+ ----

+ config_opts['rpmbuild_networking'] = True

+ ----

+ 

+ === Building the module

+ 

+ When the initial setup has been done you can build the module stream with the following command:

+ 

+ ----

+ mbs-manager build_module_locally --offline -r /etc/yum.repos.d/fedora.repo --file <path_to_modulemd_file> --stream <name_of_module_stream>

+ ----

+ 

+ The logs and artifacts can be found in:

+ 

+ ----

+ ~/modulebuild/builds/MODULE/results/

+ ----

+ 

+ Where `MODULE` is "module-NAME-STREAM-VERSION", so for example "module-testmodule-master-20180612122805".

+ 

+ The `MODULE` directory holds the MBS main log. Further in the `results/` directory you will find logs for each package buildroot and the build artifacts.

+ 

+ == Testing your module

  

  WARNING: This requires Fedora 28 or higher.

  

WDYT? @jkaluza @frostyx

Issue: modularity:#174

Signed-off-by: Martin Curlej mcurlej@redhat.com

I think it's good for now. It's definitely an improvement to the previous version. Feel free to merge it.

This actually sounds more like "setup your own build infrastructure" rather than "build modules locally". Users are looking for something easier to use, closer to calling rpmbuild on spec.
That's something we do not have fully implemented (but @frostyx created dir2module already and we're getting close).

rebased onto edc5c51

3 years ago

Pull-Request has been merged by mcurlej

3 years ago

LGTM, I would merge because it adds a lot of helpful information.

However, I agree with @dmach. If somebody would tell me to do this so I could build modules locally, I would probably just not do it unless I was forced to.

Users are looking for something easier to use, closer to calling rpmbuild on spec.

Exactly.

That's something we do not have fully implemented (but @frostyx created dir2module already and we're getting close).

Just for the record, the project lives here

https://github.com/rpm-software-management/modulemd-tools

and we already have a blog post out

http://frostyx.cz/posts/introducing-modulemd-tools