From 4402dbe2dcd3df59e6ae90639b22d8a5e8506274 Mon Sep 17 00:00:00 2001 From: Jakub Kadlcik Date: May 29 2020 11:40:17 +0000 Subject: Added COPR tutorial and updated organization of the wiki Added COPR tutorial created by @frostyx (frostyx@email.cz). Changed the file structure of the wiki to more mirror the wiki navigation. Grouped related pages. Issue: modularity#174 Signed-off-by: Martin Curlej --- diff --git a/modules/ROOT/assets/images/copr-build-new-module.png b/modules/ROOT/assets/images/copr-build-new-module.png new file mode 100644 index 0000000..356dd6d Binary files /dev/null and b/modules/ROOT/assets/images/copr-build-new-module.png differ diff --git a/modules/ROOT/assets/images/copr-build-result.png b/modules/ROOT/assets/images/copr-build-result.png new file mode 100644 index 0000000..e9adb72 Binary files /dev/null and b/modules/ROOT/assets/images/copr-build-result.png differ diff --git a/modules/ROOT/assets/images/copr-disgit.png b/modules/ROOT/assets/images/copr-disgit.png new file mode 100644 index 0000000..d04b7d3 Binary files /dev/null and b/modules/ROOT/assets/images/copr-disgit.png differ diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 2daee0a..c8e531e 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -12,17 +12,19 @@ ** xref:installing-modules.adoc[Installation and Discovery] ** xref:using-modules-switching-streams.adoc[Switching Streams] ** xref:removing-modules.adoc[Removing Modules] -* xref:making-modules.adoc[Making Modules — Guides] -** xref:making-modules/adding-new-modules.adoc[Adding New Modules] -** xref:making-modules/updating-modules.adoc[Updating Existing Modules] -** xref:making-modules/defining-modules.adoc[Defining Modules in modulemd] -** xref:making-modules/building-modules.adoc[Building Modules] -** xref:making-modules/managing-defaults.adoc[Managing Defaults] -** xref:making-modules/inspecting-build-failures.adoc[Inspecting Build Failures] -** xref:making-modules/building-modules-locally.adoc[Building Modules Locally] -* xref:making-modules-policies.adoc[Making Modules — Policies] -** xref:making-modules/packaging-guidelines.adoc[Packaging Policy] -** xref:making-modules/naming-guidelines.adoc[Naming Policy] +* xref:building-modules/index.adoc[Building Modules] +** xref:building-modules/fedora/index.adoc[Fedora] +*** xref:building-modules/fedora/adding-new-modules.adoc[Adding New Modules] +*** xref:building-modules/fedora/updating-modules.adoc[Updating Existing Modules] +*** xref:building-modules/fedora/defining-modules.adoc[Defining Modules in modulemd] +*** xref:building-modules/fedora/building-modules.adoc[Building Modules] +*** xref:building-modules/fedora/managing-defaults.adoc[Managing Defaults] +*** xref:building-modules/fedora/inspecting-build-failures.adoc[Inspecting Build Failures] +** xref:building-modules/copr/building-modules-in-copr.adoc[COPR] +** xref:building-modules/local/building-modules-locally.adoc[Localhost] +* xref:policies/index.adoc[Policies] +** xref:policies/packaging-guidelines.adoc[Packaging Policy] +** xref:policies/naming-guidelines.adoc[Naming Policy] * xref:faq.adoc[FAQ] * **Advanced** diff --git a/modules/ROOT/pages/building-modules/copr/building-modules-in-copr.adoc b/modules/ROOT/pages/building-modules/copr/building-modules-in-copr.adoc new file mode 100644 index 0000000..e58e446 --- /dev/null +++ b/modules/ROOT/pages/building-modules/copr/building-modules-in-copr.adoc @@ -0,0 +1,201 @@ += Building modules in Copr + + +== Introduction + +Copr ("Community projects") is a service that builds your open-source projects and creates your own +RPM repositories. Read the https://docs.pagure.org/copr.copr/user_documentation.html[documentation] +to find out more or see it in action https://copr.fedorainfracloud.org/[here]. + +There are multiple ways how to build modules in Copr and it is up to each user to choose the most +suitable one for their specific use-case. + +1. Do you know what modulemd yaml file is and do you have it written for your module? + Then <> to Copr. +2. Do you have a Copr project providing some RPM packages and want to make module from them? + Then don't write a modulemd yaml file manually and + <>. + +Discover more advanced topics such as <> from an existing modulemd +yaml file, <>, and +<>. + + +== Disclaimer + +Copr is a third-party service with its own implementation of the module build process. It does not +rely on the https://pagure.io/fm-orchestrator[Module Build Service^] and it is not governed by xref:community.adoc[Fedora Modularity team]. +As a consequence, some features might behave differently. + +Some features are also in rather experimental or proof-of-concept state (e.g. depending on other +Copr modules) and therefore might be cumbersome to use. Feedback from users is more than welcome and + it is the deciding factor of what is going to be implemented or improved. Please submit your +requests to https://pagure.io/copr/copr/issues[pagure issue tracker]. + +Additionally, only versions 1 and 2 of the modulemd format are supported. +Version 3 is not supported yet. + + +== Submit an existing modulemd yaml + +Currently, there is no web UI for submitting a module build from an existing modulemd yaml file. +The only way is to use `copr-cli` package. Make sure it is installed. + +---- +dnf install copr-cli +---- + +And make sure that an https://copr.fedorainfracloud.org/api/[API token is properly configured]. + +To submit a build from locally stored modulemd yaml file, use: + +---- +copr-cli build-module --yaml /some/path/testmodule.yaml +---- + +If the modulemd yaml file is available on a URL accessible for public, use following: + +---- +copr-cli build-module --url https://example/some/path/testmodule.yaml +---- + +To specify a different project owner (e.g. when building into a group project), use: + +---- +copr-cli build-module --yaml /some/path/testmodule.yaml @mygroup/testmodule +---- + +For more information, see `copr-cli build-module --help` or `man copr-cli`. + + +== Generate modulemd from a Copr project + +This method aims to provide modularity features with the lowest entry barrier possible. It doesn't +demand a user to know either how to write modulemd yaml file nor using the command line. The only +prerequisite is to have a Copr project with some package successfully built in it. + +Open your project, click on `Modules`, then click to `New Module`. The input page looks like this. + +image::copr-build-new-module.png[,100%,] + +It provides a list of all successfully built packages in the project. By default, they are all +selected to become part of the module. Uncheck those that are not wanted. Optionally specify +https://pagure.io/modulemd/blob/ade28f3f3b39fcddcb626ca915df1a6ce35c14fd/f/spec.yaml#_137[a module API] +or its https://pagure.io/modulemd/blob/ade28f3f3b39fcddcb626ca915df1a6ce35c14fd/f/spec.yaml#_90[profiles]. + + +== Modules with Copr packages + +It is trivial to achieve when generating modulemd from Copr project. This section +describes a special case of how to add a Copr package into an existing custom modulemd yaml file. + +Update `components.rpms` section and add a new package definition. + +---- +hello: + rationale: Showing how to use a package from Copr + ref: ... + repository: ... +---- + +The `...` placeholders need to be replaced with real values. Open a web browser and navigate +to a build detail whose results should be used by the module. The build needs to be finished +successfully. Scroll down to the `Results` section. + +image::copr-build-result.png[,100%,] + +- Use the hash value from the `Dist Git Source` column as a `ref` in the modulemd. +- Follow the link of the `Dist Git Source` value to a Copr distgit instance. Use the URL as + a `repository` in modulemd - with some small adjustments. + * Trim everything after `.git` + * Replace `/cgit/` with `/git/` + +image::copr-disgit.png[,100%,] + +To avoid confusion, here are some example values. + +---- +hello: + rationale: Showing how to use a package from Copr + ref: 9d1ced1 + repository: http://copr-dist-git.fedorainfracloud.org/git/frostyx/hello/hello.git +---- + + +== Modules in Copr as dependencies + +At this moment, Copr doesn't properly parse module dependencies from modulemd yaml file. They need +to be explicitly enabled in the project settings - more precisely chroot settings within a project. + +When the dependency comes from Fedora repositories, simply edit `Enable module` field and append +the module in `name:stream` format. + +If the dependency was built in Copr, it is necessary to edit also the `Repos` field and provide a +repository URL from which the module can be installed. See the <> +section to learn how to find a repofile for a module (in this case the module that is used +as a dependency). Use it's `baseurl` as the value for `Repos` field. + + +== Installing modules from Copr + +Currently `dnf copr` plugin does not support enabling module repositories and therefore it needs to +be done manually. Navigate to a Copr project in a web browser, see the `Modules` page, and open +detail of a module that is to be installed. If the build is still running, wait until it +finishes. For successfully built modules, a "How to use" section is displayed. Follow its +instructions. After that, standard DNF commands for manipulation with modules can be used. + +See https://docs.fedoraproject.org/en-US/modularity/installing-modules/ + + +== Override module packages + +The package version resolution was straightforward in the pre-modularity era. If multiple +repositories provided the same package, the one with the highest version was preferred. +It is https://dnf.readthedocs.io/en/latest/modularity.html#package-filtering[more complicated now]. +If there is a module stream enabled within the system and this stream provides a package, it is +preferred over its non-modular variant regardless of its version. + +To suppress this behavior, `module_hotfixes` repositories were invented. For them, DNF upgrades to a +higher version of a package regardless of where it comes from. Enable them in Copr by going to +project settings and turning on this checkbox. + +---- +[ ] This repository contains module hotfixes + This will make packages from this project available on along with + packages from the active module streams. +---- + +Or in a command line + +---- +copr-cli modify frostyx/foo --module-hotfixes on +---- + + +== Examples + +Build a module from a modulemd yaml file hosted online: + +---- +copr-cli create testproject1 --chroot fedora-rawhide-x86_64 +copr-cli build-module testproject1 \ + --url https://src.fedoraproject.org/modules/httpd/raw/2.4/f/httpd.yaml +---- + +Alternatively, build a module from a localy stored modulemd yaml file: + +---- +fedpkg clone -a modules/httpd +cd httpd +copr-cli create testproject2 --chroot fedora-rawhide-x86_64 +copr-cli build-module testproject2 --yaml httpd.yaml +---- + + +== External resources + +- http://frostyx.cz/posts/how-to-build-modules-in-copr +- http://frostyx.cz/posts/modules-with-copr-packages +- http://frostyx.cz/posts/copr-loves-modularity +- http://frostyx.cz/posts/module-hotfixes-in-copr +- http://frostyx.cz/posts/copr-modularity-in-retrospect diff --git a/modules/ROOT/pages/building-modules/fedora/adding-new-modules.adoc b/modules/ROOT/pages/building-modules/fedora/adding-new-modules.adoc new file mode 100644 index 0000000..c065ace --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/adding-new-modules.adoc @@ -0,0 +1,98 @@ += Adding new modules to Fedora + +This page will guide you through the whole process of adding a new module to Fedora: + +1. **<>** — To `dist-git/rpms` using stream branches. +2. **<>** — To `dist-git/modules` using stream branches. +3. **<>** — Module is built as a unit. No individual package builds are done. +4. **<>** — Submitting a Bodhi update. + +== RPM Sources + +For each package in your module, you need to have a Fedora Distribution Git (dist-git) **repository** under `dist-git/rpms` with an appropriate **stream branch**. New packages need to go through a **package review**. + +=== Repositories and Stream Branches — New packages + +The https://fedoraproject.org/wiki/Package_Review_Process#Contributor[Fedora Package Review Process] covers all the steps you need to take, except for stream branches. For that, please continue below to Existing packages. + +=== Repositories and Stream Branches — Existing packages +Requesting new stream branches for existing packages is done with `fedpkg`, and it doesn't require a package review. + +To create a new branch for a package that shares a name with the module (e.g. "postgresql"), run the following command which will create the package stream branch as well as the module and its stream branch together: + +---- +$ fedpkg request-branch --repo=NAME --sl rawhide:2020-12-01 -- BRANCH +---- +* `NAME` — name of the package and the module +* `BRANCH` — name of the stream branch of the package and the module +* `rawhide:2020-12-01` — expected end of life, must end with either `12-01` or `06-01`. Right now, this is not actively used to do anything useful, although some systems might block you from maintaining your module. Please choose a date far in the future so you're not affected. [Work to fix this is in progress](https://tree.taiga.io/project/modularity-wg/epic/3). + +If this package does not share a name with the module (such as for dependencies), then only request the stream branch for this package: + +---- +$ fedpkg request-branch --repo=NAME --no-auto-module --sl rawhide:2020-12-01 -- BRANCH +---- +* `--no-auto-module` — Skip the creation of a module of the same name + +Creation of the module and its stream branches separately will be covered later. + +=== Importing the RPM sources into dist-git + +The last thing to do is to import the RPM sources into dist-git. Please refer to the https://fedoraproject.org/wiki/New_package_process_for_existing_contributors[New package process for existing contributors +] for the complete steps. + +== Module Definition + +For your module definition, you also need to have a dist-git **repository**, this time under `dist-git/modules`, with an appropriate **stream branch**. + +=== Repositories and Stream Branches — New modules + +To request a new module repository in dist-git run: + +---- +$ fedpkg request-repo --namespace modules --exception NAME +$ fedpkg request-branch --namespace modules --repo NAME BRANCH +---- + +* `NAME` — name of the module +* `BRANCH` — name of the stream branch of the module + +=== Repositories and Stream Branches — Existing modules + +to request a new stream branch, run: + +---- +$ fedpkg request-branch --namespace modules --repo NAME BRANCH +---- + +* `NAME` — name of the module +* `BRANCH` — name of the stream branch of the module + +=== Write and push the modulemd + +Writing a modulemd is covered in detail in the xref:building-modules/fedora/defining-modules.adoc[Defining Modules in modulemd] section. + +NOTE: The name of the modulemd must match the name of the repository. Remember to replace every `NAME` in the following example. + +---- +$ fedpkg clone modules/NAME +$ cd NAME +$ fedpkg switch-branch BRANCH +$ vim NAME.yaml +$ git add NAME.yaml +$ git commit -m "add the initial modulemd" +$ git push +---- + +* `NAME` — name of the module +* `BRANCH` — name of the stream branch of the module + +== Module Build + +NOTE: NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. + +Submitting module builds is done using `fedpkg` and is covered in the xref:building-modules/fedora/building-modules.adoc[Building modules] section. + +== Publishing the Module + +include::{partialsdir}/submit-module-update.adoc[] \ No newline at end of file diff --git a/modules/ROOT/pages/building-modules/fedora/building-modules.adoc b/modules/ROOT/pages/building-modules/fedora/building-modules.adoc new file mode 100644 index 0000000..cdc96c3 --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/building-modules.adoc @@ -0,0 +1,68 @@ += Submitting module builds in Fedora + +NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. + +Module builds are triggered using `fedpkg` from within your dist-git repository. + +---- +$ fedpkg clone modules/NAME +$ cd NAME +$ fedpkg switch-branch BRANCH +$ fedpkg module-build +---- + +* `NAME` — name of the module +* `BRANCH` — name of the stream branch of the module + +NOTE: Please note the module build ID. You will need it to verify the build state and to publish the module later. + +To watch the state of your module build, run: + +---- +$ fedpkg module-build-watch BUILD_ID +---- + +When the module is in a "ready" state, the build has successfully completed. + +You can also watch the build(s) on Fedora Module build service: + +https://release-engineering.github.io/mbs-ui/modules + +== Module build results + +All the packages built in the module are tagged in Koji tag named as ``module-$name-$stream-$version-$context``. The link to this tag is also available in Fedora Module build service UI mentioned earlier. + +There are also two Koji Builds representing single module build. + +* ``$name-$stream-$version.$context`` - Contains the metadata with final RPMs which will be included in a compose and will be therefore visible by the end user. +* ``$name-devel-$stream-$version.$context`` - Contains the metadata with filtered RPMs which are not normally visible to end user, but might be sometimes useful when building other packages against the module. Generally, in Fedora we do not ship the -devel modules. + +NOTE: You will not find ``$name-devel`` module build in Module Build Service, because this is only another representation of non-devel module build. It is not separate module build on its own and it only exists as Koji build. + +Both Koji module builds contains various metadata files: + +- ``modulemd.src.txt`` - The unchanged input modulemd metadata used to build a module as stored in dist-git. +- ``modulemd.txt`` - The expanded modulemd metadata which was used by Module Build Service to build this particular module. This for example contains the right build requirements based on the Module Stream Expansion. +- ``modulemd.$arch.txt`` - The per-architecture modulemd metadata listing the RPMs which will end up in a compose and therefore be visible to end user. This for example respects filters or multilib modulemd options. + +== Rebuild strategies + +In case you want to control which packages get rebuilt and which get reused, you can enforce a specific rebuild strategy while submitting a build. + +There are different rebuild strategies to choose from: + +* ``all`` — All packages in the module get rebuilt. +* ``only-changed`` — Only packages that have changed since the last successful build get rebuilt. This is the **default** in Fedora. +* ``changed-and-after`` — This one leverages xref:building-modules/fedora/defining-modules.adoc#_building_in_a_specific_order_optional[build groups (buildorder)]. Packages that have changed since the last successful build get rebuilt, and also all packages with a buildorder higher than any of the changed ones get rebuilt as well. + +For more detailed information, please see the +https://pagure.io/fm-orchestrator/blob/master/f/docs/REBUILD_STRATEGIES.rst[MBS rebuild strategies documentation]. + +To enforce a specific rebuild strategy, submit the module build with the following command: + +---- +$ fedpkg module-build --optional rebuild_strategy=STRATEGY +---- + +* `STRATEGY` — name of a specific build strategy (listed above) + diff --git a/modules/ROOT/pages/building-modules/fedora/defining-modules.adoc b/modules/ROOT/pages/building-modules/fedora/defining-modules.adoc new file mode 100644 index 0000000..155d255 --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/defining-modules.adoc @@ -0,0 +1,477 @@ += Defining modules in modulemd + +Simply put, **modulemd is a file that defines which packages get built for which releases**. It includes a summary and a description, a list of source RPM packages, build information i.e. build order and macros, and usage information i.e. installation profiles and licenses. + +== A typical modulemd example + +A typical modulemd file looks similar to the following examples. Read on for more details about each part of the modulemd file. + +Feel free to copy/paste this example when creating your new module. + +[source,yaml] +---- +document: modulemd +version: 2 +data: + # === Information about this module ================================== + # (Can be copied from the main RPM package, but doesn't need to be) + summary: An example module + description: >- + A module for the demonstration of the metadata format. Also, + the obligatory lorem ipsum dolor sit amet goes right here. + + # === License of this modulemd file ================================== + # (Package licenses will be added automatically by the build system) + license: + module: + - MIT + + # === Modular dependencies =========================================== + # (For which Fedora releases to build?) + dependencies: + - buildrequires: + platform: [] # <- Build for all Fedora releases + requires: + platform: [] # <- Run on all Fedora releases + + # === Module API (optional, but encouraged) ========================== + # (Which packages are API-stable?) + api: + rpms: + - package-one # <- Binary RPM package name + - package-one-extras # <- Binary RPM package name + - package-one-cli # <- Binary RPM package name + - package-one-devel # <- Binary RPM package name + - package-two # <- Binary RPM package name + + # === Installation profiles (optional, but encouraged) =============== + # (Helping users with installation by providing predefined groups) + profiles: + default: # <- Name of the profile + description: A standard installation. + rpms: + - package-one # <- Binary RPM package name + - package-one-extras # <- Binary RPM package name + - package-two # <- Binary RPM package name + cli: # <- Name of the profile + description: A command-line client. + rpms: + - package-one-cli # <- Binary RPM package name + + # === Packages in this module ======================================== + # (Referenced by their dist-git repo name + branch name) + components: + rpms: + first-package: # <- Source RPM package name + ref: 3.0 # <- Branch name in dist-git + rationale: Provides the core functionality. + second-package: # <- Source RPM package name + ref: latest # <- Branch name in dist-git + rationale: Web UI for the first-package. +---- + +== Common modulemd definitions + +These are the common parts of a modulemd file, used in the example above. Advanced definitions, including a complex example of Module Stream Expansion (MSE), are towards the end of this page. + +=== Document header + +Every modulemd starts with these three lines: + +[source,yaml] +---- +document: modulemd +version: 2 +data: + ... <1> +---- + +<1> All the following definitions go here, under _data_. + +=== Information about this module + +Tell users what this module represents by writing a summary and a description. + +[source,yaml] +---- + summary: An example module + description: >- <1> + A module for the demonstration of the metadata format. Also, + the obligatory lorem ipsum dolor sit amet goes right here. +---- + +<1> The `>-` means new line in YAML. Useful for longer blocks of text, such as the description! + + +=== License of this modulemd file + +This is a license of this very modulemd file and it doesn't need to be modified. +The build system adds licenses of all packages to this list automatically. + +[source,yaml] +---- + license: + module: + - MIT <1> +---- + +<1> A license for this modulemd file. Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. + +=== Modular dependencies + +Simply put: For which Fedora releases to build? + +To build your module for all Fedora releases that are actively maintained, use the following definition. For anything more than this, such as building against other modules or requiring other modules during run time, please see the Advanced section below. + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [] + requires: + platform: [] +---- + +=== Installation profiles (optional, but encouraged) + +To help users install your module, define installation profiles. These profiles represent a specific use case of your module. Most modules have at least a default profile. But you can specify more. For example, a database module can have a server and a client profile. + +[source,yaml] +---- + profiles: + default: <1> + description: A standard installation. <2> + rpms: + - package-one <3> + - package-one-extras <3> + - package-two <3> + cli: + description: A command-line client. + rpms: + - package-one-cli + ... <4> +---- + +<1> Name of the profile. + +<2> A quick summary of the profile. + +<3> Binary packages to be installed with this profile. + +<4> List as many profiles as you need. + +=== Module API (optional, but encouraged) + +List all binary RPM packages in your module that you consider to be the main stable feature of the module. Other (unlisted) packages should be considered unsupported, or an implementation detail. + +[source,yaml] +---- + api: + rpms: + - package-one + - package-one-extras + - package-one-cli + - package-one-devel + - package-two +---- + +=== Packages in this module + +List all source SRPM packages this module should include, referenced them by their dist-git repo name + branch name. + +[source,yaml] +---- + components: + rpms: + first-package: <1> + rationale: Provides the core functionality. <2> + ref: 3.0 <3> + second-package: + rationale: Web UI for the first-package. + ref: latest + ... <4> +---- + +<1> Name of the package — maps to a DistGit repository name. + +<2> The reason why is this package here. Mostly for humans. + +<3> DistGit branch, tag, or a commit — so the right version of the package gets included. + +<4> List as many packages as you need. + +== Advanced definitions + +=== References to the upstream (optional) + +You can also provide references to the upstream community, documentation, or to an issue tracker. + +[source,yaml] +---- + references: + community: http://www.example.com/ <1> + documentation: http://www.example.com/ <2> + tracker: http://www.example.com/ <3> +---- + +<1> Upstream community website, if it exists. + +<2> Upstream documentation, if it exists. + +<3> Upstream bug tracker, if it exists. + +=== Building in a specific order (optional) + +Packages are built in batches. By default, all packages are part of a single group, and therefore built concurrently. + +To build packages in a specific order, assign them to multiple build groups. Build groups are identified by an integer. Groups with lower number are built first. Negative values are allowed, `0` is the implicit default value. + +In this specific example, `first-package` gets built first, and `second-package` gets built second. + +[source,yaml] +---- + components: + rpms: + first-package: + rationale: Provides the core functionality. + ref: 3.0 + buildorder: 0 <1> + second-package: + rationale: Web UI for the first-package. + ref: latest + buildorder: 10 <1> +---- + +<1> A number of the build group. + +=== Build macros (optional) + +Use this if you need to build your packages with a specific RPM macro. Applies to all packages in the module. + +[source,yaml] +---- + buildopts: + rpms: + macros: | + %demomacro 1 + %demomacro2 %{demomacro}23 +---- + +=== Advanced dependencies (optional) + +modules can be + +* built against other modules +* require other modules during run time +* built against one or more streams of the same module +* work with one or more streams of another module + +**Building only for Fedora 28**: + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [f28] + requires: + platform: [f28] +---- + +**Building for everything else than Fedora 28**: + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [-f28] + requires: + platform: [-f28] +---- + +**Building only for Fedora 28 and Fedora 29**: + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [f28, f29] + requires: + platform: [f28, f29] +---- + +**Building against other modules:**: + +Your module can also depend on another modules. Specific streams can be referenced the same way as above in "Building for a specific Fedora release(s) only". + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [] + nodejs: [] + requires: + platform: [] + nodejs: [] +---- + +**A complex example:** + +Simple things should simple, complex things should be possible. Let's say my module requires `nodejs` during build and run time. It also requires `build-tools` only during build. To make it even more complex, it also requires a specific stream of a `pizza-module` during build and run time, but only on Fedora 27. + +[source,yaml] +---- + dependencies: + - buildrequires: + platform: [-f27] + buildtools: [v1, v2] + nodejs: [] + requires: + platform: [-f27] + nodejs: [] + - buildrequires: + platform: [f27] + buildtools: [v1, v2] + nodejs: [] + pizza-module: [3.6] + requires: + platform: [f27] + nodejs: [] + pizza-module: [3.6] +---- + +For even more complex scenarios, please study the https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml[modulemd specification]. + + +=== Excluding binary packages (optional) + +One source RPM package might produce multiple binary RPM packages. If you don't want to include some binary packages, list them under `filter`. + +Filtered RPMs are still available to use as build dependencies in subsequent stages of the module build, but are not included in the composed repository for users. + +[source,yaml] +---- + filter: + rpms: + - first-package-debuginfo + - second-package-nope +---- + +=== Creating build-only components (optional) + +In addition to filtering subpackages, it's possible to filter out all of the artifacts produced by a component in a module. +This is useful in cases where your module's primary packages have a build-time dependency that you do not want to ship. +An example of such a case would be if you need to build with a specially-patched documentation-generator that would conflict +with the version used as the default in Fedora. + +[source,yaml] +---- + components: + rpms: + customdocgen: + rationale: A patched version of docgen that enables an experimental feature. + ref: experimental + buildorder: 0 + buildonly: 1 + myapp: + rationale: My application + ref: latest + buildorder: 10 +---- + +In this example, `customdocgen` would be built first and made available in the buildroot for `myapp` to use during its build. +Once the module build is finished and it is composed into a DNF repository, only the unfiltered artifacts from `myapp` will be available. +All of the `customdocgen` artifacts will be automatically added to the `data.filters.rpms` section of the module metadata. + +== A minimal modulemd + +=== An absolute minimum + +This module includes two source RPM packages built for all Fedora releases. + +[source,yaml] +---- +document: modulemd +version: 2 +data: + summary: An example module + description: >- + A module for the demonstration of the metadata format. Also, + the obligatory lorem ipsum dolor sit amet goes right here. + license: + module: + - MIT + dependencies: + - buildrequires: + platform: [] + requires: + platform: [] + components: + rpms: + first-package: + rationale: Provides the core functionality. + ref: 3.0 + second-package: + rationale: Web UI for the first-package. + ref: latest +---- + +=== Including profiles and API (recommended) + +This module includes two source RPM packages built for all Fedora releases. It makes clear which packages are considered the API, and helps users with installation thanks to the profiles. + +[source,yaml] +---- +document: modulemd +version: 2 +data: + summary: An example module + description: >- + A module for the demonstration of the metadata format. Also, + the obligatory lorem ipsum dolor sit amet goes right here. + license: + module: + - MIT + dependencies: + - buildrequires: + platform: [] + requires: + platform: [] + api: + rpms: + - package-one + - package-one-extras + - package-one-cli + - package-one-devel + - package-two + profiles: + default: + description: A standard installation. + rpms: + - package-one + - package-one-extras + - package-two + cli: + description: A command-line client. + rpms: + - package-one-cli + components: + rpms: + first-package: + rationale: Provides the core functionality. + ref: 3.0 + second-package: + rationale: Web UI for the first-package. + ref: latest +---- + +== Using fedmod + +To get started quickly, you can use the `fedmod` tool (from the package of the same name) to create a skeletal modulemd file for your module based on one or more source packages, e.g. + +[source,shell] +---- +$ fedmod fetch-metadata +$ fedmod rpm2module dwm > dwm.yaml +---- + +If you specify more than one package it will fill in the summary and description fields from their counterparts in the first specified package. diff --git a/modules/ROOT/pages/building-modules/fedora/index.adoc b/modules/ROOT/pages/building-modules/fedora/index.adoc new file mode 100644 index 0000000..3092431 --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/index.adoc @@ -0,0 +1,5 @@ += Building Modules for Fedora + +If you wish to maintain an additional version of an application, language stack, or any other piece of software, modules might be the way to go for you. + +The xref:building-modules/fedora/adding-new-modules.adoc[Adding New Modules] covers the whole process of adding a new module to Fedora. \ No newline at end of file diff --git a/modules/ROOT/pages/building-modules/fedora/inspecting-build-failures.adoc b/modules/ROOT/pages/building-modules/fedora/inspecting-build-failures.adoc new file mode 100644 index 0000000..122e5a5 --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/inspecting-build-failures.adoc @@ -0,0 +1,15 @@ += Inspecting module build failures in Fedora + +Have you submitted a build which failed? This page will point you to the right places to get all the information about build failures. + +== Inspecting build failures + +Run the following command using the BUILD_ID from the previous step: + +NOTE: Reminder: You got the BUILD_ID when you xref:building-modules.adoc[submitted a module build]. + +---- +$ fedpkg module-build-info BUILD_ID +---- + +which shows you "State Reason" — a short summary of the build failure regarding the whole module, as well as a list of individual packages in the module. Find the ones that failed and go to the "Koji Task" URL. There, click on the "Build" link which gets you to the module build page. On this page, inspect the `root.log`, `build.log`, and other files to get the build error. diff --git a/modules/ROOT/pages/building-modules/fedora/managing-defaults.adoc b/modules/ROOT/pages/building-modules/fedora/managing-defaults.adoc new file mode 100644 index 0000000..25caa9d --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/managing-defaults.adoc @@ -0,0 +1,23 @@ += Managing module defaults in Fedora + +Setting or changing a default stream or a default installation profile of a module constitutes a major behavior change as defined in the Fedora Updates Policy. The following rules apply: + +1. Module stream defaults MUST be only changed in an upcoming Fedora release +2. Changes of stream defaults should be communicated by a Fedora Change based on the change's significance and its maintainer's best judgement. When in doubt, file a Change. +3. Changes of the default stream of a module are not permitted within a released Fedora without the approval of FESCo. +4. Introducing a new default stream not replacing any existing default stream or a traditional package is not considered a change. That means it can be done. + +== Setting or changing a default + +Submit an issue to the Fedora Engineering Steering Committee in https://pagure.io/fesco/issues[pagure.io/fesco]. This ticket must provide the following information: + +* The list of SRPM components in the module +* Whether the contents of this module will obsolete and replace non-modular RPMs. +* Which releases of Fedora will this new default apply to. + + +When requesting a change of default stream, strongly consider https://fedoraproject.org/wiki/Changes/Policy#For_developers[submitting a Fedora Change]. This will help ensure that the change is communicated to the rest of the Fedora community. + +To check the current defaults, have a look at the https://pagure.io/releng/fedora-module-defaults[fedora module defaults repository]. + +Please note that if the module stream masks part of the Traditional RPM repos (e.g it replaces an existing RPM or it introduces a non-trivial set of conflicts) it may not be made a default stream without the express permission of FESCo. Release Engineering will be responsible for escalating any PR that is questionable on this point to FESCo for a final decision. diff --git a/modules/ROOT/pages/building-modules/fedora/updating-modules.adoc b/modules/ROOT/pages/building-modules/fedora/updating-modules.adoc new file mode 100644 index 0000000..479cbf3 --- /dev/null +++ b/modules/ROOT/pages/building-modules/fedora/updating-modules.adoc @@ -0,0 +1,42 @@ += Updating modules in Fedora + +This page will guide you through the process of updating an existing module. + +1. **<>** — Pushing new sources. +2. **<>** — Pushing a new version of the modulemd. +3. **<>** — Module is built as a unit. No individual package builds are done. +4. **<>** — Submitting a Bodhi update. + +== Updating RPM Packages + +Update your RPM packages the same way you would do traditionally, except submitting individual package builds. + +Useful resources: + +* https://fedoraproject.org/wiki/Packaging:Guidelines[Fedora Packaging Guidelines] +* https://fedoraproject.org/wiki/Package_maintenance_guide[Fedora Package Maintenance Guide] + +== Updating the Module + +Even when you don't need to make any changes to the modulemd, you still need to push a new commit to build a new version. + +---- +$ fedpkg clone modules/NAME +$ cd NAME +$ fedpkg switch-branch BRANCH +$ git commit --allow-empty -m "update" +$ git push +---- + +* `NAME` — name of the module +* `BRANCH` — name of the stream branch of the module + +== Module Build + +NOTE: NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. + +Submitting module builds is done using `fedpkg` and is covered in the xref:building-modules/fedora/building-modules.adoc[Building modules] section. + +== Publishing the Module + +include::{partialsdir}/submit-module-update.adoc[] \ No newline at end of file diff --git a/modules/ROOT/pages/building-modules/index.adoc b/modules/ROOT/pages/building-modules/index.adoc new file mode 100644 index 0000000..6eec3a4 --- /dev/null +++ b/modules/ROOT/pages/building-modules/index.adoc @@ -0,0 +1,5 @@ += Building Modules + +1. **xref::building-modules/fedora/index.adoc[Fedora]** +2. **xref::building-modules/copr/building-modules-in-copr.adoc[COPR]** +3. **xref::building-modules/local/building-modules-locally.adoc[Localhost]** \ No newline at end of file diff --git a/modules/ROOT/pages/building-modules/local/building-modules-locally.adoc b/modules/ROOT/pages/building-modules/local/building-modules-locally.adoc new file mode 100644 index 0000000..1be0c38 --- /dev/null +++ b/modules/ROOT/pages/building-modules/local/building-modules-locally.adoc @@ -0,0 +1,64 @@ += Building Fedora modules locally + +This page walks you through building modules locally on your machine. + +== Prerequisites + +You need to install the `module-build-service` as well as `fedpkg`: + +---- +$ sudo dnf install module-build-service fedpkg +---- + +and add yourself to the `mock` group: + +---- +$ sudo usermod -a -G mock USERNAME +$ newgrp +---- + +== Step 1: Build your module + +Local builds are started using `fedpkg` from within your dist-git repository. + +For example, to submit a build of the `testmodule:master` module, run: + +NOTE: Local builds don’t support stream expansion. If your module depends on multiple streams of another module, such as `platform`, you need to specify what stream you want to build against. For example: `fedpkg module-build-local -s platform:f28`. + +---- +$ fedpkg clone modules/testmodule +$ cd testmodule +$ git checkout master +$ fedpkg module-build-local +---- + +When the build finishes, you'll be able to find the results in: + +---- +~/modulebuild/builds/MODULE/results/ +---- + +where MODULE is "module-NAME-STREAM-VERSION", so for example "module-testmodule-master-20180612122805". + +== Step 2: Test your module + +WARNING: This requires Fedora 28 or higher. + +To test your module, add the repository to your system by creating an entry similar to this in `/etc/yum.repos.d/test.repo`: + +---- +[local] +name=My Local Repository +baseurl=file:///home/adam/modulebuild/builds/module-testmodule-master-20180612122805/results/ +gpgcheck=0 +enabled=1 +---- + +Please replace "adam" with your user name, and "module-testmodule-master-20180612122805" with the actual value for your module. + +Now, when you list the available modules on your system, you should be able to see it and install it: + +---- +$ dnf module list +$ dnf module install testmodule:master +---- \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules-policies.adoc b/modules/ROOT/pages/making-modules-policies.adoc deleted file mode 100644 index 2c171c7..0000000 --- a/modules/ROOT/pages/making-modules-policies.adoc +++ /dev/null @@ -1,3 +0,0 @@ -= Policies Regarding Modules in Fedora - -NOTE: This page needs to be written \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules.adoc b/modules/ROOT/pages/making-modules.adoc deleted file mode 100644 index e467a44..0000000 --- a/modules/ROOT/pages/making-modules.adoc +++ /dev/null @@ -1,5 +0,0 @@ -= Making Modules for Fedora - -If you wish to maintain an additional version of an application, language stack, or any other piece of software, modules might be the way to go for you. - -The xref:making-modules/adding-new-modules.adoc[Adding New Modules] covers the whole process of adding a new module to Fedora. \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules/adding-new-modules.adoc b/modules/ROOT/pages/making-modules/adding-new-modules.adoc deleted file mode 100644 index 54d201a..0000000 --- a/modules/ROOT/pages/making-modules/adding-new-modules.adoc +++ /dev/null @@ -1,98 +0,0 @@ -= Adding new modules to Fedora - -This page will guide you through the whole process of adding a new module to Fedora: - -1. **<>** — To `dist-git/rpms` using stream branches. -2. **<>** — To `dist-git/modules` using stream branches. -3. **<>** — Module is built as a unit. No individual package builds are done. -4. **<>** — Submitting a Bodhi update. - -== RPM Sources - -For each package in your module, you need to have a Fedora Distribution Git (dist-git) **repository** under `dist-git/rpms` with an appropriate **stream branch**. New packages need to go through a **package review**. - -=== Repositories and Stream Branches — New packages - -The https://fedoraproject.org/wiki/Package_Review_Process#Contributor[Fedora Package Review Process] covers all the steps you need to take, except for stream branches. For that, please continue below to Existing packages. - -=== Repositories and Stream Branches — Existing packages -Requesting new stream branches for existing packages is done with `fedpkg`, and it doesn't require a package review. - -To create a new branch for a package that shares a name with the module (e.g. "postgresql"), run the following command which will create the package stream branch as well as the module and its stream branch together: - ----- -$ fedpkg request-branch --repo=NAME --sl rawhide:2020-12-01 -- BRANCH ----- -* `NAME` — name of the package and the module -* `BRANCH` — name of the stream branch of the package and the module -* `rawhide:2020-12-01` — expected end of life, must end with either `12-01` or `06-01`. Right now, this is not actively used to do anything useful, although some systems might block you from maintaining your module. Please choose a date far in the future so you're not affected. [Work to fix this is in progress](https://tree.taiga.io/project/modularity-wg/epic/3). - -If this package does not share a name with the module (such as for dependencies), then only request the stream branch for this package: - ----- -$ fedpkg request-branch --repo=NAME --no-auto-module --sl rawhide:2020-12-01 -- BRANCH ----- -* `--no-auto-module` — Skip the creation of a module of the same name - -Creation of the module and its stream branches separately will be covered later. - -=== Importing the RPM sources into dist-git - -The last thing to do is to import the RPM sources into dist-git. Please refer to the https://fedoraproject.org/wiki/New_package_process_for_existing_contributors[New package process for existing contributors -] for the complete steps. - -== Module Definition - -For your module definition, you also need to have a dist-git **repository**, this time under `dist-git/modules`, with an appropriate **stream branch**. - -=== Repositories and Stream Branches — New modules - -To request a new module repository in dist-git run: - ----- -$ fedpkg request-repo --namespace modules --exception NAME -$ fedpkg request-branch --namespace modules --repo NAME BRANCH ----- - -* `NAME` — name of the module -* `BRANCH` — name of the stream branch of the module - -=== Repositories and Stream Branches — Existing modules - -to request a new stream branch, run: - ----- -$ fedpkg request-branch --namespace modules --repo NAME BRANCH ----- - -* `NAME` — name of the module -* `BRANCH` — name of the stream branch of the module - -=== Write and push the modulemd - -Writing a modulemd is covered in detail in the xref:making-modules/defining-modules.adoc[Defining Modules in modulemd] section. - -NOTE: The name of the modulemd must match the name of the repository. Remember to replace every `NAME` in the following example. - ----- -$ fedpkg clone modules/NAME -$ cd NAME -$ fedpkg switch-branch BRANCH -$ vim NAME.yaml -$ git add NAME.yaml -$ git commit -m "add the initial modulemd" -$ git push ----- - -* `NAME` — name of the module -* `BRANCH` — name of the stream branch of the module - -== Module Build - -NOTE: NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. - -Submitting module builds is done using `fedpkg` and is covered in the xref:making-modules/building-modules.adoc[Building modules] section. - -== Publishing the Module - -include::{partialsdir}/submit-module-update.adoc[] \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules/building-modules-locally.adoc b/modules/ROOT/pages/making-modules/building-modules-locally.adoc deleted file mode 100644 index 1be0c38..0000000 --- a/modules/ROOT/pages/making-modules/building-modules-locally.adoc +++ /dev/null @@ -1,64 +0,0 @@ -= Building Fedora modules locally - -This page walks you through building modules locally on your machine. - -== Prerequisites - -You need to install the `module-build-service` as well as `fedpkg`: - ----- -$ sudo dnf install module-build-service fedpkg ----- - -and add yourself to the `mock` group: - ----- -$ sudo usermod -a -G mock USERNAME -$ newgrp ----- - -== Step 1: Build your module - -Local builds are started using `fedpkg` from within your dist-git repository. - -For example, to submit a build of the `testmodule:master` module, run: - -NOTE: Local builds don’t support stream expansion. If your module depends on multiple streams of another module, such as `platform`, you need to specify what stream you want to build against. For example: `fedpkg module-build-local -s platform:f28`. - ----- -$ fedpkg clone modules/testmodule -$ cd testmodule -$ git checkout master -$ fedpkg module-build-local ----- - -When the build finishes, you'll be able to find the results in: - ----- -~/modulebuild/builds/MODULE/results/ ----- - -where MODULE is "module-NAME-STREAM-VERSION", so for example "module-testmodule-master-20180612122805". - -== Step 2: Test your module - -WARNING: This requires Fedora 28 or higher. - -To test your module, add the repository to your system by creating an entry similar to this in `/etc/yum.repos.d/test.repo`: - ----- -[local] -name=My Local Repository -baseurl=file:///home/adam/modulebuild/builds/module-testmodule-master-20180612122805/results/ -gpgcheck=0 -enabled=1 ----- - -Please replace "adam" with your user name, and "module-testmodule-master-20180612122805" with the actual value for your module. - -Now, when you list the available modules on your system, you should be able to see it and install it: - ----- -$ dnf module list -$ dnf module install testmodule:master ----- \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules/building-modules.adoc b/modules/ROOT/pages/making-modules/building-modules.adoc deleted file mode 100644 index 6f933d7..0000000 --- a/modules/ROOT/pages/making-modules/building-modules.adoc +++ /dev/null @@ -1,68 +0,0 @@ -= Submitting module builds in Fedora - -NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. - -Module builds are triggered using `fedpkg` from within your dist-git repository. - ----- -$ fedpkg clone modules/NAME -$ cd NAME -$ fedpkg switch-branch BRANCH -$ fedpkg module-build ----- - -* `NAME` — name of the module -* `BRANCH` — name of the stream branch of the module - -NOTE: Please note the module build ID. You will need it to verify the build state and to publish the module later. - -To watch the state of your module build, run: - ----- -$ fedpkg module-build-watch BUILD_ID ----- - -When the module is in a "ready" state, the build has successfully completed. - -You can also watch the build(s) on Fedora Module build service: - -https://release-engineering.github.io/mbs-ui/modules - -== Module build results - -All the packages built in the module are tagged in Koji tag named as ``module-$name-$stream-$version-$context``. The link to this tag is also available in Fedora Module build service UI mentioned earlier. - -There are also two Koji Builds representing single module build. - -* ``$name-$stream-$version.$context`` - Contains the metadata with final RPMs which will be included in a compose and will be therefore visible by the end user. -* ``$name-devel-$stream-$version.$context`` - Contains the metadata with filtered RPMs which are not normally visible to end user, but might be sometimes useful when building other packages against the module. Generally, in Fedora we do not ship the -devel modules. - -NOTE: You will not find ``$name-devel`` module build in Module Build Service, because this is only another representation of non-devel module build. It is not separate module build on its own and it only exists as Koji build. - -Both Koji module builds contains various metadata files: - -- ``modulemd.src.txt`` - The unchanged input modulemd metadata used to build a module as stored in dist-git. -- ``modulemd.txt`` - The expanded modulemd metadata which was used by Module Build Service to build this particular module. This for example contains the right build requirements based on the Module Stream Expansion. -- ``modulemd.$arch.txt`` - The per-architecture modulemd metadata listing the RPMs which will end up in a compose and therefore be visible to end user. This for example respects filters or multilib modulemd options. - -== Rebuild strategies - -In case you want to control which packages get rebuilt and which get reused, you can enforce a specific rebuild strategy while submitting a build. - -There are different rebuild strategies to choose from: - -* ``all`` — All packages in the module get rebuilt. -* ``only-changed`` — Only packages that have changed since the last successful build get rebuilt. This is the **default** in Fedora. -* ``changed-and-after`` — This one leverages xref:making-modules/defining-modules.adoc#_building_in_a_specific_order_optional[build groups (buildorder)]. Packages that have changed since the last successful build get rebuilt, and also all packages with a buildorder higher than any of the changed ones get rebuilt as well. - -For more detailed information, please see the -https://pagure.io/fm-orchestrator/blob/master/f/docs/REBUILD_STRATEGIES.rst[MBS rebuild strategies documentation]. - -To enforce a specific rebuild strategy, submit the module build with the following command: - ----- -$ fedpkg module-build --optional rebuild_strategy=STRATEGY ----- - -* `STRATEGY` — name of a specific build strategy (listed above) - diff --git a/modules/ROOT/pages/making-modules/defining-modules.adoc b/modules/ROOT/pages/making-modules/defining-modules.adoc deleted file mode 100644 index 155d255..0000000 --- a/modules/ROOT/pages/making-modules/defining-modules.adoc +++ /dev/null @@ -1,477 +0,0 @@ -= Defining modules in modulemd - -Simply put, **modulemd is a file that defines which packages get built for which releases**. It includes a summary and a description, a list of source RPM packages, build information i.e. build order and macros, and usage information i.e. installation profiles and licenses. - -== A typical modulemd example - -A typical modulemd file looks similar to the following examples. Read on for more details about each part of the modulemd file. - -Feel free to copy/paste this example when creating your new module. - -[source,yaml] ----- -document: modulemd -version: 2 -data: - # === Information about this module ================================== - # (Can be copied from the main RPM package, but doesn't need to be) - summary: An example module - description: >- - A module for the demonstration of the metadata format. Also, - the obligatory lorem ipsum dolor sit amet goes right here. - - # === License of this modulemd file ================================== - # (Package licenses will be added automatically by the build system) - license: - module: - - MIT - - # === Modular dependencies =========================================== - # (For which Fedora releases to build?) - dependencies: - - buildrequires: - platform: [] # <- Build for all Fedora releases - requires: - platform: [] # <- Run on all Fedora releases - - # === Module API (optional, but encouraged) ========================== - # (Which packages are API-stable?) - api: - rpms: - - package-one # <- Binary RPM package name - - package-one-extras # <- Binary RPM package name - - package-one-cli # <- Binary RPM package name - - package-one-devel # <- Binary RPM package name - - package-two # <- Binary RPM package name - - # === Installation profiles (optional, but encouraged) =============== - # (Helping users with installation by providing predefined groups) - profiles: - default: # <- Name of the profile - description: A standard installation. - rpms: - - package-one # <- Binary RPM package name - - package-one-extras # <- Binary RPM package name - - package-two # <- Binary RPM package name - cli: # <- Name of the profile - description: A command-line client. - rpms: - - package-one-cli # <- Binary RPM package name - - # === Packages in this module ======================================== - # (Referenced by their dist-git repo name + branch name) - components: - rpms: - first-package: # <- Source RPM package name - ref: 3.0 # <- Branch name in dist-git - rationale: Provides the core functionality. - second-package: # <- Source RPM package name - ref: latest # <- Branch name in dist-git - rationale: Web UI for the first-package. ----- - -== Common modulemd definitions - -These are the common parts of a modulemd file, used in the example above. Advanced definitions, including a complex example of Module Stream Expansion (MSE), are towards the end of this page. - -=== Document header - -Every modulemd starts with these three lines: - -[source,yaml] ----- -document: modulemd -version: 2 -data: - ... <1> ----- - -<1> All the following definitions go here, under _data_. - -=== Information about this module - -Tell users what this module represents by writing a summary and a description. - -[source,yaml] ----- - summary: An example module - description: >- <1> - A module for the demonstration of the metadata format. Also, - the obligatory lorem ipsum dolor sit amet goes right here. ----- - -<1> The `>-` means new line in YAML. Useful for longer blocks of text, such as the description! - - -=== License of this modulemd file - -This is a license of this very modulemd file and it doesn't need to be modified. -The build system adds licenses of all packages to this list automatically. - -[source,yaml] ----- - license: - module: - - MIT <1> ----- - -<1> A license for this modulemd file. Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. - -=== Modular dependencies - -Simply put: For which Fedora releases to build? - -To build your module for all Fedora releases that are actively maintained, use the following definition. For anything more than this, such as building against other modules or requiring other modules during run time, please see the Advanced section below. - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [] - requires: - platform: [] ----- - -=== Installation profiles (optional, but encouraged) - -To help users install your module, define installation profiles. These profiles represent a specific use case of your module. Most modules have at least a default profile. But you can specify more. For example, a database module can have a server and a client profile. - -[source,yaml] ----- - profiles: - default: <1> - description: A standard installation. <2> - rpms: - - package-one <3> - - package-one-extras <3> - - package-two <3> - cli: - description: A command-line client. - rpms: - - package-one-cli - ... <4> ----- - -<1> Name of the profile. - -<2> A quick summary of the profile. - -<3> Binary packages to be installed with this profile. - -<4> List as many profiles as you need. - -=== Module API (optional, but encouraged) - -List all binary RPM packages in your module that you consider to be the main stable feature of the module. Other (unlisted) packages should be considered unsupported, or an implementation detail. - -[source,yaml] ----- - api: - rpms: - - package-one - - package-one-extras - - package-one-cli - - package-one-devel - - package-two ----- - -=== Packages in this module - -List all source SRPM packages this module should include, referenced them by their dist-git repo name + branch name. - -[source,yaml] ----- - components: - rpms: - first-package: <1> - rationale: Provides the core functionality. <2> - ref: 3.0 <3> - second-package: - rationale: Web UI for the first-package. - ref: latest - ... <4> ----- - -<1> Name of the package — maps to a DistGit repository name. - -<2> The reason why is this package here. Mostly for humans. - -<3> DistGit branch, tag, or a commit — so the right version of the package gets included. - -<4> List as many packages as you need. - -== Advanced definitions - -=== References to the upstream (optional) - -You can also provide references to the upstream community, documentation, or to an issue tracker. - -[source,yaml] ----- - references: - community: http://www.example.com/ <1> - documentation: http://www.example.com/ <2> - tracker: http://www.example.com/ <3> ----- - -<1> Upstream community website, if it exists. - -<2> Upstream documentation, if it exists. - -<3> Upstream bug tracker, if it exists. - -=== Building in a specific order (optional) - -Packages are built in batches. By default, all packages are part of a single group, and therefore built concurrently. - -To build packages in a specific order, assign them to multiple build groups. Build groups are identified by an integer. Groups with lower number are built first. Negative values are allowed, `0` is the implicit default value. - -In this specific example, `first-package` gets built first, and `second-package` gets built second. - -[source,yaml] ----- - components: - rpms: - first-package: - rationale: Provides the core functionality. - ref: 3.0 - buildorder: 0 <1> - second-package: - rationale: Web UI for the first-package. - ref: latest - buildorder: 10 <1> ----- - -<1> A number of the build group. - -=== Build macros (optional) - -Use this if you need to build your packages with a specific RPM macro. Applies to all packages in the module. - -[source,yaml] ----- - buildopts: - rpms: - macros: | - %demomacro 1 - %demomacro2 %{demomacro}23 ----- - -=== Advanced dependencies (optional) - -modules can be - -* built against other modules -* require other modules during run time -* built against one or more streams of the same module -* work with one or more streams of another module - -**Building only for Fedora 28**: - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [f28] - requires: - platform: [f28] ----- - -**Building for everything else than Fedora 28**: - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [-f28] - requires: - platform: [-f28] ----- - -**Building only for Fedora 28 and Fedora 29**: - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [f28, f29] - requires: - platform: [f28, f29] ----- - -**Building against other modules:**: - -Your module can also depend on another modules. Specific streams can be referenced the same way as above in "Building for a specific Fedora release(s) only". - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [] - nodejs: [] - requires: - platform: [] - nodejs: [] ----- - -**A complex example:** - -Simple things should simple, complex things should be possible. Let's say my module requires `nodejs` during build and run time. It also requires `build-tools` only during build. To make it even more complex, it also requires a specific stream of a `pizza-module` during build and run time, but only on Fedora 27. - -[source,yaml] ----- - dependencies: - - buildrequires: - platform: [-f27] - buildtools: [v1, v2] - nodejs: [] - requires: - platform: [-f27] - nodejs: [] - - buildrequires: - platform: [f27] - buildtools: [v1, v2] - nodejs: [] - pizza-module: [3.6] - requires: - platform: [f27] - nodejs: [] - pizza-module: [3.6] ----- - -For even more complex scenarios, please study the https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml[modulemd specification]. - - -=== Excluding binary packages (optional) - -One source RPM package might produce multiple binary RPM packages. If you don't want to include some binary packages, list them under `filter`. - -Filtered RPMs are still available to use as build dependencies in subsequent stages of the module build, but are not included in the composed repository for users. - -[source,yaml] ----- - filter: - rpms: - - first-package-debuginfo - - second-package-nope ----- - -=== Creating build-only components (optional) - -In addition to filtering subpackages, it's possible to filter out all of the artifacts produced by a component in a module. -This is useful in cases where your module's primary packages have a build-time dependency that you do not want to ship. -An example of such a case would be if you need to build with a specially-patched documentation-generator that would conflict -with the version used as the default in Fedora. - -[source,yaml] ----- - components: - rpms: - customdocgen: - rationale: A patched version of docgen that enables an experimental feature. - ref: experimental - buildorder: 0 - buildonly: 1 - myapp: - rationale: My application - ref: latest - buildorder: 10 ----- - -In this example, `customdocgen` would be built first and made available in the buildroot for `myapp` to use during its build. -Once the module build is finished and it is composed into a DNF repository, only the unfiltered artifacts from `myapp` will be available. -All of the `customdocgen` artifacts will be automatically added to the `data.filters.rpms` section of the module metadata. - -== A minimal modulemd - -=== An absolute minimum - -This module includes two source RPM packages built for all Fedora releases. - -[source,yaml] ----- -document: modulemd -version: 2 -data: - summary: An example module - description: >- - A module for the demonstration of the metadata format. Also, - the obligatory lorem ipsum dolor sit amet goes right here. - license: - module: - - MIT - dependencies: - - buildrequires: - platform: [] - requires: - platform: [] - components: - rpms: - first-package: - rationale: Provides the core functionality. - ref: 3.0 - second-package: - rationale: Web UI for the first-package. - ref: latest ----- - -=== Including profiles and API (recommended) - -This module includes two source RPM packages built for all Fedora releases. It makes clear which packages are considered the API, and helps users with installation thanks to the profiles. - -[source,yaml] ----- -document: modulemd -version: 2 -data: - summary: An example module - description: >- - A module for the demonstration of the metadata format. Also, - the obligatory lorem ipsum dolor sit amet goes right here. - license: - module: - - MIT - dependencies: - - buildrequires: - platform: [] - requires: - platform: [] - api: - rpms: - - package-one - - package-one-extras - - package-one-cli - - package-one-devel - - package-two - profiles: - default: - description: A standard installation. - rpms: - - package-one - - package-one-extras - - package-two - cli: - description: A command-line client. - rpms: - - package-one-cli - components: - rpms: - first-package: - rationale: Provides the core functionality. - ref: 3.0 - second-package: - rationale: Web UI for the first-package. - ref: latest ----- - -== Using fedmod - -To get started quickly, you can use the `fedmod` tool (from the package of the same name) to create a skeletal modulemd file for your module based on one or more source packages, e.g. - -[source,shell] ----- -$ fedmod fetch-metadata -$ fedmod rpm2module dwm > dwm.yaml ----- - -If you specify more than one package it will fill in the summary and description fields from their counterparts in the first specified package. diff --git a/modules/ROOT/pages/making-modules/inspecting-build-failures.adoc b/modules/ROOT/pages/making-modules/inspecting-build-failures.adoc deleted file mode 100644 index 122e5a5..0000000 --- a/modules/ROOT/pages/making-modules/inspecting-build-failures.adoc +++ /dev/null @@ -1,15 +0,0 @@ -= Inspecting module build failures in Fedora - -Have you submitted a build which failed? This page will point you to the right places to get all the information about build failures. - -== Inspecting build failures - -Run the following command using the BUILD_ID from the previous step: - -NOTE: Reminder: You got the BUILD_ID when you xref:building-modules.adoc[submitted a module build]. - ----- -$ fedpkg module-build-info BUILD_ID ----- - -which shows you "State Reason" — a short summary of the build failure regarding the whole module, as well as a list of individual packages in the module. Find the ones that failed and go to the "Koji Task" URL. There, click on the "Build" link which gets you to the module build page. On this page, inspect the `root.log`, `build.log`, and other files to get the build error. diff --git a/modules/ROOT/pages/making-modules/managing-defaults.adoc b/modules/ROOT/pages/making-modules/managing-defaults.adoc deleted file mode 100644 index 25caa9d..0000000 --- a/modules/ROOT/pages/making-modules/managing-defaults.adoc +++ /dev/null @@ -1,23 +0,0 @@ -= Managing module defaults in Fedora - -Setting or changing a default stream or a default installation profile of a module constitutes a major behavior change as defined in the Fedora Updates Policy. The following rules apply: - -1. Module stream defaults MUST be only changed in an upcoming Fedora release -2. Changes of stream defaults should be communicated by a Fedora Change based on the change's significance and its maintainer's best judgement. When in doubt, file a Change. -3. Changes of the default stream of a module are not permitted within a released Fedora without the approval of FESCo. -4. Introducing a new default stream not replacing any existing default stream or a traditional package is not considered a change. That means it can be done. - -== Setting or changing a default - -Submit an issue to the Fedora Engineering Steering Committee in https://pagure.io/fesco/issues[pagure.io/fesco]. This ticket must provide the following information: - -* The list of SRPM components in the module -* Whether the contents of this module will obsolete and replace non-modular RPMs. -* Which releases of Fedora will this new default apply to. - - -When requesting a change of default stream, strongly consider https://fedoraproject.org/wiki/Changes/Policy#For_developers[submitting a Fedora Change]. This will help ensure that the change is communicated to the rest of the Fedora community. - -To check the current defaults, have a look at the https://pagure.io/releng/fedora-module-defaults[fedora module defaults repository]. - -Please note that if the module stream masks part of the Traditional RPM repos (e.g it replaces an existing RPM or it introduces a non-trivial set of conflicts) it may not be made a default stream without the express permission of FESCo. Release Engineering will be responsible for escalating any PR that is questionable on this point to FESCo for a final decision. diff --git a/modules/ROOT/pages/making-modules/naming-guidelines.adoc b/modules/ROOT/pages/making-modules/naming-guidelines.adoc deleted file mode 100644 index b4b406a..0000000 --- a/modules/ROOT/pages/making-modules/naming-guidelines.adoc +++ /dev/null @@ -1,75 +0,0 @@ -= Modularity Naming Guidelines - -Each module has a **name**, and one or more **streams** that usually represent a major version of the application, language stack, or other software in the module. To simplify their installation, modules can define **profiles** representing a specific use case. Modules are built out of SRPM packages that have a name and one or more branches. - -**Example 1:** A Node.js module could be named "nodejs", with three streams "6", "8" and "10", and two profiles "default" and "devel". This module would include a single package in each stream named "nodejs", branched as "6", "8", and "10". - -**Example 2:** Another one, PostgreSQL, could be named as "postgresql", its two streams could be "9.6" and "10", because that's where the upstream compatibility is, and its profiles could include "client" and "server". Both streams could be built out of two packages. One named "postgresql", having branches "9.6" and "10", which would be the main package; and another one named "python-pgsql", having a single branch "latest", which would always bind to the right version of the database during build, and would provide Python bindings. - -== Module name -A module usually represents an application, a language stack, or any other logical collection of packages. Module name should represent the name of the software it ships. Using lowercase, hyphen-separated names is preferable. Some examples could include "nodejs", "golang", "golang-ecosystem", or "cri-o". - -A module name maps to its repository name in DistGit. - -== Module stream name - -The module stream name is assigned by creating a branch of that name in DistGit and building from it. The Module Build Service will automatically set the stream name of the resulting module to match. - -=== Option 1: Major version - -A stream usually represents a major version of the software, and should follow the versioning of the major upstream component included in the module. Consideration should be given to the upstream community’s adherence to API (or even ABI) stability on the versions of their software. In other words, some communities will maintain stability on the X branch, many the Y branch, and some even the Z branch. As a result, the branch names should reflect that stability. For example, most communities are stable on the Y branch so the branch name should be "X.Y". (e.g. mariadb:10.2) included in the module. - -Compatibility on non-technical factors should be also considered. For example, a package that maintains exactly the same API but has a significant visual and UX overhaul probably belongs in a new stream. Or to put it another way, human interaction is an interface too. - -=== Option 2: CalVer - -If your module is a "collection of items" with no primary piece of software (e.g. container-tools), the versioning scheme should use the CalVer (http://calver.org/) scheme with YYYY.MINOR where the YYYY is the year of the release and MINOR version is an integer starting at 0. In other words, if I released a system-tools module in November of 2017 we would expect to see 2017.0. My next release, in May of 2018, which does not break ABI/API, is also 2017.0. However, in October of 2018 we want to introduce a new stream so we name it 2018.0. The month is omitted as ideally these modules won’t be breaking API/ABI more than once a year. - -=== Option 3: Rolling and unstable streams - -For streams that don't guarantee an API/ABI stability over time and just roll forward, we suggest packagers use one of the following stream names. - -* "rolling" for user-focused content meant for general use -* "unstable" for pre-release content or content in active development meant for preview and testing rather than general use - -We mandate a good use of the description and/or summary field to clearly state what this particular stream represents. - -This policy doesn't enforce changes in existing branch names, however, packagers are welcome to do so in order to bring more consistency to the distribution. - -=== Option 4: Upstream conventions - -In case there are well-established upstream conventions, using them is also a good option. - -Again, especially in this case, we mandate a good use of the description and/or summary field to clearly state what this particular stream represents. - -== Module profile name and description - -Profiles make installation easier by giving the user some predefined package groups that represent a common installation for a specific use case. Profiles have name and a description — using both is mandatory. - -Profile name should be a one word that best represent the use case. Some examples could include "client" or "server" for database modules, "default" or "devel" for language runtime modules, but also "minimal" and others. There some reserved profile names for specific purposes e.g. setting up a buildroot. Please see the https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml[modulemd spec] for a complete list. - -Descriptions should be a short summary describing the profile. Including it is important especially for potential translations, as profile names won't be translated. - -=== Unified profile names - -In some cases, there are many suitable alternatives for for one meaning i.e. "dev", "devel", and "development". We believe that choosing one that would be consistently used across all modules that need such profile would lead to a better user experience. The ones identified are listed below: - -* "default" a default profile for a "standard" installation, should the module have something like this -* "devel" for development-related profiles -* "client" only the client component of a piece of software (e.g. database client) -* "server" only the server component of a piece of software (e.g. database server) - -== Package name - -No changes for package names with Modularity. Please refer to the official https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines[Guidelines for Naming Fedora Packages]. - -== Package branch name - -Packages included in modules should be branched in a similar manner as modules. So instead of having one branch for release (such as "f27", "f28, etc.), using upstream major versions as branches is recommended. Please read the Module stream name section above for guidance. - -The exact branching should be chosen with consideration given to the upstream community’s adherence to API/ABI stability on the versions of their software. In other words, in the context of X.Y.Z style naming, some communities will maintain stability on the X branch, many the Y branch, and some even the Z branch. As a result, the branch names should reflect that stability. For example, most communities are stable on the Y branch so the branch name should be "X.Y". While this rule is focusing on examples using X.Y.Z style naming, CalVer (http://calver.org/), or other versioning schemes, should be faithfully reflected using the same guidance regarding which branches to create. - -For "rolling release" projects that don’t change their API/ABI (e.g. Python’s timezone database, `pytz`) a single "stable" branch is appropriate. - - - diff --git a/modules/ROOT/pages/making-modules/packaging-guidelines.adoc b/modules/ROOT/pages/making-modules/packaging-guidelines.adoc deleted file mode 100644 index b0f1556..0000000 --- a/modules/ROOT/pages/making-modules/packaging-guidelines.adoc +++ /dev/null @@ -1,133 +0,0 @@ -= Fedora Packaging Guidelines for Modules - -These guidelines specify restrictions and guidance for xref:defining-modules.adoc[defining modules in modulemd]. - -== Short Overview - -* Packager **MUST** specify `summary` -* Packager **MUST** specify `description` -* Packager **MUST** specify `license/module` license(s) based on allowed licenses in Fedora -* Packager **MUST** specify `dependencies` -* Packager **MUST** specify `components` -* Packager **SHOULD** specify `profiles` -* Packager **SHOULD** specify `api` -* Packager **MAY** specify `data` in key-value format in xmd -* Packager **MAY** specify `references` -* Packager **MAY** specify `filter` -* Packager **MAY** specify `buildopts` -* Packager **MUST NOT** specify `name`, name is taken from name of dist-git repository -* Packager **MUST NOT** specify `stream`, stream is taken from branch of dist-git repository -* Packager **MUST NOT** specify `version`, it is automatically generated by buildsystem -* Packager **MUST NOT** specify `arch`, it is automatically filled in by buildsystem -* Packager **MUST NOT** specify `servicelevels`, they are automatically filled in by buildsystem from PDC -* Packager **MUST NOT** specify `license/content` license(s), they are picked up from components which are built -* Packager **MUST NOT** specify `artifacts` - -== Long Overview - -=== Document header and the data block - -Every modulemd file **MUST** contain a modulemd document header which consists of the document type tag and the document format version, and a data block holding the module data. - ----- -document: modulemd -version: 2 -data: - (...) ----- - -The version is an integer and denotes the version of the metadata format the rest of the document is written in. Packagers **SHOULD** use latest available version of specification. - -=== Summary and description - -Every module **MUST** include human-readable short summary and description. Both should be written in US English. - ----- -summary: An example module -description: >- - An example long description of an example module, written just - to demonstrate the purpose of this field. ----- - -The summary is a one sentence concise description of the module and **SHOULD NOT** end in a period. - -The description expands on this and **SHOULD** end in a period. Description **SHOULD NOT** contain installation instructions or configuration manuals. - -=== Licensing - -Every module **MUST** contain a license block and declare a list of the module's licenses. Note these aren't the module's components' licenses. - ----- -license: - module: - - MIT ----- - -Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. Therefore MIT might be a reasonable default for most module authors as well. - -=== Dependencies - -Modules **MAY** depend on other modules. These module relationships are listed in the dependencies block. However, none of modules are buildable without `platform` module hence packagers are required to specify it. - -Packagers **SHOULD** be using stream name expansion to make their modules compatible with other existing modules. The following example builds and runs the module on any `platform:` - ----- -dependencies: - - buildrequires: - platform: [] - requires: - platform: [] ----- - -Note the stream name expansion is only supported by the second version of the format. Unlike in the first version, the `dependencies` block is a list of dictionaries. - -=== Extensible module metadata block - -Modules **MAY** also contain an extensible metadata block, a list of vendor-defined key-value pairs. - ----- -xmd: - user-defined-key: 42 - another-user-defined-key: - - the first value of the list - - the second value of the list ----- - -=== References - -Modules **MAY** define links referencing various upstream resources, such as community website, project documentation or upstream bug tracker. - ----- -references: - community: http://www.example.com/ - documentation: http://www.example.com/docs/1.23/ - tracker: http://www.example.com/bugs/ ----- - -=== Profiles - -The module author **MAY** define lists of packages that would be installed when the module is enabled and the particular profile is selected. Whether the packages actually get installed depends on the user's configuration. It is possible to define a profile that doesn't install any packages. - -Profile names are arbitrary strings. There are currently few special-purpose profile names defined, see specification for details. - -If the module includes one or more profile definition, xref:managing-defaults.adoc[module defaults] **SHOULD** also be defined. - -In the case of RPM content, the profile package lists reference binary RPM package names. - -=== API - -Module API are components, symbols, files or abstract features the module explicitly declares to be its supported interface. Everything else is considered an internal detail and shouldn't be relied on by any other module. - -Every module **SHOULD** define its public API. - -=== Filters - -Module filters define lists of components or other content that should not be part of the resulting, composed module deliverable. They can be used to only ship a limited subset of generated RPM packages, for instance. - ----- -filter: - rpms: - - mypackage-plugins ----- - -Currently the only supported type of filter are binary RPM packages. \ No newline at end of file diff --git a/modules/ROOT/pages/making-modules/updating-modules.adoc b/modules/ROOT/pages/making-modules/updating-modules.adoc deleted file mode 100644 index 7c3e6a7..0000000 --- a/modules/ROOT/pages/making-modules/updating-modules.adoc +++ /dev/null @@ -1,42 +0,0 @@ -= Updating modules in Fedora - -This page will guide you through the process of updating an existing module. - -1. **<>** — Pushing new sources. -2. **<>** — Pushing a new version of the modulemd. -3. **<>** — Module is built as a unit. No individual package builds are done. -4. **<>** — Submitting a Bodhi update. - -== Updating RPM Packages - -Update your RPM packages the same way you would do traditionally, except submitting individual package builds. - -Useful resources: - -* https://fedoraproject.org/wiki/Packaging:Guidelines[Fedora Packaging Guidelines] -* https://fedoraproject.org/wiki/Package_maintenance_guide[Fedora Package Maintenance Guide] - -== Updating the Module - -Even when you don't need to make any changes to the modulemd, you still need to push a new commit to build a new version. - ----- -$ fedpkg clone modules/NAME -$ cd NAME -$ fedpkg switch-branch BRANCH -$ git commit --allow-empty -m "update" -$ git push ----- - -* `NAME` — name of the module -* `BRANCH` — name of the stream branch of the module - -== Module Build - -NOTE: NOTE: With Modularity, you no longer build individual packages. Instead, you need to submit a module build. - -Submitting module builds is done using `fedpkg` and is covered in the xref:making-modules/building-modules.adoc[Building modules] section. - -== Publishing the Module - -include::{partialsdir}/submit-module-update.adoc[] \ No newline at end of file diff --git a/modules/ROOT/pages/policies/index.adoc b/modules/ROOT/pages/policies/index.adoc new file mode 100644 index 0000000..2c171c7 --- /dev/null +++ b/modules/ROOT/pages/policies/index.adoc @@ -0,0 +1,3 @@ += Policies Regarding Modules in Fedora + +NOTE: This page needs to be written \ No newline at end of file diff --git a/modules/ROOT/pages/policies/naming-guidelines.adoc b/modules/ROOT/pages/policies/naming-guidelines.adoc new file mode 100644 index 0000000..b4b406a --- /dev/null +++ b/modules/ROOT/pages/policies/naming-guidelines.adoc @@ -0,0 +1,75 @@ += Modularity Naming Guidelines + +Each module has a **name**, and one or more **streams** that usually represent a major version of the application, language stack, or other software in the module. To simplify their installation, modules can define **profiles** representing a specific use case. Modules are built out of SRPM packages that have a name and one or more branches. + +**Example 1:** A Node.js module could be named "nodejs", with three streams "6", "8" and "10", and two profiles "default" and "devel". This module would include a single package in each stream named "nodejs", branched as "6", "8", and "10". + +**Example 2:** Another one, PostgreSQL, could be named as "postgresql", its two streams could be "9.6" and "10", because that's where the upstream compatibility is, and its profiles could include "client" and "server". Both streams could be built out of two packages. One named "postgresql", having branches "9.6" and "10", which would be the main package; and another one named "python-pgsql", having a single branch "latest", which would always bind to the right version of the database during build, and would provide Python bindings. + +== Module name +A module usually represents an application, a language stack, or any other logical collection of packages. Module name should represent the name of the software it ships. Using lowercase, hyphen-separated names is preferable. Some examples could include "nodejs", "golang", "golang-ecosystem", or "cri-o". + +A module name maps to its repository name in DistGit. + +== Module stream name + +The module stream name is assigned by creating a branch of that name in DistGit and building from it. The Module Build Service will automatically set the stream name of the resulting module to match. + +=== Option 1: Major version + +A stream usually represents a major version of the software, and should follow the versioning of the major upstream component included in the module. Consideration should be given to the upstream community’s adherence to API (or even ABI) stability on the versions of their software. In other words, some communities will maintain stability on the X branch, many the Y branch, and some even the Z branch. As a result, the branch names should reflect that stability. For example, most communities are stable on the Y branch so the branch name should be "X.Y". (e.g. mariadb:10.2) included in the module. + +Compatibility on non-technical factors should be also considered. For example, a package that maintains exactly the same API but has a significant visual and UX overhaul probably belongs in a new stream. Or to put it another way, human interaction is an interface too. + +=== Option 2: CalVer + +If your module is a "collection of items" with no primary piece of software (e.g. container-tools), the versioning scheme should use the CalVer (http://calver.org/) scheme with YYYY.MINOR where the YYYY is the year of the release and MINOR version is an integer starting at 0. In other words, if I released a system-tools module in November of 2017 we would expect to see 2017.0. My next release, in May of 2018, which does not break ABI/API, is also 2017.0. However, in October of 2018 we want to introduce a new stream so we name it 2018.0. The month is omitted as ideally these modules won’t be breaking API/ABI more than once a year. + +=== Option 3: Rolling and unstable streams + +For streams that don't guarantee an API/ABI stability over time and just roll forward, we suggest packagers use one of the following stream names. + +* "rolling" for user-focused content meant for general use +* "unstable" for pre-release content or content in active development meant for preview and testing rather than general use + +We mandate a good use of the description and/or summary field to clearly state what this particular stream represents. + +This policy doesn't enforce changes in existing branch names, however, packagers are welcome to do so in order to bring more consistency to the distribution. + +=== Option 4: Upstream conventions + +In case there are well-established upstream conventions, using them is also a good option. + +Again, especially in this case, we mandate a good use of the description and/or summary field to clearly state what this particular stream represents. + +== Module profile name and description + +Profiles make installation easier by giving the user some predefined package groups that represent a common installation for a specific use case. Profiles have name and a description — using both is mandatory. + +Profile name should be a one word that best represent the use case. Some examples could include "client" or "server" for database modules, "default" or "devel" for language runtime modules, but also "minimal" and others. There some reserved profile names for specific purposes e.g. setting up a buildroot. Please see the https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml[modulemd spec] for a complete list. + +Descriptions should be a short summary describing the profile. Including it is important especially for potential translations, as profile names won't be translated. + +=== Unified profile names + +In some cases, there are many suitable alternatives for for one meaning i.e. "dev", "devel", and "development". We believe that choosing one that would be consistently used across all modules that need such profile would lead to a better user experience. The ones identified are listed below: + +* "default" a default profile for a "standard" installation, should the module have something like this +* "devel" for development-related profiles +* "client" only the client component of a piece of software (e.g. database client) +* "server" only the server component of a piece of software (e.g. database server) + +== Package name + +No changes for package names with Modularity. Please refer to the official https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines[Guidelines for Naming Fedora Packages]. + +== Package branch name + +Packages included in modules should be branched in a similar manner as modules. So instead of having one branch for release (such as "f27", "f28, etc.), using upstream major versions as branches is recommended. Please read the Module stream name section above for guidance. + +The exact branching should be chosen with consideration given to the upstream community’s adherence to API/ABI stability on the versions of their software. In other words, in the context of X.Y.Z style naming, some communities will maintain stability on the X branch, many the Y branch, and some even the Z branch. As a result, the branch names should reflect that stability. For example, most communities are stable on the Y branch so the branch name should be "X.Y". While this rule is focusing on examples using X.Y.Z style naming, CalVer (http://calver.org/), or other versioning schemes, should be faithfully reflected using the same guidance regarding which branches to create. + +For "rolling release" projects that don’t change their API/ABI (e.g. Python’s timezone database, `pytz`) a single "stable" branch is appropriate. + + + diff --git a/modules/ROOT/pages/policies/packaging-guidelines.adoc b/modules/ROOT/pages/policies/packaging-guidelines.adoc new file mode 100644 index 0000000..b0f1556 --- /dev/null +++ b/modules/ROOT/pages/policies/packaging-guidelines.adoc @@ -0,0 +1,133 @@ += Fedora Packaging Guidelines for Modules + +These guidelines specify restrictions and guidance for xref:defining-modules.adoc[defining modules in modulemd]. + +== Short Overview + +* Packager **MUST** specify `summary` +* Packager **MUST** specify `description` +* Packager **MUST** specify `license/module` license(s) based on allowed licenses in Fedora +* Packager **MUST** specify `dependencies` +* Packager **MUST** specify `components` +* Packager **SHOULD** specify `profiles` +* Packager **SHOULD** specify `api` +* Packager **MAY** specify `data` in key-value format in xmd +* Packager **MAY** specify `references` +* Packager **MAY** specify `filter` +* Packager **MAY** specify `buildopts` +* Packager **MUST NOT** specify `name`, name is taken from name of dist-git repository +* Packager **MUST NOT** specify `stream`, stream is taken from branch of dist-git repository +* Packager **MUST NOT** specify `version`, it is automatically generated by buildsystem +* Packager **MUST NOT** specify `arch`, it is automatically filled in by buildsystem +* Packager **MUST NOT** specify `servicelevels`, they are automatically filled in by buildsystem from PDC +* Packager **MUST NOT** specify `license/content` license(s), they are picked up from components which are built +* Packager **MUST NOT** specify `artifacts` + +== Long Overview + +=== Document header and the data block + +Every modulemd file **MUST** contain a modulemd document header which consists of the document type tag and the document format version, and a data block holding the module data. + +---- +document: modulemd +version: 2 +data: + (...) +---- + +The version is an integer and denotes the version of the metadata format the rest of the document is written in. Packagers **SHOULD** use latest available version of specification. + +=== Summary and description + +Every module **MUST** include human-readable short summary and description. Both should be written in US English. + +---- +summary: An example module +description: >- + An example long description of an example module, written just + to demonstrate the purpose of this field. +---- + +The summary is a one sentence concise description of the module and **SHOULD NOT** end in a period. + +The description expands on this and **SHOULD** end in a period. Description **SHOULD NOT** contain installation instructions or configuration manuals. + +=== Licensing + +Every module **MUST** contain a license block and declare a list of the module's licenses. Note these aren't the module's components' licenses. + +---- +license: + module: + - MIT +---- + +Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. Therefore MIT might be a reasonable default for most module authors as well. + +=== Dependencies + +Modules **MAY** depend on other modules. These module relationships are listed in the dependencies block. However, none of modules are buildable without `platform` module hence packagers are required to specify it. + +Packagers **SHOULD** be using stream name expansion to make their modules compatible with other existing modules. The following example builds and runs the module on any `platform:` + +---- +dependencies: + - buildrequires: + platform: [] + requires: + platform: [] +---- + +Note the stream name expansion is only supported by the second version of the format. Unlike in the first version, the `dependencies` block is a list of dictionaries. + +=== Extensible module metadata block + +Modules **MAY** also contain an extensible metadata block, a list of vendor-defined key-value pairs. + +---- +xmd: + user-defined-key: 42 + another-user-defined-key: + - the first value of the list + - the second value of the list +---- + +=== References + +Modules **MAY** define links referencing various upstream resources, such as community website, project documentation or upstream bug tracker. + +---- +references: + community: http://www.example.com/ + documentation: http://www.example.com/docs/1.23/ + tracker: http://www.example.com/bugs/ +---- + +=== Profiles + +The module author **MAY** define lists of packages that would be installed when the module is enabled and the particular profile is selected. Whether the packages actually get installed depends on the user's configuration. It is possible to define a profile that doesn't install any packages. + +Profile names are arbitrary strings. There are currently few special-purpose profile names defined, see specification for details. + +If the module includes one or more profile definition, xref:managing-defaults.adoc[module defaults] **SHOULD** also be defined. + +In the case of RPM content, the profile package lists reference binary RPM package names. + +=== API + +Module API are components, symbols, files or abstract features the module explicitly declares to be its supported interface. Everything else is considered an internal detail and shouldn't be relied on by any other module. + +Every module **SHOULD** define its public API. + +=== Filters + +Module filters define lists of components or other content that should not be part of the resulting, composed module deliverable. They can be used to only ship a limited subset of generated RPM packages, for instance. + +---- +filter: + rpms: + - mypackage-plugins +---- + +Currently the only supported type of filter are binary RPM packages. \ No newline at end of file