From 3dc869e80c0921288fcffa54b1f3920da12cb534 Mon Sep 17 00:00:00 2001 From: Otto Urpelainen Date: Nov 10 2021 19:21:08 +0000 Subject: [PATCH 1/2] Import GNU Hello packaging tutorial Quick Docs have some pages about RPM Packaging. This material goes better into Package Maintainer Docs. The pages are out of date, thus they also need to be updated to match todays packaging practices. Updating all of them in one to go would be a quite large undertaking, so instead, just one page is updated and imported here. The rest can be handled later. --- diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 1c81e3f..78de237 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,15 +1,29 @@ * xref:Joining_the_Package_Maintainers.adoc[Joining the Package Maintainers] ** xref:How_to_Get_Sponsored_into_the_Packager_Group.adoc[How to Get Sponsored into the Packager Group] + * xref:How_to_Sponsor_a_New_Contributor.adoc[How to Sponsor a New Contributor] + * New Package Process ** xref:New_Package_Process_for_Existing_Contributors.adoc[for Existing Contributors] ** xref:New_Package_Process_for_New_Contributors.adoc[for New Contributors] + +* Creating RPM packages +** xref:Packaging_Tutorial_GNU_Hello.adoc[Packaging Tutorial: GNU Hello] + * xref:Package_Review_Process.adoc[Package Review Process] + * xref:Package_Renaming_Process.adoc[Package Renaming Process] + * xref:Package_Orphaning_Process.adoc[Package Orphaning Process] + * xref:Package_Retirement_Process.adoc[Package Retirement Process] + * xref:Package_Maintenance_Guide.adoc[Package Maintenance Guide] + * xref:Package_Update_Guide.adoc[Package Update Guide] + * xref:Staying_Close_to_Upstream_Projects.adoc[Staying Close to Upstream Projects] + * xref:Using_the_Koji_Build_System.adoc[Using the Koji Build System] + * xref:Upstream_Release_Monitoring.adoc[Upstream Release Monitoring] diff --git a/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc b/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc new file mode 100644 index 0000000..149e405 --- /dev/null +++ b/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc @@ -0,0 +1,393 @@ +include::{partialsdir}/attributes.adoc[] + += Packaging Tutorial: GNU Hello +:toc: + +This tutorial demonstrates RPM packaging +by packaging the https://www.gnu.org/software/hello/[GNU Hello] program. +While the program itself is simple, +it also comes with most of the usual peripheral components of a FOSS project: +configuration/build/install environment, +documentation, +internationalization, +etc. +However, it does not include RPM packaging information, +therefore it is a reasonable vehicle to practice building RPMs on. + +For comprehensive information on how to create RPM files, +including more detailed tips, +refer to https://docs.fedoraproject.org/en-US/quick-docs/creating-rpm-packages/[Creating RPM Packages]. +If you plan to create an RPM package for the Fedora repository, +follow the process for xref:Joining_the_Package_Maintainers.adoc[Joining the Package Maintainers], +including following the various Fedora guidance. + +[#dev_env] +== Development Environment + +To build RPMs we need a set of development tools. + +This is a one-time-only setup, +installed by running these commands from a system administration (root) account: + +---- +# dnf install fedora-packager @development-tools +---- + +To be able to test the build procedure in a clean chroot +you need to configure your non-privileged account to be a member of the `mock` group: + +---- +# usermod -a -G mock +---- + +Those are the only commands requiring root privileges. +All the remaining work should be done from your regular, non-privileged account, +or even from a separate account created just for development work. +Modern RPM-based systems, including Fedora, +are set up to build and test RPM packages purely from within a non-privileged account. +The following command sets up an RPM build area in your `+~/rpmbuild+` directory. +This directory will contain several subdirectories, +for the project source code, +RPM configuration files +and for the resulting source and binary packages. + +---- +$ rpmdev-setuptree +---- + +[#building] +== Building the RPM + +We need the source code of the project we are packaging, +often referred to as the _upstream_ source. +We will download it from the project's website +into the `+~/rpmbuild/SOURCES+` directory. +We are getting the compressed tarball archive, +which happens to be the preferred distribution form for most FOSS projects. + +---- +$ cd ~/rpmbuild/SOURCES +$ wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz +---- + +The RPM package is configured by `.spec` files. +We will create a template file `hello.spec` in the appropriate directory: + +---- +$ cd ~/rpmbuild/SPECS +$ rpmdev-newspec hello +---- + +Recent versions of `Emacs` and `vi` have `.spec` file editing modes +which will also bring up a similar template upon creating a new file. +So you can just use the following command +for example to use the template automatically. + +---- +$ emacs hello.spec +---- + +[#inside_spec] +== Inside a Spec File + +The fields in our `.spec` file need slight editing. +Please follow https://docs.fedoraproject.org/en-US/quick-docs/creating-rpm-packages/[Creating RPM Packages] for these fields. +In our case, the file might start as follows: + +[source, RPMSpec] +---- +Name: hello +Version: 2.10 +Release: 1%{?dist} +Summary: Produces a familiar, friendly greeting +License: GPLv3+ +URL: https://www.gnu.org/software/hello/ +Source0: https://ftp.gnu.org/gnu/hello/hello-%{version}.tar.gz + +%description +The GNU Hello program produces a familiar, friendly greeting. Yes, this is +another implementation of the classic program that prints “Hello, world!” when +you run it. + +%prep +%autosetup + +%build +%configure +%make_build + +%install +%make_install + +%files + +%changelog +* Sat Oct 23 2021 The Coon of Ty - 2.10-1 +- Initial version of the package +---- + +The `+Version+` should mirror the upstream, +while `+Release+` numbers our work within Fedora. + +The first letter of the `+Summary+` should be uppercase +to avoid `+rpmlint+` complaints. + +Often, `+Summary+` and `+%description+` can be copied from the upstream README. + +It is your responsibility to check the `+License+` status of the software. +Inspect the source files and their `+LICENSE+` files, +and talk to the authors as needed. + +The `+%changelog+` should document the work on preparing the RPM, +especially if there are security and bug patches +included on top of the base upstream source. +Changelog data can be displayed by `+rpm --changelog -q PACKAGE_NAME+`, +which is very useful, for instance, +to find out if specific bug and security patches were included in the installed software, +thanks to the diligent Fedora packagers who include this info +with the relevant https://cve.mitre.org/[CVE] numbers. + +The `+%changelog+` entry should include the version string +to avoid `+rpmlint+` complaints. + +Multi-line sections like `+%changelog+` or `+%description+` +start on a line under the directive, +and end when the next section starts or the file ends. + +Lines which are not needed (e.g. `+BuildRequires+` and `+Requires+`) +can be commented out with the hash `+#+` for now. + +In many cases, +many lines in the template do not need to be changed at all, +at least for the initial attempt. + +== Building the Package + +We are ready for the first run to build source, binary and debugging packages: + +---- +$ rpmbuild -ba hello.spec +---- + +It will complain and list the unpackaged files, +i.e. the files that would be installed in the system, +but were not declared as belonging to the package. +We need to declare them in the `+%files+` section. +This is an iterative process; +after declaring a missing file in the `+.spec+` file, +rerun `+rpmbuild+`. + +We will go through the file list one by one. + +=== Executable + +---- +Installed (but unpackaged) file(s) found: +/usr/bin/hello +---- + +This is the executable binary program. +`+/usr/bin+`, like many other system directories, have a +https://docs.fedoraproject.org/en-US/packaging-guidelines/RPMMacros/#_macros_for_paths_set_and_used_by_build_systems[default rpm macro] defined. +The macros should always be used when available, +so the executable is listed in `+%files+` as follows: + +---- +%files +%{_bindir}/hello +---- + +=== Man pages + +---- +Installed (but unpackaged) file(s) found: +/usr/share/man/man1/hello.1.gz +---- + +The Packaging Guidelines have dedicated section for +https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages[Manpages]. +Following its instructions, manpages are list as follows: + +---- +%{_mandir}/man1/hello.1.* +---- + +=== Texinfo pages + +---- +Installed (but unpackaged) file(s) found: +/usr/share/info/dir +/usr/share/info/hello.info.gz +---- + +Texinfo pages are handled much in the same way as man pages. +The directory is defined by the default macro `+{_infodir}+`, +so the Texinfo manual can be added as follows: + +---- +%{_infodir}/hello.info.* +---- + +The https://src.fedoraproject.org/rpms/texinfo[texinfo package] +has rpm triggers that automatically generate the Texinfo `+dir+` file +from all the texinfo pages in the system. +Thus, the `+dir+` generated by GNU Hello build script must not be installed. +This is done by calling `+rm+` in `+%install+`. +However, note that files are installed in the _buildroot_ directory, +and thus the removal is done like this: + +---- +%install +rm %{buildroot}/%{_infodir}/dir +---- + +=== Translations + +---- +Installed (but unpackaged) file(s) found: +/usr/share/locale/bg/LC_MESSAGES/hello.mo +/usr/share/locale/ca/LC_MESSAGES/hello.mo +/usr/share/locale/da/LC_MESSAGES/hello.mo +... +---- + +Since our program uses translations and internationalization, +we are seeing a lot of undeclared i18n files. +The https://docs.fedoraproject.org/en-US/packaging-guidelines/#_handling_locale_files[recommended method] +to declare them is: + +. Add the required build dependency with `+BuildRequires: gettext+`. +. Find the filenames in the `+%install+` step with `+%find_lang %{name}+`. +. Install the files with `+%files -f %{name}.lang+`. + +=== License file + +Every package must install its license, +tagged with `+%license+` directive. +In GNU Hello's case, as well as for many other projects, +the license file is located the source tarball's top level, +and perhaps not copied to the buildroot during installation at all. +Regardless, it can be installed to the standard license directory +by using a relative path: + +---- +%license COPYING +---- + +=== Additional documentation === + +Often, package sources contain documentation +that could be useful for the end users as well. +These can be installed and marked as documentation with the `+%doc+` directive. +Similarly to `+%license+`, +relative paths can be used to include files directly from the source tarball +rather than from the buildroot: + +---- +%doc AUTHORS ChangeLog NEWS README THANKS TODO +---- + +== Checking the result with rpmlint + +Next you should check them for conformance with RPM design rules, +by running `rpmlint` on the `.spec` file and all RPMs: + +---- +$ rpmlint hello.spec ../SRPMS/hello-2.10-1* ../RPMS/*/hello-2.10-1* +---- + +There should be no warnings or errors. +Otherwise, use `+rpmlint -e +` +to see a more verbose description of the `+rpmlint+` diagnostics. + +In the GNU Hello case, +one warning can be expected: + +---- +hello.x86_64: W: file-not-utf8 /usr/share/doc/hello/THANKS +---- + +In order to ensure a pure utf-8 installation, +the file needs to be converted in `+%prep+`. +This can be done, for example, with the `+iconv+` utility: + +---- +mv THANKS THANKS.old +iconv --from-code=ISO-8859-1 --to-code=UTF-8 --output=THANKS THANKS.old +---- + +== A Complete hello.spec File + +Here is the initial version of `hello.spec`: + +[source,RPMSpec] +---- +Name: hello +Version: 2.10 +Release: 1%{?dist} +Summary: Produces a familiar, friendly greeting + +License: GPLv3+ +URL: http://ftp.gnu.org/gnu/%{name} +Source0: http://ftp.gnu.org/gnu/%{name}/%{name}-%{version}.tar.gz + +BuildRequires: gettext + +%description +The GNU Hello program produces a familiar, friendly greeting. Yes, this is +another implementation of the classic program that prints “Hello, world!” when +you run it. + +%prep +%autosetup +mv THANKS THANKS.old +iconv --from-code=ISO-8859-1 --to-code=UTF-8 --output=THANKS THANKS.old + +%build +%configure +%make_build + +%install +%make_install +rm %{buildroot}/%{_infodir}/dir +%find_lang %{name} + +%files -f %{name}.lang +%{_mandir}/man1/hello.1.* +%{_infodir}/hello.info.* +%{_bindir}/hello +%doc AUTHORS ChangeLog NEWS README THANKS TODO +%license COPYING + +%changelog +* Sat Oct 23 2021 The Coon of Ty 2.10-1 +- Initial version of the package +---- + +With this `.spec` file, +you should be able to successfully complete the build process, +and create the source and binary RPM packages. + +== The mock Builds + +To check that the package build will succeed in the Fedora restricted build environment, +check it with `mock`. +The default `mock` configuration builds the package against Rawhide, +the Fedora development branch. + +[subs=+attributes] +---- +$ mock --verbose ../SRPMS/hello-2.10-1.fc{MAJOROSVER}.src.rpm +---- + +== References + +* https://rpm-software-management.github.io/rpm/manual/[RPM Reference Manual] + +* https://docs.fedoraproject.org/en-US/quick-docs/creating-rpm-packages/[Creating RPM packages] + +* https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds[Using Mock to test package builds] + +* xref:Using_the_Koji_Build_System.adoc[Using the Koji build system] + +* https://www.redhat.com/sysadmin/create-rpm-package[How to create a Linux RPM package] From 9e819e10b0de7a1d0cc5271b633ec00036e14a65 Mon Sep 17 00:00:00 2001 From: Otto Urpelainen Date: Nov 10 2021 19:21:08 +0000 Subject: [PATCH 2/2] Use fedpkg in GNU Hello packaging tutorial Actual packaing work in Fedora is very much based on fedpkg usage, so it is much better to use fedpkg in the tutorial, too, instead of using rpmbuild there and the require packagers to learn another tool for actual packaing. --- diff --git a/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc b/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc index 149e405..28babce 100644 --- a/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc +++ b/modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc @@ -24,13 +24,13 @@ including following the various Fedora guidance. [#dev_env] == Development Environment -To build RPMs we need a set of development tools. +To build RPMs for Fedora we need a set of development tools. This is a one-time-only setup, installed by running these commands from a system administration (root) account: ---- -# dnf install fedora-packager @development-tools +# dnf install fedora-packager ---- To be able to test the build procedure in a clean chroot @@ -45,15 +45,6 @@ All the remaining work should be done from your regular, non-privileged account, or even from a separate account created just for development work. Modern RPM-based systems, including Fedora, are set up to build and test RPM packages purely from within a non-privileged account. -The following command sets up an RPM build area in your `+~/rpmbuild+` directory. -This directory will contain several subdirectories, -for the project source code, -RPM configuration files -and for the resulting source and binary packages. - ----- -$ rpmdev-setuptree ----- [#building] == Building the RPM @@ -61,32 +52,22 @@ $ rpmdev-setuptree We need the source code of the project we are packaging, often referred to as the _upstream_ source. We will download it from the project's website -into the `+~/rpmbuild/SOURCES+` directory. +into a directory we create for packaging GNU Hello. We are getting the compressed tarball archive, which happens to be the preferred distribution form for most FOSS projects. ---- -$ cd ~/rpmbuild/SOURCES +$ mkdir hello && cd hello $ wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz ---- The RPM package is configured by `.spec` files. -We will create a template file `hello.spec` in the appropriate directory: +We will create a template file `hello.spec`: ---- -$ cd ~/rpmbuild/SPECS $ rpmdev-newspec hello ---- -Recent versions of `Emacs` and `vi` have `.spec` file editing modes -which will also bring up a similar template upon creating a new file. -So you can just use the following command -for example to use the template automatically. - ----- -$ emacs hello.spec ----- - [#inside_spec] == Inside a Spec File @@ -163,19 +144,59 @@ at least for the initial attempt. == Building the Package -We are ready for the first run to build source, binary and debugging packages: +We are ready for the first run to build source, binary and debugging packages. +This, and many other tasks, are done with the `fedpkg` tool. +The production builds for Fedora are built +in the https://koji.fedoraproject.org[Koji] build system, +which in turn uses https://rpm-software-management.github.io/mock/[Mock] +to manage isolated build environments. +To get as close to a production build as is locally possible, +we use the `fedpkg mockbuild` command +which also invokes Mock: ---- -$ rpmbuild -ba hello.spec +$ fedpkg --release rawhide mockbuild ---- -It will complain and list the unpackaged files, +The build environment created by Mock is very basic. +It does not include a C compiler by default, +so the build will fail. +The reason is explained in the output: + +---- +checking whether the C compiler works... no +configure: error: in `/builddir/build/BUILD/hello-2.10': +configure: error: C compiler cannot create executables +See `config.log' for more details +error: Bad exit status from /var/tmp/rpm-tmp.D2nN0w (%build) + Bad exit status from /var/tmp/rpm-tmp.D2nN0w (%build) +---- + +Additional build tools are defined +by adding `+BuildRequires:+` rows to the specfile. +In Fedora, GCC is the standard compiler, +so we need to add a row for `+gcc+`. +GNU Hello also uses `+make+`, so a row should be added for it, too. +Add these lines after `Source0`: + +---- +BuildRequires: gcc +BuildRequires: make +---- + +Run a mockbuild again. +The earlier error should be gone. + +== Installing files + +The next thing rpm will complain about are unpackaged files, i.e. the files that would be installed in the system, but were not declared as belonging to the package. We need to declare them in the `+%files+` section. -This is an iterative process; -after declaring a missing file in the `+.spec+` file, -rerun `+rpmbuild+`. +Fixing these errors is an iterative process. +After declaring a missing file in the `+.spec+` file, +run `+fedpkg+` again, +then declare the next missing file and so on. We will go through the file list one by one. @@ -287,19 +308,58 @@ rather than from the buildroot: %doc AUTHORS ChangeLog NEWS README THANKS TODO ---- +== Running tests + +GNU Hello, like many other projects, +includes an automated test suite in the sources. +If at all possible, +the test suite should be run during the rpm build. +This helps ensuring that a working build was produced. +This is done by adding the test suite invocation +to specfile `+%check%+` section, +which comes after `+%install+` in order. +In GNU Hello's case: + +---- +%check +make check +---- + +Run a mockbuild again +and check the output to ensure that the tests were actually run. +Something like this should be somewhere in the output: + +---- +============================================================================ +Testsuite summary for GNU Hello 2.10 +============================================================================ +# TOTAL: 5 +# PASS: 4 +# SKIP: 1 +# XFAIL: 0 +# FAIL: 0 +# XPASS: 0 +# ERROR: 0 +============================================================================ +---- + == Checking the result with rpmlint Next you should check them for conformance with RPM design rules, -by running `rpmlint` on the `.spec` file and all RPMs: +by running `rpmlint` on specfile, source rpm and binary rpm. +Command `+fedpkg lint+` _should_ do this, +but as of version 1.41, +it suffers from a bug +causing it not to find the rpms created by `+fedpkg mockbuild+`. +So instead, `+rpmlint+` needs to be called directly. +Pass files to check as arguments: +[subs="attributes+"] ---- -$ rpmlint hello.spec ../SRPMS/hello-2.10-1* ../RPMS/*/hello-2.10-1* +$ rpmlint hello.spec results_hello/2.10/1.fc{NEXTOSVER}/hello-2.10*.{x86_64,src}.rpm ---- -There should be no warnings or errors. -Otherwise, use `+rpmlint -e +` -to see a more verbose description of the `+rpmlint+` diagnostics. - +If all is good, there should be no warnings or errors. In the GNU Hello case, one warning can be expected: @@ -307,9 +367,11 @@ one warning can be expected: hello.x86_64: W: file-not-utf8 /usr/share/doc/hello/THANKS ---- -In order to ensure a pure utf-8 installation, +Descriptions of various error codes can be queried +with `+rpmlint -e +`. +In this case, in order to ensure a pure utf-8 installation, the file needs to be converted in `+%prep+`. -This can be done, for example, with the `+iconv+` utility: +This can be done with the `+iconv+` utility: ---- mv THANKS THANKS.old @@ -331,7 +393,9 @@ License: GPLv3+ URL: http://ftp.gnu.org/gnu/%{name} Source0: http://ftp.gnu.org/gnu/%{name}/%{name}-%{version}.tar.gz -BuildRequires: gettext +BuildRequires: gcc +BuildRequires: gettext +BuildRequires: make %description The GNU Hello program produces a familiar, friendly greeting. Yes, this is @@ -352,6 +416,9 @@ iconv --from-code=ISO-8859-1 --to-code=UTF-8 --output=THANKS THANKS.old rm %{buildroot}/%{_infodir}/dir %find_lang %{name} +%check +make check + %files -f %{name}.lang %{_mandir}/man1/hello.1.* %{_infodir}/hello.info.* @@ -368,18 +435,6 @@ With this `.spec` file, you should be able to successfully complete the build process, and create the source and binary RPM packages. -== The mock Builds - -To check that the package build will succeed in the Fedora restricted build environment, -check it with `mock`. -The default `mock` configuration builds the package against Rawhide, -the Fedora development branch. - -[subs=+attributes] ----- -$ mock --verbose ../SRPMS/hello-2.10-1.fc{MAJOROSVER}.src.rpm ----- - == References * https://rpm-software-management.github.io/rpm/manual/[RPM Reference Manual] diff --git a/modules/ROOT/partials/attributes.adoc b/modules/ROOT/partials/attributes.adoc index a507416..7b0e7b1 100644 --- a/modules/ROOT/partials/attributes.adoc +++ b/modules/ROOT/partials/attributes.adoc @@ -1,4 +1,5 @@ :year: 2021 -:MAJOROSVER: 34 :PREVIOUSOSVER: 33 +:MAJOROSVER: 34 +:NEXTOSVER: 35 :MAJOREPELVER: 8