From 97bc0eb61ae2af1bfa205dd72c979855b501dc18 Mon Sep 17 00:00:00 2001 From: Tomas Kopecek Date: Sep 22 2020 12:36:12 +0000 Subject: doc: livecd/livemedia updates Fixes: https://pagure.io/koji/issue/2094 --- diff --git a/docs/source/image_build.rst b/docs/source/image_build.rst index b44db39..23a814d 100644 --- a/docs/source/image_build.rst +++ b/docs/source/image_build.rst @@ -6,9 +6,9 @@ Image Building ============== Koji is capable of building many different types of images or appliances. They -broadly fall into a few categories: LiveCDs, Disk Images, and Containers. All -types of images end up with a Name-Version-Release just like an RPM build. -What you need to provide Koji to perform a build varies by category, and the +broadly fall into a few categories: LiveMedias, Disk Images, and Containers. All +types of images end up with a Name-Version-Release just like an RPM build. What +you need to provide Koji to perform a build varies by category, and the specifics will be explained below. For additional questions and information, ask around in ``#koji`` on FreeNode @@ -67,100 +67,111 @@ If we passed ``--ksurl`` the string above, and gave ``server-ec2.ks``, and pass that to Anaconda * Perform an automated install -.. _building-livecds: +.. _building-livemedia: -Building LiveCDs -================ +Building LiveMedias +=================== -Let's describe building LiveCDs and the mechanics behind it. +Let's describe building LiveMedias and the mechanics behind it. We also still +suport older method LiveCD which is deprecated now. Getting Started --------------- What you need before proceeding: -* a name for your LiveCD, and a version number +* a name for your LiveMedia, and a version number * a Koji build target * what architectures you want * a flattened kickstart file * you may also need yum repositories generated by release engineering if you - require signed packages be installed into the LiveCD -* the livecd permission in Koji + require signed packages be installed into the LiveMedia +* the permission for building these in Koji - in default installation permission + is not needed, but in production environments there will be probably some + permission or at least policy in place -The Koji command that creates a LiveCD is called ``spin-livecd``. It calls out -to `livecd-tools`_ in a mock chroot to construct the LiveCD. To run a LiveCD -build, use the spin-livecd command like so: +The Koji command that creates a LiveMedia is called ``spin-livemedia``. It calls +out to `livemedia-creator`_ from `lorax`_ project in a mock chroot to construct +the LiveMedia. To run a LiveMedia build, use the ``spin-livemedia`` command like +so: :: -$ koji spin-livecd --release 4 fedora-workstation 23 f23-image-build fedora-workstation.ks + $ koji spin-livemedia --release 20200922.n.0 \ + --ksurl 'git+https://pagure.io/fedora-kickstarts.git?#b4956c05028a088c641d0ce6e1a31b6d8b20176f' \ + --install-tree-url 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \ + --repo 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \ + --can-fail ppc64le,aarch64 \ + Fedora-Workstation-Live 33 f33 x86_64,ppc64le,aarch64 fedora-live-workstation.ks -In this example a LiveCD will be created with the N-V-R of -fedora-workstation-23-4. If ``--release`` was not included an incrementing -value would be computed by Koji instead. spin-livecd takes a minimum of 5 -arguments: +In this example (real task is `here +`_) a LiveMedia will +be created with the N-V-R of ``Fedora-Workstation-Live-33-20200922.n.0``. If +``--release`` was not included an incrementing value would be computed by Koji +instead. spin-livemedia takes a minimum of 4 arguments: Name the name of the image, without versioning information. Examples could be - ``fedora`` or ``fedora-workstation``. + ``Fedora-Live`` or ``Fedora-Workstation-Live``. Version an arbitrary version string. For Fedora images, this usually matches the - release version, such as 21, 22, or 23. + release version, such as 31, 32, or 33. Build Target just like RPM builds Koji must be told which target to use. This controls what tag the image will be tagged into, and what packages are available to install into the image. -Architecture - only x86_64 or i386 is supported - Kickstart File this is a recipe file that performs drives the construction of the image. + Note, that you're can either upload your kickstart or just point to some SCM + (``--ksurl``). In our example fedora repo contains + ``fedora-live-workstation.ks`` kickstart file. -Use ``--help`` to discover more options to ``spin-livecd``. You can override -the Release field with ``--release``, or ``--repo`` to override what yum repo -is used to provide packages to install to the image. Note that regardless of -what repo you use, it must contain RPMs built by Koji, otherwise you will get -an error. (unless you are building a scratch image) +Use ``--help`` to discover more options to ``spin-livemedia``. You can override +the Release field with ``--release``, or ``--repo`` to override what yum repo is +used to provide packages to install to the image. Note that regardless of what +repo you use, it must contain RPMs built by Koji, otherwise you will get an +error. (unless you are building a scratch image). ``--can-fail`` options says +which architectures are optional and can fail without failing the whole build. Mechanics --------- -The way LiveCDs are created in Koji is fairly straightforward. A chroot is +The way LiveMedias are created in Koji is fairly straightforward. A chroot is initialized and populated with the packages and their dependencies from the -``livecd-build`` package group. Next, the kickstart file is copied into it if +``livemedia-build`` package group. Next, the kickstart file is copied into it if it was provided from local storage. If not, it is checked out into it from an SCM. It is then modified to use the repo associated with the build tag for the -target specified in the command unless the ``--repo`` option was given. Both -the original and the modified kickstart files are saved as part of the output -for the task for later review. A `livecd-creator`_ command is executed using the -mock('--chroot', ...) method. +target specified in the command unless the ``--repo`` option was given. Both the +original and the modified kickstart files are saved as part of the output for +the task for later review. A `livemedia-creator`_ command is executed using the +``mock('--chroot', ...)`` method. .. note:: - This process runs as root. This produces the desired image which is - uploaded to /mnt/koji/images//$imageID if it is not a scratch image. + This process runs as root. This produces the desired image which is uploaded + to ``/mnt/koji/images//$imageID`` if it is not a scratch image. -.. _caveats-for-livecds: +.. _caveats-for-livemedias: -Caveats for LiveCDs -------------------- +Caveats for LiveMedias +---------------------- -There are some known caveats with using the spin-livecd command that users +There are some known caveats with using the spin-livemedia command that users should be aware of. %include macros in the kickstart ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A word of caution about kickstart files and the ``%include`` macro. -`livecd-creator`_ is smart enough to search the current directory of the -submitted kickstart file if it has %include macros. If the kickstart specified -to koji is from local storage, only that kickstart file will be copied into -the chroot, and this creates a problem if it has ``%include`` macros, because -the other kickstart files it needs will be inaccessible. This issue is not -present when the kickstart file is retrieved from a remote SCM (such as the -spin-kickstarts git repo), because the entire repository is checked out. +`livemedia-creator`_ is smart enough to search the current directory of the +submitted kickstart file if it has ``%include`` macros. If the kickstart +specified to koji is from local storage, only that kickstart file will be copied +into the chroot, and this creates a problem if it has ``%include`` macros, +because the other kickstart files it needs will be inaccessible. This issue is +not present when the kickstart file is retrieved from a remote SCM (such as the +fedora-kickstarts git repo), because the entire repository is checked out. Presumably it will include any other kickstart files the specified one is including in the same directory. A workaround for the issue would be to use ``ksflatten`` (from pykickstart) on kickstart files with ``%include`` macros @@ -170,11 +181,11 @@ Package Groups in the Kickstart File ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Package Groups in the kickstart file cause a problem if the Koji repos do not -define them, which they most likely don't since Koji's comps.xml is based on -the "groups" set up from the CLI. `livecd-creator`_'s behavior is to ignore +define them, which they most likely don't since Koji's comps.xml is based on the +"groups" set up from the CLI. `livemedia-creator`_'s behavior is to ignore package groups that are not defined in the repo it is using, so this can be -troublesome when creating the image since packages could be left out. There -are a couple possible workarounds: +troublesome when creating the image since packages could be left out. There are +a couple possible workarounds: * do not use package groups in the kickstart file and just specify a huge list of packages @@ -205,21 +216,22 @@ system. %post Section in Kickstart ^^^^^^^^^^^^^^^^^^^^^^^^^^ -While livecd-tools does support building on SELinux disabled hosts, you can run -into denials when booting if you create and use new files in the ``%post`` -section of your kickstart file. If you do, you should either set the labels -appropriately at the end of the ``%post`` section, or instigate an autorelabel -at boot time. +While `livemedia-creator`_ does support building on SELinux disabled hosts, you +can run into denials when booting if you create and use new files in the +``%post`` section of your kickstart file. If you do, you should either set the +labels appropriately at the end of the ``%post`` section, or instigate an +autorelabel at boot time. Troubleshooting --------------- If your build fails, you will be notified on the command line. In the output will be a URL to the Koji UI, visit that and click on the red subtask link. -From that page review ``root.log`` and ``livecd.log`` for errors. Often errors -are caused by packages being missing, or malformed kickstart files. The log -files are at the bottom of the page. If you're stuck, contact Release -Engineering. +From that page review ``root.log`` and ``livemedia-out.log`` for errors. Often +errors are caused by packages being missing, or malformed kickstart files. The +log files are at the bottom of the page. If problem occurs later during +installation `livemedia-creator`_ will also upload a lot of other logs like +anaconda's, etc. If you're stuck, contact Release Engineering. Build System Preparation ------------------------ @@ -228,32 +240,30 @@ This section assumes you have know-how required to install and configure a new instance of Koji, and that you have already done so. You can learn how to do so :doc:`here ` if you need to. Please ensure you are using the latest version of the software and that your database schema is updated as -well. You should also have some familiarity with how `livecd-creator`_ works. -This section only covers preparation for LiveCD builds. +well. You should also have some familiarity with how `livemedia-creator`_ works. +This section only covers preparation for LiveMedia builds. Follow this procedure step by step to get things prepared they way they need to be. -#. ``koji add-host-to-channel livecd`` - add a builder to the livecd channel -#. ``koji grant-permission livecd `` +#. ``koji add-host-to-channel livemedia`` + add a builder to the livemedia channel +#. ``koji grant-permission livemedia `` grant the permission to build an image type to a user. This step is optional since admins have all permissions. #. You will need a tag and target to build the images from. The yum repo generated for the build tag of the target is what Koji will use to populate - the LiveCDs with by default. (the alternative is to use the ``--repo`` + the LiveMedias with by default. (the alternative is to use the ``--repo`` option, more on that later) -#. ``koji add-group livecd-build`` - add the livecd-build group -#. ``koji add-group-pkg livecd-build ...`` - add packages to the livecd-build group. These package lists vary has - packages and dependencies change. As of October, 2015 for Fedora 23 the +#. ``koji add-group livemedia-build`` + add the livemedia-build group +#. ``koji add-group-pkg livemedia-build ...`` + add packages to the livemedia-build group. These package lists vary has + packages and dependencies change. As of September, 2020 for Fedora 33 the needed packages for each image type are: - * bash, coreutils, fedora-logos, fedora-release, livecd-tools, - policycoreutils, python-dbus, sed, selinux-policy-targeted, - shadow-utils, squashfs-tools, sssd-client, tar, unzip, util-linux, - which, yum + * bash, coreutils, glibc-all-langpacks, lorax-lmc-novirt, + selinux-policy-targeted, shadow-utils, util-linux Building Disk Images @@ -268,7 +278,7 @@ Getting Started What you need: -* a name for your LiveCD, and a version number +* a name for your image, and a version number * what architectures you want * a Koji build target * kickstart file @@ -869,13 +879,13 @@ error. (unless you are building a scratch image) Mechanics --------- -The way Appliances are created in Koji is the same as :ref:`building-livecds`. +The way Appliances are created in Koji is the same as :ref:`building-livemedia`. Caveats for Appliances ---------------------- -Caveats for using ``spin-appliance`` are the same as using ``spin-livecd`` to -:ref:`caveats-for-livecds`. +Caveats for using ``spin-appliance`` are the same as using ``spin-livemedia`` to +:ref:`caveats-for-livemedias`. Troubleshooting --------------- @@ -939,8 +949,8 @@ to be. https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst .. _Anaconda Kickstart Guide: https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst -.. _livecd-tools: https://github.com/rhinstaller/livecd-tools/ -.. _livecd-creator: https://fedoraproject.org/wiki/FedoraLiveCD +.. _lorax: https://github.com/weldr/lorax +.. _livemedia-creator: https://weldr.io/lorax/livemedia-creator.html .. _ImageFactory: http://imgfac.org/ .. _Oz: https://github.com/clalancette/oz .. _how to use Oz: https://github.com/clalancette/oz/wiki