#2500 doc: livecd/livemedia updates
Merged 2 years ago by tkopecek. Opened 2 years ago by tkopecek.
tkopecek/koji issue2094  into  master

file modified
+91 -81
@@ -6,9 +6,9 @@ 

  ==============

  

  Koji is capable of building many different types of images or appliances. They

- broadly fall into a few categories: LiveCDs, Disk Images, and Containers. All

- types of images end up with a Name-Version-Release just like an RPM build.

- What you need to provide Koji to perform a build varies by category, and the

+ broadly fall into a few categories: LiveMedias, Disk Images, and Containers. All

+ types of images end up with a Name-Version-Release just like an RPM build.  What

+ you need to provide Koji to perform a build varies by category, and the

  specifics will be explained below.

  

  For additional questions and information, ask around in ``#koji`` on FreeNode
@@ -67,100 +67,111 @@ 

    ``server-ec2.ks``, and pass that to Anaconda

  * Perform an automated install

  

- .. _building-livecds:

+ .. _building-livemedia:

  

- Building LiveCDs

- ================

+ Building LiveMedias

+ ===================

  

- Let's describe building LiveCDs and the mechanics behind it.

+ Let's describe building LiveMedias and the mechanics behind it. We also still

+ suport older method LiveCD which is deprecated now.

  

  Getting Started

  ---------------

  

  What you need before proceeding:

  

- * a name for your LiveCD, and a version number

+ * a name for your LiveMedia, and a version number

  * a Koji build target

  * what architectures you want

  * a flattened kickstart file

  * you may also need yum repositories generated by release engineering if you

-   require signed packages be installed into the LiveCD

- * the livecd permission in Koji

+   require signed packages be installed into the LiveMedia

+ * the permission for building these in Koji - in default installation permission

+   is not needed, but in production environments there will be probably some

+   permission or at least policy in place

  

- The Koji command that creates a LiveCD is called ``spin-livecd``. It calls out

- to `livecd-tools`_ in a mock chroot to construct the LiveCD. To run a LiveCD

- build, use the spin-livecd command like so:

+ The Koji command that creates a LiveMedia is called ``spin-livemedia``. It calls

+ out to `livemedia-creator`_ from `lorax`_ project in a mock chroot to construct

+ the LiveMedia. To run a LiveMedia build, use the ``spin-livemedia`` command like

+ so:

  

  ::

  

- $ koji spin-livecd --release 4 fedora-workstation 23 f23-image-build fedora-workstation.ks

+     $ koji spin-livemedia --release 20200922.n.0  \

+                           --ksurl 'git+https://pagure.io/fedora-kickstarts.git?#b4956c05028a088c641d0ce6e1a31b6d8b20176f' \

+                           --install-tree-url 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \

+                           --repo 'https://kojipkgs.fedoraproject.org/compose/branched/Fedora-33-20200922.n.0/compose/Everything/$basearch/os' \

+                           --can-fail ppc64le,aarch64 \

+                           Fedora-Workstation-Live 33 f33 x86_64,ppc64le,aarch64 fedora-live-workstation.ks

  

- In this example a LiveCD will be created with the N-V-R of

- fedora-workstation-23-4. If ``--release`` was not included an incrementing

- value would be computed by Koji instead. spin-livecd takes a minimum of 5

- arguments:

+ In this example (real task is `here

+ <https://koji.fedoraproject.org/koji/taskinfo?taskID=52013359>`_) a LiveMedia will

+ be created with the N-V-R of ``Fedora-Workstation-Live-33-20200922.n.0``. If

+ ``--release`` was not included an incrementing value would be computed by Koji

+ instead. spin-livemedia takes a minimum of 4 arguments:

  

  Name

      the name of the image, without versioning information. Examples could be

-     ``fedora`` or ``fedora-workstation``.

+     ``Fedora-Live`` or ``Fedora-Workstation-Live``.

  

  Version

      an arbitrary version string. For Fedora images, this usually matches the

-     release version, such as 21, 22, or 23.

+     release version, such as 31, 32, or 33.

  

  Build Target

      just like RPM builds Koji must be told which target to use. This controls

      what tag the image will be tagged into, and what packages are available to

      install into the image.

  

- Architecture

-     only x86_64 or i386 is supported

- 

  Kickstart File

      this is a recipe file that performs drives the construction of the image.

+     Note, that you're can either upload your kickstart or just point to some SCM

+     (``--ksurl``).  In our example fedora repo contains

+     ``fedora-live-workstation.ks`` kickstart file.

  

- Use ``--help`` to discover more options to ``spin-livecd``. You can override

- the Release field with ``--release``, or ``--repo`` to override what yum repo

- is used to provide packages to install to the image. Note that regardless of

- what repo you use, it must contain RPMs built by Koji, otherwise you will get

- an error. (unless you are building a scratch image)

+ Use ``--help`` to discover more options to ``spin-livemedia``. You can override

+ the Release field with ``--release``, or ``--repo`` to override what yum repo is

+ used to provide packages to install to the image. Note that regardless of what

+ repo you use, it must contain RPMs built by Koji, otherwise you will get an

+ error. (unless you are building a scratch image). ``--can-fail`` options says

+ which architectures are optional and can fail without failing the whole build.

  

  Mechanics

  ---------

  

- The way LiveCDs are created in Koji is fairly straightforward. A chroot is

+ The way LiveMedias are created in Koji is fairly straightforward. A chroot is

  initialized and populated with the packages and their dependencies from the

- ``livecd-build`` package group. Next, the kickstart file is copied into it if

+ ``livemedia-build`` package group. Next, the kickstart file is copied into it if

  it was provided from local storage. If not, it is checked out into it from an

  SCM. It is then modified to use the repo associated with the build tag for the

- target specified in the command unless the ``--repo`` option was given. Both

- the original and the modified kickstart files are saved as part of the output

- for the task for later review. A `livecd-creator`_ command is executed using the

- mock('--chroot', ...) method.

+ target specified in the command unless the ``--repo`` option was given. Both the

+ original and the modified kickstart files are saved as part of the output for

+ the task for later review. A `livemedia-creator`_ command is executed using the

+ ``mock('--chroot', ...)`` method.

  

  .. note::

-     This process runs as root. This produces the desired image which is

-     uploaded to /mnt/koji/images/<image>/$imageID if it is not a scratch image.

+     This process runs as root. This produces the desired image which is uploaded

+     to ``/mnt/koji/images/<image>/$imageID`` if it is not a scratch image.

  

- .. _caveats-for-livecds:

+ .. _caveats-for-livemedias:

  

- Caveats for LiveCDs

- -------------------

+ Caveats for LiveMedias

+ ----------------------

  

- There are some known caveats with using the spin-livecd command that users

+ There are some known caveats with using the spin-livemedia command that users

  should be aware of.

  

  %include macros in the kickstart

  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  

  A word of caution about kickstart files and the ``%include`` macro.

- `livecd-creator`_ is smart enough to search the current directory of the

- submitted kickstart file if it has %include macros. If the kickstart specified

- to koji is from local storage, only that kickstart file will be copied into

- the chroot, and this creates a problem if it has ``%include`` macros, because

- the other kickstart files it needs will be inaccessible. This issue is not

- present when the kickstart file is retrieved from a remote SCM (such as the

- spin-kickstarts git repo), because the entire repository is checked out.

+ `livemedia-creator`_ is smart enough to search the current directory of the

+ submitted kickstart file if it has ``%include`` macros. If the kickstart

+ specified to koji is from local storage, only that kickstart file will be copied

+ into the chroot, and this creates a problem if it has ``%include`` macros,

+ because the other kickstart files it needs will be inaccessible. This issue is

+ not present when the kickstart file is retrieved from a remote SCM (such as the

+ fedora-kickstarts git repo), because the entire repository is checked out.

This is not true for livemedia-creator. Since Lorax uses Anaconda, it inherits Anaconda's inability to expand %include properly.

  Presumably it will include any other kickstart files the specified one is

  including in the same directory. A workaround for the issue would be to use

  ``ksflatten`` (from pykickstart) on kickstart files with ``%include`` macros
@@ -170,11 +181,11 @@ 

  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  

  Package Groups in the kickstart file cause a problem if the Koji repos do not

- define them, which they most likely don't since Koji's comps.xml is based on

- the "groups" set up from the CLI. `livecd-creator`_'s behavior is to ignore

+ define them, which they most likely don't since Koji's comps.xml is based on the

+ "groups" set up from the CLI. `livemedia-creator`_'s behavior is to ignore

  package groups that are not defined in the repo it is using, so this can be

- troublesome when creating the image since packages could be left out. There

- are a couple possible workarounds:

+ troublesome when creating the image since packages could be left out. There are

+ a couple possible workarounds:

  

  * do not use package groups in the kickstart file and just specify a huge list

    of packages
@@ -205,21 +216,22 @@ 

  %post Section in Kickstart

  ^^^^^^^^^^^^^^^^^^^^^^^^^^

  

- While livecd-tools does support building on SELinux disabled hosts, you can run

- into denials when booting if you create and use new files in the ``%post``

- section of your kickstart file. If you do, you should either set the labels

- appropriately at the end of the ``%post`` section, or instigate an autorelabel

- at boot time.

+ While `livemedia-creator`_ does support building on SELinux disabled hosts, you

+ can run into denials when booting if you create and use new files in the

+ ``%post`` section of your kickstart file. If you do, you should either set the

+ labels appropriately at the end of the ``%post`` section, or instigate an

+ autorelabel at boot time.

  

  Troubleshooting

  ---------------

  

  If your build fails, you will be notified on the command line. In the output

  will be a URL to the Koji UI, visit that and click on the red subtask link.

- From that page review ``root.log`` and ``livecd.log`` for errors. Often errors

- are caused by packages being missing, or malformed kickstart files. The log

- files are at the bottom of the page. If you're stuck, contact Release

- Engineering.

+ From that page review ``root.log`` and ``livemedia-out.log`` for errors. Often

+ errors are caused by packages being missing, or malformed kickstart files. The

+ log files are at the bottom of the page. If problem occurs later during

+ installation `livemedia-creator`_ will also upload a lot of other logs like

+ anaconda's, etc. If you're stuck, contact Release Engineering.

  

  Build System Preparation

  ------------------------
@@ -228,32 +240,30 @@ 

  instance of Koji, and that you have already done so. You can learn how to do so

  :doc:`here <server_howto>` if you need to. Please ensure you are using the

  latest version of the software and that your database schema is updated as

- well. You should also have some familiarity with how `livecd-creator`_ works.

- This section only covers preparation for LiveCD builds.

+ well. You should also have some familiarity with how `livemedia-creator`_ works.

+ This section only covers preparation for LiveMedia builds.

  

  Follow this procedure step by step to get things prepared they way they need to

  be.

  

- #.  ``koji add-host-to-channel <your-host> livecd``

-         add a builder to the livecd channel

- #.  ``koji grant-permission livecd <user>``

+ #.  ``koji add-host-to-channel <your-host> livemedia``

+         add a builder to the livemedia channel

+ #.  ``koji grant-permission livemedia <user>``

          grant the permission to build an image type to a user. This step is

          optional since admins have all permissions.

  #.  You will need a tag and target to build the images from. The yum repo

      generated for the build tag of the target is what Koji will use to populate

-     the LiveCDs with by default. (the alternative is to use the ``--repo``

+     the LiveMedias with by default. (the alternative is to use the ``--repo``

      option, more on that later)

- #.  ``koji add-group <build-tag> livecd-build``

-     add the livecd-build group

- #.  ``koji add-group-pkg <build-tag> livecd-build <pkg> ...``

-         add packages to the livecd-build group. These package lists vary has

-         packages and dependencies change. As of October, 2015 for Fedora 23 the

+ #.  ``koji add-group <build-tag> livemedia-build``

+     add the livemedia-build group

+ #.  ``koji add-group-pkg <build-tag> livemedia-build <pkg> ...``

+         add packages to the livemedia-build group. These package lists vary has

+         packages and dependencies change. As of September, 2020 for Fedora 33 the

          needed packages for each image type are:

  

-         * bash, coreutils, fedora-logos, fedora-release, livecd-tools,

-           policycoreutils, python-dbus, sed, selinux-policy-targeted,

-           shadow-utils, squashfs-tools, sssd-client, tar, unzip, util-linux,

-           which, yum

+         * bash, coreutils, glibc-all-langpacks, lorax-lmc-novirt,

+           selinux-policy-targeted, shadow-utils, util-linux

  

  

  Building Disk Images
@@ -268,7 +278,7 @@ 

  

  What you need:

  

- * a name for your LiveCD, and a version number

+ * a name for your image, and a version number

  * what architectures you want

  * a Koji build target

  * kickstart file
@@ -869,13 +879,13 @@ 

  Mechanics

  ---------

  

- The way Appliances are created in Koji is the same as :ref:`building-livecds`.

+ The way Appliances are created in Koji is the same as :ref:`building-livemedia`.

  

  Caveats for Appliances

  ----------------------

  

- Caveats for using ``spin-appliance`` are the same as using ``spin-livecd`` to

- :ref:`caveats-for-livecds`.

+ Caveats for using ``spin-appliance`` are the same as using ``spin-livemedia`` to

+ :ref:`caveats-for-livemedias`.

  

  Troubleshooting

  ---------------
@@ -939,8 +949,8 @@ 

      https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst

  .. _Anaconda Kickstart Guide:

      https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst

- .. _livecd-tools: https://github.com/rhinstaller/livecd-tools/

- .. _livecd-creator: https://fedoraproject.org/wiki/FedoraLiveCD

+ .. _lorax: https://github.com/weldr/lorax

+ .. _livemedia-creator: https://weldr.io/lorax/livemedia-creator.html

  .. _ImageFactory: http://imgfac.org/

  .. _Oz: https://github.com/clalancette/oz

  .. _how to use Oz: https://github.com/clalancette/oz/wiki

Metadata Update from @tkopecek:
- Pull-request tagged with: doc

2 years ago

Metadata Update from @jcupova:
- Pull-request tagged with: no_qe

2 years ago

Commit 146d3c0 fixes this pull-request

Pull-Request has been merged by tkopecek

2 years ago

This is not true for livemedia-creator. Since Lorax uses Anaconda, it inherits Anaconda's inability to expand %include properly.