#1588 1.18 release
Merged 4 years ago by mikem. Opened 4 years ago by mikem.
mikem/koji release-1.18.0  into  master

@@ -34,4 +34,12 @@ 

  

  ALTER TABLE build ADD COLUMN cg_id INTEGER REFERENCES content_generator(id);

  

+ 

+ -- new indexes added in 1.18

+ CREATE INDEX tag_packages_active_tag_id ON tag_packages(active, tag_id);

+ CREATE INDEX tag_packages_create_event ON tag_packages(create_event);

+ CREATE INDEX tag_packages_revoke_event ON tag_packages(revoke_event);

+ CREATE INDEX tag_packages_owner ON tag_packages(owner);

+ 

+ 

  COMMIT;

file modified
+2 -2
@@ -54,9 +54,9 @@ 

  # built documents.

  #

  # The short X.Y version.

- version = '1.17'

+ version = '1.18'

  # The full version, including alpha/beta/rc tags.

- release = '1.17.0'

+ release = '1.18.0'

  

  # The language for content autogenerated by Sphinx. Refer to documentation

  # for a list of supported languages.

@@ -0,0 +1,42 @@ 

+ Migrating to Koji 1.18

+ ======================

+ 

+ ..

+   reStructured Text formatted

+ 

+ You should consider the following changes when migrating to 1.18:

+ 

+ DB Updates

+ ----------

+ 

+ This release has a few schema changes:

+ 

+     * Several new indexes to speed operations

+     * A ``cg_id`` field has been added to the ``build`` table

+     * A new ``build_reservations`` table

+     * A new ``build_notifications_block`` table

+     * Updates to the data in the ``archivetypes`` table

+ 

+ As in previous releases, we provide a migration script that updates the

+ database.

+ 

+ ::

+ 

+     # psql koji koji  </usr/share/doc/koji/docs/schema-upgrade-1.17-1.18.sql

+ 

+ 

+ More granular permissions

+ -------------------------

+ 

+ The new ``host``, ``tag``, ``target`` permissions allow access to a number of

+ actions that were previously admin-only.

+ Koji administrators should consider using these to reduce the number of users

+ with full ``admin`` permission.

+ 

+ 

+ Other changes

+ -------------

+ 

+ There are numerous other changes in 1.18 that should not have a direct impact

+ on migration. For details see:

+ :doc:`release_notes_1.18`

@@ -5,6 +5,7 @@ 

  .. toctree::

      :maxdepth: 1

  

+     migrating_to_1.18

      migrating_to_1.17

      migrating_to_1.16

      migrating_to_1.15

@@ -5,6 +5,7 @@ 

  .. toctree::

      :maxdepth: 1

  

+     release_notes_1.18

      release_notes_1.17

      release_notes_1.16.2

      release_notes_1.16.1

@@ -0,0 +1,378 @@ 

+ Koji 1.18.0 Release notes

+ =========================

+ 

+ 

+ Migrating from Koji 1.17

+ ------------------------

+ 

+ For details on migrating see :doc:`migrating_to_1.18`

+ 

+ 

+ 

+ Security Fixes

+ --------------

+ 

+ 

+ 

+ Client Changes

+ --------------

+ 

+ **Add option for custom cert location**

+ 

+ | PR: https://pagure.io/koji/pull-request/1253

+ 

+ The CLI now has an option for setting a custom SSL certificate, similar to the

+ options for Kerberos authentication.

+ 

+ 

+ **Load client plugins from ~/.koji/plugins**

+ 

+ | PR: https://pagure.io/koji/pull-request/892

+ 

+ 

+ This change allows users to load their own cli plugins from ``~/.koji/plugins``

+ or from another location by using the ``plugin_paths`` setting.

+ 

+ 

+ **Show load/capacity in list-channels**

+ 

+ | PR: https://pagure.io/koji/pull-request/1449

+ 

+ The ``list-channels`` display has been expanded to show overall totals for load

+ and capacity.

+ 

+ 

+ **Allow taginfo cli to use tag IDs**

+ 

+ | PR: https://pagure.io/koji/pull-request/1476

+ 

+ The ``taginfo`` command can now accept a numeric tag id on the command line.

+ 

+ 

+ **Add option to show channels in list-hosts**

+ 

+ | PR: https://pagure.io/koji/pull-request/1425

+ 

+ The ``list-hosts`` command will now display channel subscriptions if the

+ ``--show-channels`` option is given.

+ 

+ 

+ **Remove merge option from edit-external-repo**

+ 

+ | PR: https://pagure.io/koji/pull-request/1499

+ 

+ This option was mistakenly added to the command and never did anything.

+ It is gone now.

+ 

+ 

+ **Honor mock.package_manager tag setting in mock-config cli**

+ 

+ | PR: https://pagure.io/koji/pull-request/1374

+ 

+ The ``mock-config`` command will now honor this setting just as ``kojid`` does.

+ 

+ 

+ 

+ 

+ Library Changes

+ ---------------

+ 

+ **New multicall interface**

+ 

+ | PR: https://pagure.io/koji/pull-request/957

+ 

+ This feature implements a new and much better way to use multicall in the Koji

+ library.

+ These changes create a new implementation outside of ClientSession.

+ The old way will still work.

+ 

+ With this new implementation:

+ 

+ * a multicall is tracked as an instance of `MultiCallSession`

+ * the original session is unaffected

+ * multiple multicalls can be managed in parallel, if desired

+ * `MultiCallSession` behaves more or less like a session in multicall mode

+ * method calls return a `VirtualCall` instance that can later be used to access the result

+ * `MultiCallSession` can be used as a context manager, ensuring that the calls are executed

+ 

+ Usage examples can be found in the :doc:`Writing Koji Code <writing_koji_code>`

+ document.

+ 

+ 

+ 

+ 

+ Web UI Changes

+ --------------

+ 

+ **Retain old search pattern in web ui**

+ 

+ | PR: https://pagure.io/koji/pull-request/1258

+ 

+ The search results page of the web ui now retains a search form with the

+ current search pre-filled.

+ This makes it easier for users to refine their searches.

+ 

+ 

+ **Display task durations in webui**

+ 

+ | PR: https://pagure.io/koji/pull-request/1383

+ 

+ 

+ The ``taskinfo`` page in the web ui now shows task durations in addition to

+ timestamps.

+ 

+ 

+ 

+ Builder Changes

+ ---------------

+ 

+ **Rebuild SRPMS before building**

+ 

+ | PR: https://pagure.io/koji/pull-request/1462

+ 

+ For rpm builds from an uploaded srpm, Koji will now rebuild the srpm in the

+ build environment first.

+ This ensures that the NVR is correct for the resulting build.

+ 

+ The old behavior can be requested by setting ``rebuild_srpm=False`` in the tag

+ extra data for the build tag in question.

+ 

+ 

+ **User createrepo_c by default**

+ 

+ | PR: https://pagure.io/koji/pull-request/1278

+ 

+ 

+ The ``use_createrepo_c`` configuration option for ``kojid`` now defaults to True.

+ 

+ 

+ **Use createrepo update option even for first repo run**

+ 

+ | PR: https://pagure.io/koji/pull-request/1363

+ 

+ If there is no older repo for a tag, Koji will now attempt to find

+ a related repo to use ``createrepo --update`` with.

+ This will speed up first-time repo generations for tags that

+ predominantly inherit their content from another build tag.

+ 

+ 

+ **Scale task_avail_delay based on bin rank**

+ 

+ | PR: https://pagure.io/koji/pull-request/1386

+ 

+ This is an adjustment to Koji's decentralized scheduling algorithm.

+ It should result in better utilization of host capacity, particularly when

+ a channel has hosts that are very heterogeneous in capacity.

+ 

+ The meaning of the ``task_avail_delay`` setting is different now.

+ Within a channel-arch bin, the hosts with highest capacity will take the task

+ immediately, while hosts lower down will have a delay proportional to their

+ rank.

+ The "rank" here is a float between 0.0 and 1.0 used as a multiplier.

+ So ``task_avail_delay`` is the maximum time that any host will wait to

+ take a task.

+ 

+ Hosts with higher available capacity will be more likely to claim a

+ task, resulting in better utilization of the highest capacity hosts.

+ 

+ 

+ **Use RawConfigParser for kojid**

+ 

+ | PR: https://pagure.io/koji/pull-request/1544

+ 

+ The use of percent signs is common in ``kojid.conf`` because of the

+ ``host_principal_format`` setting.

+ This causes an error in python3 if ``SafeConfigParser`` is used, so we use

+ ``RawConfigParser`` instead.

+ 

+ 

+ **Handle bare merge mode**

+ 

+ | PR: https://pagure.io/koji/pull-request/1411

+ | PR: https://pagure.io/koji/pull-request/1516

+ | PR: https://pagure.io/koji/pull-request/1502

+ 

+ 

+ This feature adds a new merge mode for external repos named ``bare``.

+ This mode is intended for use with modularity.

+ 

+ Use of this mode requires createrepo_c version 0.14.0 or later on the builders

+ that handle the createrepo tasks.

+ 

+ 

+ 

+ 

+ System Changes

+ --------------

+ 

+ 

+ **API for reserving NVRs for content generators**

+ 

+ | PR: https://pagure.io/koji/pull-request/1464

+ | PR: https://pagure.io/koji/pull-request/1597

+ | PR: https://pagure.io/koji/pull-request/1601

+ | PR: https://pagure.io/koji/pull-request/1602

+ | PR: https://pagure.io/koji/pull-request/1606

+ 

+ This feature allows content generators to reserve NVRs earlier in the build

+ process similar to builds performed by ``kojid``. The NVR is reserved by

+ calling ``CGInitBuild()`` and finalized by the ``CGImport()`` call.

+ 

+ 

+ 

+ **Per-tag configuration of rpm macros**

+ 

+ | PR: https://pagure.io/koji/pull-request/898

+ 

+ This feature allows setting rpm macros via the tag extra field. These macros

+ will be added to the mock configuration for the buildroot. The system

+ looks for extra values of the form ``rpm.macro.NAME``.

+ 

+ For example, to set the dist tag for a given tag, you could use a command like:

+ 

+ ::

+ 

+     $ koji edit-tag f30-build -x rpm.macro.dist=MYDISTTAG

+ 

+ 

+ 

+ **Per-tag configuration for module_hotfixes setting**

+ 

+ | PR: https://pagure.io/koji/pull-request/1524

+ | PR: https://pagure.io/koji/pull-request/1578

+ 

+ Koji now handles the field ``mock.yum.module_hotfixes`` in the tag extra.

+ When set, kojid will set ``module_hotfixes=0/1`` in the yum portion of the

+ mock configuration for a buildroot.

+ 

+ 

+ **Allow users to opt out of notifications**

+ 

+ | PR: https://pagure.io/koji/pull-request/1417

+ | PR: https://pagure.io/koji/pull-request/1580

+ 

+ This feature lets users opt out of notifications that they would otherwise

+ automatically recieve, such as build and tag notifications for:

+ 

+ - the build owner (the user who submitted the build)

+ - the package owner within the given tag

+ 

+ These opt-outs are user controlled and can be managed with the new

+ ``block-notification`` and ``unblock-notificiation`` commands.

+ 

+ 

+ **Allow hub policy to match version and release**

+ 

+ | PR: https://pagure.io/koji/pull-request/1513

+ 

+ 

+ This feature adds new policy tests to match ``version`` and ``release``.

+ This tests are glob pattern matches.

+ 

+ 

+ **Allow hub policy to match build type**

+ 

+ | PR: https://pagure.io/koji/pull-request/1415

+ 

+ 

+ Koji added btypes in version 1.11 along with content generators.

+ Now, all builds have one or more btypes.

+ 

+ This change allows policies to check the btype value using the ``buildtype`` test.

+ 

+ 

+ 

+ **More granular admin permissions**

+ 

+ | PR: https://pagure.io/koji/pull-request/1454

+ 

+ A number of actions that were previously admin-only are now governed by

+ separate permissions:

+ 

+     ``host``

+         This permission governs most host management operations, such as

+         adding, editing, enabling/disabling, and restarting.

+ 

+     ``tag``

+         This permission governs adding, editing, and deleting tags.

+ 

+     ``target``

+         This permission governs adding, editing, and deleting targets.

+ 

+ Koji administrators may want to consider reducing the number of users with

+ full ``admin`` permission.

+ 

+ 

+ **Option to generate separate source repo**

+ 

+ | PR: https://pagure.io/koji/pull-request/1273

+ 

+ The (non-dist) yum repos that Koji generates for building normally don't

+ include srpms.

+ An old option allowed them to be included in some cases, but they were simply

+ added to each repo.

+ Newer options have been added that instruct Koji to include them as a separate

+ src repo.

+ 

+ In the cli, the ``regen-repo`` command now accepts a ``--separate-source``

+ option that triggers this behavior.

+ 

+ In ``kojira``, the ``separate_source_tags`` option is a list of tag patterns.

+ Build tags that match any of these patterns will have their repos generated

+ with a separate src repo.

+ 

+ 

+ 

+ **Add volume option for dist-repo**

+ 

+ | PR: https://pagure.io/koji/pull-request/1327

+ 

+ Dist repos can now be generated on volumes other than the main one.

+ Use the ``--volume`` option to the ``dist-repo`` command to do so.

+ 

+ Generally you want the repo to be on the same volume as the rpms it will

+ contain.

+ Dist repos hard link (same volume) or copy (different volume) their rpms into

+ place.

+ Using the appropriate volume can drastically improve the efficiency, both in

+ generation time and space consumption.

+ 

+ 

+ **Minor gc optimizations**

+ 

+ | PR: https://pagure.io/koji/pull-request/1337

+ | PR: https://pagure.io/koji/pull-request/1442

+ | PR: https://pagure.io/koji/pull-request/1437

+ 

+ This change speeds up portions of garbage collection by making the

+ ``build_references`` check lazy by default.

+ 

+ 

+ 

+ **Rollback errors in multiCall**

+ 

+ | PR: https://pagure.io/koji/pull-request/1358

+ 

+ If one of the calls in a multicall raises an error, then the transaction will

+ be rolled back to the start of that call before Koji proceeds to the next call.

+ This matches the behavior of normal calls more closely.

+ 

+ Multicalls are still handled within single database transaction.

+ 

+ 

+ 

+ **Support tilde in search**

+ 

+ | PR: https://pagure.io/koji/pull-request/1297

+ 

+ 

+ The tilde character is no longer prohibited in search terms.

+ 

+ 

+ 

+ **Remove 'keepalive' option**

+ 

+ | PR: https://pagure.io/koji/pull-request/1277

+ 

+ The ``keepalive`` setting is no longer used anywhere in koji.

+ It has been removed.

file modified
+111 -1
@@ -81,7 +81,7 @@ 

  %define release %{baserelease}

  %endif

  Name: koji

- Version: 1.17.0

+ Version: 1.18.0

  Release: %{release}%{?dist}

  License: LGPLv2 and GPLv2+

  # the included arch lib from yum's rpmUtils is GPLv2+
@@ -713,6 +713,116 @@ 

  %endif

  

  %changelog

+ * Fri Aug  9 2019 Mike McLean <mikem at redhat.com> - 1.18.0-1

+ - PR#1606: pull owner from correct place

+ - PR#1602: copy updated policy for reserved cg builds

+ - PR#1601: fix recycling build due to cg

+ - PR#1597: Backward-compatible fix for CG import

+ - PR#1591: secrets import is missing 'else' variant

+ - PR#1555: use _writeInheritanceData in _create_tag

+ - PR#1580: cli: verify user in block-notification command

+ - PR#1578: cli:fix typo in mock-config

+ - PR#1464: API for reserving NVRs for content generators

+ - PR#898: Add support for tag/target macros for Mageia

+ - PR#1544: use RawConfigParser for kojid

+ - PR#863: cli: change --force to real bool arg for add-tag-inheritance

+ - PR#1253: cli: add option for custom cert location

+ - PR#1353: Create db index for listTagged

+ - PR#1375: docs: add architecture diagram

+ - PR#892: cli: also load plugins from ~/.koji/plugins

+ - PR#1516: kojibuilder: Pass mergerepo_c --all for bare mode as well.

+ - PR#1524: set module_hotfixes=1 in yum.conf via tag config

+ - PR#1417: notification's optouts

+ - PR#1515: add debug message to new multicall to match original

+ - PR#1480: Add raw-gz and compressed QCOW2 archive types.

+ - PR#1260: use LANG=C for running all tests

+ - PR#1447: handle deleted tags in kojira

+ - PR#1513: Allow hub policy to match version and release

+ - PR#1462: rebuildSRPM task

+ - PR#1498: Pass bytes to md5_constructor

+ - PR#1502: Don't pass block list in bare merge mode

+ - PR#1489: pass bytes to sha1 constructor

+ - PR#1499: remove merge option from edit-external-repo

+ - PR#1427: Fix typo in getArchiveTypes docstring

+ - PR#957: New multicall interface

+ - PR#1280: put fix_pyver before printing command help

+ - PR#1415: New 'buildtype' test for policies

+ - PR#1258: retain old search pattern in web ui

+ - PR#1479: use better index for sessions

+ - PR#1279: let hub decide, what headers are supported

+ - PR#1454: introduce host-admin permission + docs

+ - PR#1303: fix history display for parallel host_channels updates

+ - PR#1278: createrepo_c is used by default now

+ - PR#1449: show load/capacity in list-channels

+ - PR#1476: Allow taginfo cli to use tag IDs; fixed Inheritance printing bug

+ - PR#1445: turn back on test skipped due to coverage bug

+ - PR#1452: fix parentheses for tuple in _writeInheritanceData

+ - PR#1456: deprecate BuildRoot.uploadDir method

+ - PR#1461: check existence of tag_id in getInheritanceData

+ - PR#1471: list-hosts shouldn't error on empty list

+ - PR#1273: Allow generating separate src repo for build repos

+ - PR#1255: always check existence of tag in setInheritanceData

+ - PR#1256: add strict option to getTaskChildren

+ - PR#1257: fail runroot task on non-existing tag

+ - PR#1272: check architecture names for mistakes

+ - PR#1322: Reduce duplicate "fixEncoding" code

+ - PR#1327: volume option for dist-repo

+ - PR#1442: delete_build: handle results of lazy build_references call

+ - PR#1425: add --show-channels listing to list-hosts

+ - PR#1432: py2.6 compatibility fix

+ - PR#1434: hub: fix check_fields and duplicated parent_id in _writeInheritanceData

+ - PR#1439: user correct column in sql (getTask)

+ - PR#1437: fix table name in build_references query

+ - PR#1414: Fix jenkins config for new python mock

+ - PR#1411: handle bare merge mode

+ - PR#1410: build_srpm: Wait until after running the sources command to check for alt_sources_dir

+ - PR#1383: display task durations in webui

+ - PR#1358: rollback errors in multiCall

+ - PR#1413: Makefile: print correct urls for test coverage

+ - PR#1409: Fix SQL after introduction of host_config

+ - PR#1324: createEmptyBuild errors for non-existent user

+ - PR#1406: fix mapping iteration in getFullInheritance

+ - PR#1398: kojid: Download only 'origin'

+ - PR#1365: Check CLI arguments for enable/disable host

+ - PR#1390: CLI list-channels sorted output

+ - PR#1389: block_pkglist compatibility fix

+ - PR#1376: use context manager for open in CLI

+ - PR#1392: Replace references to latest-pkg with latest-build

+ - PR#1386: scale task_avail_delay based on bin rank

+ - PR#1363: Use createrepo_update even for first repo run

+ - PR#1368: update test requirements in jenkins

+ - PR#1374: honor mock.package_manager tag setting in mock-config cli

+ - PR#1387: remove unused variable

+ - PR#1143: hub: document CG access method arguments

+ - PR#1169: docs: use systemctl enable --now for postgres and kojid

+ - PR#1155: hub: document addHost and editHost arguments

+ - PR#1242: kojid.conf documentation

+ - PR#1340: Update server doc for newer TLS and event worker

+ - PR#1359: docs: remove "TBD" sections

+ - PR#1360: docs: remove mod_python references

+ - PR#1361: docs: kojirepod -> kojira

+ - PR#1370: add vhdx archivetype

+ - PR#1331: provide lower level versions of build_target functions

+ - PR#1348: rm old references to Mozilla

+ - PR#1297: Support tilde in search

+ - PR#1356: kojira: fix iteration over repos in py3

+ - PR#1342: Remove python2.4 OptionParse fix

+ - PR#1347: Fix hub startup handling

+ - PR#1346: Rely on ozif_enabled switch in BaseImageTask

+ - PR#1344: add .tgz to list of tar's possible extensions

+ - PR#1086: hub: unittest for get_external_repos

+ - PR#1170: docs: koji package provides schema.sql file

+ - PR#1281: remove urlescape from package name

+ - PR#1304: hub: document setInheritanceData arguments

+ - PR#1277: Remove 'keepalive' option

+ - PR#1330: fix docs typos

+ - PR#1339: fix typo in usage of six's import of MIMEText

+ - PR#1337: minor gc optimizations

+ - PR#1254: doc: Include AnyStor mention to 'koji runs here' doc

+ - PR#1325: run py3 tests in CI by default

+ - PR#1326: README: link to Pungi project instead of mash

+ - PR#1329: Update plugin doc (confusing sentence)

+ 

  * Wed Mar  6 2019 Mike McLean <mikem at redhat.com> - 1.17.0-1

  - PR#1320: also remove nonprintable changelog chars in py3

  - PR#1293: fix dict encoding in py3

file modified
+1 -1
@@ -32,7 +32,7 @@ 

  

  setup(

      name="koji",

-     version="1.17.0",

+     version="1.18.0",

      description=("Koji is a system for building and tracking RPMS. The base"

                   " package contains shared libraries and the command-line"

                   " interface."),

This PR contains release notes and other minutia for the upcoming 1.18 release

8 new commits added

  • update version for PyPi
  • migration note about permissions
  • cleanup
  • more relnotes updates
  • initial release notes
  • bump version, add changelog
  • migration doc
  • add new indexes in schema upgrade
4 years ago

1 new commit added

  • typo
4 years ago

rebased onto af02d8faa7603443764d8adddc13958812da9d4c

4 years ago

1 new commit added

  • update changelog again
4 years ago

I've pushed the docs changes to my fork if anyone would like to view them that way:

https://docs.pagure.org/fork/mikem/koji/

1 new commit added

  • another changelog update
4 years ago

1 new commit added

  • include a few more PRs in release notes
4 years ago

rebased onto b9c1af4

4 years ago

Commit c765733 fixes this pull-request

Pull-Request has been merged by mikem

4 years ago