From 413e683daa85f77277b35268766911e59d59653a Mon Sep 17 00:00:00 2001 From: Tomas Kopecek Date: Oct 31 2019 10:37:40 +0000 Subject: split docs to subdirectories Fixes: https://pagure.io/koji/issue/1715 --- diff --git a/docs/source/CVE-2017-1002153.rst b/docs/source/CVE-2017-1002153.rst deleted file mode 100644 index f1360e6..0000000 --- a/docs/source/CVE-2017-1002153.rst +++ /dev/null @@ -1,26 +0,0 @@ -================ -CVE-2017-1002153 -================ - -Koji 1.13.0 does not properly validate SCM paths. - - -Summary -------- - -Koji 1.13.0 does not properly validate SCM paths, allowing an attacker to work around blacklisted paths for build submission. - - -Bug fix -------- - -Koji versions 1.14.0 and forward contain the fix. - -This bug was tracked as `issue#563 `_ - -Links ------ - -Fixed versions can be found at our releases page: - - `https://pagure.io/koji/releases `_ diff --git a/docs/source/CVE-2018-1002150-FAQ.rst b/docs/source/CVE-2018-1002150-FAQ.rst deleted file mode 100644 index ba2ff56..0000000 --- a/docs/source/CVE-2018-1002150-FAQ.rst +++ /dev/null @@ -1,64 +0,0 @@ -======================== -FAQ for CVE-2018-1002150 -======================== - -Following are answers to some questions regarding CVE-2018-1002150 -for Koji. If you haven’t already, you should read the -:doc:`announcement `. - -If you have questions not covered here or in the announcement, please -ask them on the koji-devel mailing list. - - https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.org/ - -Q: Does this issue affect Koji clients or builders? - - The issue only affects the Koji hub. - -Q: How can I tell if I’ve been attacked? - - We don’t know of any exploits in the wild. However, to be - safe, we will release an intrusion detection document in a few - days. - -Q: Where are the fixed versions? - - | Koji versions before 1.12.0 are unaffected - | For Koji 1.12, 1.12.1 and higher includes the fix - | For Koji 1.13, 1.13.1 and higher includes the fix - | For Koji 1.14, 1.14.1 and higher includes the fix - | For Koji 1.15, 1.15.1 and higher includes the fix - | Koji 1.16.0 and higher will include the fix - - You can find all of these versions on our releases page: - - https://pagure.io/koji/releases - -Q: What about versions before 1.12.0? - - Koji versions before 1.12.0 are unaffected (they don't have the dist-repo - feature). However, it would be wise to update your system to the current - version. - -Q: What can be done with this exploit? - - The attacker can trick Koji into moving files around. These can be - almost any file that the httpd user can write. The attacker could - use this to corrupt Koji’s file store or to reveal any secret files - that the httpd user can read. - -Q: Can the attacker execute arbitrary code? - - Not that we know of. - -Q: Where can I get more help? - - You can ask questions on the koji-devel mailing list - (`koji-devel@fedorahosted.org `_). - - For real time communication, we have the #koji IRC channel on - `Freenode `_. - The best time to ask would be during the Koji devel team - “office hours”, which are held each Tuesday and Thursday from - 10-11am eastern time. - diff --git a/docs/source/CVE-2018-1002150.rst b/docs/source/CVE-2018-1002150.rst deleted file mode 100644 index 19006da..0000000 --- a/docs/source/CVE-2018-1002150.rst +++ /dev/null @@ -1,93 +0,0 @@ -================ -CVE-2018-1002150 -================ - -Dist repo call missing authorization check allowing filesystem manipulation - - -.. toctree:: - :hidden: - - CVE-2018-1002150-FAQ - -Summary -------- - -This is a critical security bug. - -From versions 1.12.0 to 1.15.0, the Koji hub did not perform proper -access checks for the hub.distRepoMove call. By passing carefully -constructed arguments to the call, an unauthenticated user can trick -Koji into moving content around that it should not. This could result in -corrupting any files that the httpd process can write to, or revealing -any files that the httpd process can read. If the user can authenticate -(at any privilege level), then they can use this mechanism to replace a -file with one that they have uploaded. - -Workaround ----------- - -*We strongly recommend that all Koji admins implement this workaround -immediately.* This workaround will effectively disable dist-repo -functionality. - -Because use of the hub.distRepoMove call requires a valid dist repo that -exists on disk, exploitation can be blocked by ensuring that there are -none. There are many ways this might be done. We recommend the -following: - -1. Move the repos-dist directory to another location (if it exists) -2. Replace it with a plain text file warning of the situation. Do not - skip this step. - -For example:: - - $ cd /mnt/koji - $ mv repos-dist repos-dist.old - $ echo "DO NOT REMOVE. CVE-2018-1002150" > repos-dist - $ ls -l /mnt/koji/repos-dist - -rw-r--r--. 1 root root 32 Mar 19 14:35 /mnt/koji/repos-dist - -When applying this workaround, make sure to take both steps. If you do -not, then the system will recreate the directory if anyone creates -a new dist repo. - - -Bug fix -------- - -*Note: because code fixes can take time to deploy, we strongly recommend -that all admins apply the above workaround first. The workaround can be -easily undone once the fix is in place.* - -We are releasing updates for each affected version of Koji to fix this -bug. The following `releases `_ all -contain the fix: - -- 1.15.1 -- 1.14.1 -- 1.13.1 -- 1.12.1 - -Versions prior to 1.12.0 are not vulnerable because they do not have the -dist-repo feature. Also, the legacy-py24 branch is unaffected since it -is client-only (no hub). - -For users who have customized their Koji code, we recommend rebasing -your work onto the appropriate update release. If this is not feasible, -the patch should be very easy to apply. Please see `issue -#850 `_ for the code details. - -As with all changes to hub code, you must restart httpd for the changes -to take effect. - -Links ------ - -Fixed versions can be found at our releases page: - - https://pagure.io/koji/releases - -Questions and answers about this issue - - :doc:`CVE-2018-1002150-FAQ` diff --git a/docs/source/CVE-2018-1002161-FAQ.rst b/docs/source/CVE-2018-1002161-FAQ.rst deleted file mode 100644 index bb7b83d..0000000 --- a/docs/source/CVE-2018-1002161-FAQ.rst +++ /dev/null @@ -1,66 +0,0 @@ -======================== -FAQ for CVE-2018-1002161 -======================== - -Following are answers to some questions regarding CVE-2018-1002161 -for Koji. If you haven’t already, you should read the -:doc:`announcement `. - -If you have questions not covered here or in the announcement, please -ask them on the koji-devel mailing list. - - https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.org/ - -Q: Does this issue affect Koji clients or builders? - - The issue only affects the Koji hub. - -Q: Which versions of Koji are affected? - - All previous versions of Koji are affected, except for the legacy-py24 - branch because it contains no hub code. - -Q: Where are the fixed versions? - - | For Koji 1.11, 1.11.1 and higher include the fix - | For Koji 1.12, 1.12.2 and higher include the fix - | For Koji 1.13, 1.13.2 and higher include the fix - | For Koji 1.14, 1.14.2 and higher include the fix - | For Koji 1.15, 1.15.2 and higher include the fix - | For Koji 1.16.2 and higher include the fix - - You can find all of these versions on our releases page: - - https://pagure.io/koji/releases - -Q: What about older versions? - - We have only backported the fix to Koji versions released in the past few - years. If you are still using a very old version of Koji, we strongly - recommend that you shut it down and migrate to a newer version. - -Q: What can be done with this exploit? - - The attacker can directly manipulate the database as they see fit. This - would, among other things, allow them to gain the admin permission within - Koji. They could destroy or corrupt the database, add new builds, replace - existing builds, or any number of other things. - -Q: Can the attacker execute arbitrary code? - - On the hub, not that we know of. - - However, they could create arbitrary tasks, which would be run by the build - hosts. - -Q: Where can I get more help? - - You can ask questions on the koji-devel mailing list - (`koji-devel@fedorahosted.org `_). - - For real time communication, we have the #koji IRC channel on - `Freenode `_. - The best time to ask would be during the Koji devel team - “office hours”, which are held each Tuesday and Thursday from - 10-11am eastern time. - diff --git a/docs/source/CVE-2018-1002161.rst b/docs/source/CVE-2018-1002161.rst deleted file mode 100644 index 33fb6c9..0000000 --- a/docs/source/CVE-2018-1002161.rst +++ /dev/null @@ -1,66 +0,0 @@ -================ -CVE-2018-1002161 -================ - -SQL injection in multiple remote calls - -.. toctree:: - :hidden: - - CVE-2018-1002161-FAQ - - -Summary -------- - -This is a critical security bug. - -Multiple xmlrpc call handlers in Koji’s hub code contain SQL injection bugs. By -passing carefully constructed arguments to these calls, an unauthenticated user -can issue arbitrary SQL commands to Koji’s database. This gives the attacker -broad ability to manipulate or destroy data. - -There is no known workaround. All Koji admins are encouraged to update to a -fixed version as soon as possible. - - - -Bug fix -------- - -Note: because code fixes can take time to deploy, we recommend -that all admins shut down their Koji hub instances until the fix -can be applied. - -We are releasing updates for several recent versions of Koji to fix this -bug. The following `releases `_ all -contain the fix: - -- 1.16.2 -- 1.15.2 -- 1.14.2 -- 1.13.2 -- 1.12.2 -- 1.11.1 - -Note: the legacy-py24 branch is unaffected since it -is client-only (no hub). - -For users who have customized their Koji code, we recommend rebasing -your work onto the appropriate update release. If this is not feasible, -the patch should be very easy to apply. Please see `issue -#1183 `_ for the code details. - -As with all changes to hub code, you must restart httpd for the changes -to take effect. - -Links ------ - -Fixed versions can be found at our releases page: - - https://pagure.io/koji/releases - -Questions and answers about this issue - - :doc:`CVE-2018-1002161-FAQ` diff --git a/docs/source/CVE-2019-17109.rst b/docs/source/CVE-2019-17109.rst deleted file mode 100644 index 9784167..0000000 --- a/docs/source/CVE-2019-17109.rst +++ /dev/null @@ -1,53 +0,0 @@ -============== -CVE-2019-17109 -============== - -Koji hub allows arbitrary upload destinations - - -Summary -------- - -The way that the hub code validates upload paths allows for an attacker to -choose an arbitrary destination for the uploaded file. - -Uploading still requires login. However, an attacker with credentials could -damage the integrity of the Koji system. - -There is no known workaround. All Koji admins are encouraged to update to a -fixed version as soon as possible. - - - -Bug fix -------- - -We are releasing updates for affected versions of Koji from within the -past two years. -The following releases all contain the fix: - -- 1.18.1 -- 1.17.1 -- 1.16.3 -- 1.15.3 -- 1.14.3 - -Note: the legacy-py24 branch is unaffected since it is client-only (no hub). - -Anyone using a Koji version older than two years should update to a more -current version as soon as possible. - -For users who have customized their Koji code, we recommend rebasing your work -onto the appropriate update release. Please see Koji -`issue #1634 `_ for the code details. - -As with all changes to hub code, you must restart httpd for the changes to -take effect. - - -Links ------ - -Fixed versions can be found at our releases page: - - https://pagure.io/koji/releases diff --git a/docs/source/CVEs.rst b/docs/source/CVEs.rst deleted file mode 100644 index 86b06ab..0000000 --- a/docs/source/CVEs.rst +++ /dev/null @@ -1,11 +0,0 @@ -========= -Koji CVEs -========= - -.. toctree:: - :titlesonly: - - CVE-2019-17109 - CVE-2018-1002161 - CVE-2018-1002150 - CVE-2017-1002153 diff --git a/docs/source/CVEs/CVE-2017-1002153.rst b/docs/source/CVEs/CVE-2017-1002153.rst new file mode 100644 index 0000000..f1360e6 --- /dev/null +++ b/docs/source/CVEs/CVE-2017-1002153.rst @@ -0,0 +1,26 @@ +================ +CVE-2017-1002153 +================ + +Koji 1.13.0 does not properly validate SCM paths. + + +Summary +------- + +Koji 1.13.0 does not properly validate SCM paths, allowing an attacker to work around blacklisted paths for build submission. + + +Bug fix +------- + +Koji versions 1.14.0 and forward contain the fix. + +This bug was tracked as `issue#563 `_ + +Links +----- + +Fixed versions can be found at our releases page: + + `https://pagure.io/koji/releases `_ diff --git a/docs/source/CVEs/CVE-2018-1002150-FAQ.rst b/docs/source/CVEs/CVE-2018-1002150-FAQ.rst new file mode 100644 index 0000000..34c2070 --- /dev/null +++ b/docs/source/CVEs/CVE-2018-1002150-FAQ.rst @@ -0,0 +1,63 @@ +======================== +FAQ for CVE-2018-1002150 +======================== + +Following are answers to some questions regarding CVE-2018-1002150 +for Koji. If you haven’t already, you should read the +:doc:`announcement `. + +If you have questions not covered here or in the announcement, please +ask them on the koji-devel mailing list. + + https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.org/ + +Q: Does this issue affect Koji clients or builders? + + The issue only affects the Koji hub. + +Q: How can I tell if I’ve been attacked? + + We don’t know of any exploits in the wild. However, to be + safe, we will release an intrusion detection document in a few + days. + +Q: Where are the fixed versions? + + | Koji versions before 1.12.0 are unaffected + | For Koji 1.12, 1.12.1 and higher includes the fix + | For Koji 1.13, 1.13.1 and higher includes the fix + | For Koji 1.14, 1.14.1 and higher includes the fix + | For Koji 1.15, 1.15.1 and higher includes the fix + | Koji 1.16.0 and higher will include the fix + + You can find all of these versions on our releases page: + + https://pagure.io/koji/releases + +Q: What about versions before 1.12.0? + + Koji versions before 1.12.0 are unaffected (they don't have the dist-repo + feature). However, it would be wise to update your system to the current + version. + +Q: What can be done with this exploit? + + The attacker can trick Koji into moving files around. These can be + almost any file that the httpd user can write. The attacker could + use this to corrupt Koji’s file store or to reveal any secret files + that the httpd user can read. + +Q: Can the attacker execute arbitrary code? + + Not that we know of. + +Q: Where can I get more help? + + You can ask questions on the koji-devel mailing list + (`koji-devel@fedorahosted.org `_). + + For real time communication, we have the #koji IRC channel on + `Freenode `_. + The best time to ask would be during the Koji devel team + “office hours”, which are held each Tuesday and Thursday from + 10-11am eastern time. diff --git a/docs/source/CVEs/CVE-2018-1002150.rst b/docs/source/CVEs/CVE-2018-1002150.rst new file mode 100644 index 0000000..19006da --- /dev/null +++ b/docs/source/CVEs/CVE-2018-1002150.rst @@ -0,0 +1,93 @@ +================ +CVE-2018-1002150 +================ + +Dist repo call missing authorization check allowing filesystem manipulation + + +.. toctree:: + :hidden: + + CVE-2018-1002150-FAQ + +Summary +------- + +This is a critical security bug. + +From versions 1.12.0 to 1.15.0, the Koji hub did not perform proper +access checks for the hub.distRepoMove call. By passing carefully +constructed arguments to the call, an unauthenticated user can trick +Koji into moving content around that it should not. This could result in +corrupting any files that the httpd process can write to, or revealing +any files that the httpd process can read. If the user can authenticate +(at any privilege level), then they can use this mechanism to replace a +file with one that they have uploaded. + +Workaround +---------- + +*We strongly recommend that all Koji admins implement this workaround +immediately.* This workaround will effectively disable dist-repo +functionality. + +Because use of the hub.distRepoMove call requires a valid dist repo that +exists on disk, exploitation can be blocked by ensuring that there are +none. There are many ways this might be done. We recommend the +following: + +1. Move the repos-dist directory to another location (if it exists) +2. Replace it with a plain text file warning of the situation. Do not + skip this step. + +For example:: + + $ cd /mnt/koji + $ mv repos-dist repos-dist.old + $ echo "DO NOT REMOVE. CVE-2018-1002150" > repos-dist + $ ls -l /mnt/koji/repos-dist + -rw-r--r--. 1 root root 32 Mar 19 14:35 /mnt/koji/repos-dist + +When applying this workaround, make sure to take both steps. If you do +not, then the system will recreate the directory if anyone creates +a new dist repo. + + +Bug fix +------- + +*Note: because code fixes can take time to deploy, we strongly recommend +that all admins apply the above workaround first. The workaround can be +easily undone once the fix is in place.* + +We are releasing updates for each affected version of Koji to fix this +bug. The following `releases `_ all +contain the fix: + +- 1.15.1 +- 1.14.1 +- 1.13.1 +- 1.12.1 + +Versions prior to 1.12.0 are not vulnerable because they do not have the +dist-repo feature. Also, the legacy-py24 branch is unaffected since it +is client-only (no hub). + +For users who have customized their Koji code, we recommend rebasing +your work onto the appropriate update release. If this is not feasible, +the patch should be very easy to apply. Please see `issue +#850 `_ for the code details. + +As with all changes to hub code, you must restart httpd for the changes +to take effect. + +Links +----- + +Fixed versions can be found at our releases page: + + https://pagure.io/koji/releases + +Questions and answers about this issue + + :doc:`CVE-2018-1002150-FAQ` diff --git a/docs/source/CVEs/CVE-2018-1002161-FAQ.rst b/docs/source/CVEs/CVE-2018-1002161-FAQ.rst new file mode 100644 index 0000000..426994b --- /dev/null +++ b/docs/source/CVEs/CVE-2018-1002161-FAQ.rst @@ -0,0 +1,65 @@ +======================== +FAQ for CVE-2018-1002161 +======================== + +Following are answers to some questions regarding CVE-2018-1002161 +for Koji. If you haven’t already, you should read the +:doc:`announcement `. + +If you have questions not covered here or in the announcement, please +ask them on the koji-devel mailing list. + + https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.org/ + +Q: Does this issue affect Koji clients or builders? + + The issue only affects the Koji hub. + +Q: Which versions of Koji are affected? + + All previous versions of Koji are affected, except for the legacy-py24 + branch because it contains no hub code. + +Q: Where are the fixed versions? + + | For Koji 1.11, 1.11.1 and higher include the fix + | For Koji 1.12, 1.12.2 and higher include the fix + | For Koji 1.13, 1.13.2 and higher include the fix + | For Koji 1.14, 1.14.2 and higher include the fix + | For Koji 1.15, 1.15.2 and higher include the fix + | For Koji 1.16.2 and higher include the fix + + You can find all of these versions on our releases page: + + https://pagure.io/koji/releases + +Q: What about older versions? + + We have only backported the fix to Koji versions released in the past few + years. If you are still using a very old version of Koji, we strongly + recommend that you shut it down and migrate to a newer version. + +Q: What can be done with this exploit? + + The attacker can directly manipulate the database as they see fit. This + would, among other things, allow them to gain the admin permission within + Koji. They could destroy or corrupt the database, add new builds, replace + existing builds, or any number of other things. + +Q: Can the attacker execute arbitrary code? + + On the hub, not that we know of. + + However, they could create arbitrary tasks, which would be run by the build + hosts. + +Q: Where can I get more help? + + You can ask questions on the koji-devel mailing list + (`koji-devel@fedorahosted.org `_). + + For real time communication, we have the #koji IRC channel on + `Freenode `_. + The best time to ask would be during the Koji devel team + “office hours”, which are held each Tuesday and Thursday from + 10-11am eastern time. diff --git a/docs/source/CVEs/CVE-2018-1002161.rst b/docs/source/CVEs/CVE-2018-1002161.rst new file mode 100644 index 0000000..33fb6c9 --- /dev/null +++ b/docs/source/CVEs/CVE-2018-1002161.rst @@ -0,0 +1,66 @@ +================ +CVE-2018-1002161 +================ + +SQL injection in multiple remote calls + +.. toctree:: + :hidden: + + CVE-2018-1002161-FAQ + + +Summary +------- + +This is a critical security bug. + +Multiple xmlrpc call handlers in Koji’s hub code contain SQL injection bugs. By +passing carefully constructed arguments to these calls, an unauthenticated user +can issue arbitrary SQL commands to Koji’s database. This gives the attacker +broad ability to manipulate or destroy data. + +There is no known workaround. All Koji admins are encouraged to update to a +fixed version as soon as possible. + + + +Bug fix +------- + +Note: because code fixes can take time to deploy, we recommend +that all admins shut down their Koji hub instances until the fix +can be applied. + +We are releasing updates for several recent versions of Koji to fix this +bug. The following `releases `_ all +contain the fix: + +- 1.16.2 +- 1.15.2 +- 1.14.2 +- 1.13.2 +- 1.12.2 +- 1.11.1 + +Note: the legacy-py24 branch is unaffected since it +is client-only (no hub). + +For users who have customized their Koji code, we recommend rebasing +your work onto the appropriate update release. If this is not feasible, +the patch should be very easy to apply. Please see `issue +#1183 `_ for the code details. + +As with all changes to hub code, you must restart httpd for the changes +to take effect. + +Links +----- + +Fixed versions can be found at our releases page: + + https://pagure.io/koji/releases + +Questions and answers about this issue + + :doc:`CVE-2018-1002161-FAQ` diff --git a/docs/source/CVEs/CVE-2019-17109.rst b/docs/source/CVEs/CVE-2019-17109.rst new file mode 100644 index 0000000..9784167 --- /dev/null +++ b/docs/source/CVEs/CVE-2019-17109.rst @@ -0,0 +1,53 @@ +============== +CVE-2019-17109 +============== + +Koji hub allows arbitrary upload destinations + + +Summary +------- + +The way that the hub code validates upload paths allows for an attacker to +choose an arbitrary destination for the uploaded file. + +Uploading still requires login. However, an attacker with credentials could +damage the integrity of the Koji system. + +There is no known workaround. All Koji admins are encouraged to update to a +fixed version as soon as possible. + + + +Bug fix +------- + +We are releasing updates for affected versions of Koji from within the +past two years. +The following releases all contain the fix: + +- 1.18.1 +- 1.17.1 +- 1.16.3 +- 1.15.3 +- 1.14.3 + +Note: the legacy-py24 branch is unaffected since it is client-only (no hub). + +Anyone using a Koji version older than two years should update to a more +current version as soon as possible. + +For users who have customized their Koji code, we recommend rebasing your work +onto the appropriate update release. Please see Koji +`issue #1634 `_ for the code details. + +As with all changes to hub code, you must restart httpd for the changes to +take effect. + + +Links +----- + +Fixed versions can be found at our releases page: + + https://pagure.io/koji/releases diff --git a/docs/source/CVEs/CVEs.rst b/docs/source/CVEs/CVEs.rst new file mode 100644 index 0000000..86b06ab --- /dev/null +++ b/docs/source/CVEs/CVEs.rst @@ -0,0 +1,11 @@ +========= +Koji CVEs +========= + +.. toctree:: + :titlesonly: + + CVE-2019-17109 + CVE-2018-1002161 + CVE-2018-1002150 + CVE-2017-1002153 diff --git a/docs/source/index.rst b/docs/source/index.rst index bbbd99c..086842e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -27,9 +27,9 @@ Contents external_repo_server_bootstrap image_build misc - release_notes - migrations - CVEs + release_notes/release_notes + migrations/migrations + CVEs/CVEs runs_here server_bootstrap server_howto diff --git a/docs/source/migrating_to_1.10.rst b/docs/source/migrating_to_1.10.rst deleted file mode 100644 index 7ae49e6..0000000 --- a/docs/source/migrating_to_1.10.rst +++ /dev/null @@ -1,85 +0,0 @@ -Migrating to Koji 1.10 -====================== - -.. reStructured Text formatted - -The 1.10 release of Koji includes a few changes that you should consider when -migrating. - -DB Updates ----------- - -The new ``tag_extra`` table tracks extra data for tags. - -There is a new entry in the ``channels`` table and some additions and updates to -the ``archivetypes`` table. - -As in previous releases, we provide a migration script that updates the -database. - -:: - - # psql koji koji DROP TABLE imageinfo_listing; - koji=> DROP TABLE imageinfo; - # rm -rf /mnt/koji/images - - -Command line changes --------------------- - -For clarity and consistency, all of the ``-pkg`` commands have been renamed to -``-build`` commands. - -:: - - latest-pkg -> latest-build - move-pkg -> move-build - tag-pkg -> tag-build - untag-pkg -> untag-build - -For backwards compatibility, the old commands names are also recognized. - -A new command has been added, ``remove-pkg``. - -Several commands have been modified to support images. - -The ``spin-livecd`` and ``spin-appliance`` commands now require additional -arguments. These arguments specify the name and version to use for the image. - - -New kojira options ------------------- - -The following options are new to kojira: - -:: - - max_delete_processes - max_repo_tasks_maven - -Previously, kojira ran as a single process and repo deletions could potentially -slow things down (particularly for Maven-enabled repos). Now kojira spawns -a separate process to handle these deletions. The ``max_delete_processes`` -determines how many such processes it will launch at one time. - -When Maven-enabled repos are in use, they can potentially take a very long time -to regenerate. If a number of these pile up it can severely slow down -regeneration of non-Maven repos. The ``max_repo_tasks_maven`` limits how many -Maven repos kojira will attempt to regenerate at once. - -Also the following kojira option has been removed: - -:: - - prune_batch_size - diff --git a/docs/source/migrating_to_1.9.rst b/docs/source/migrating_to_1.9.rst deleted file mode 100644 index fbfb27c..0000000 --- a/docs/source/migrating_to_1.9.rst +++ /dev/null @@ -1,86 +0,0 @@ -Migrating to Koji 1.9 -===================== - -.. reStructured Text formatted - -The 1.9 release of Koji includes a few changes that you should consider when -migrating. - -DB Updates ----------- - -ImageFactory support introduced some new archive types. These have been added to -the ``archivetypes`` table. The inaccurate ``vmx`` entry has been removed. - -As in previous releases, we provide a migration script that updates the -database. - -:: - - # psql koji koji DROP TABLE imageinfo_listing; + koji=> DROP TABLE imageinfo; + # rm -rf /mnt/koji/images + + +Command line changes +-------------------- + +For clarity and consistency, all of the ``-pkg`` commands have been renamed to +``-build`` commands. + +:: + + latest-pkg -> latest-build + move-pkg -> move-build + tag-pkg -> tag-build + untag-pkg -> untag-build + +For backwards compatibility, the old commands names are also recognized. + +A new command has been added, ``remove-pkg``. + +Several commands have been modified to support images. + +The ``spin-livecd`` and ``spin-appliance`` commands now require additional +arguments. These arguments specify the name and version to use for the image. + + +New kojira options +------------------ + +The following options are new to kojira: + +:: + + max_delete_processes + max_repo_tasks_maven + +Previously, kojira ran as a single process and repo deletions could potentially +slow things down (particularly for Maven-enabled repos). Now kojira spawns +a separate process to handle these deletions. The ``max_delete_processes`` +determines how many such processes it will launch at one time. + +When Maven-enabled repos are in use, they can potentially take a very long time +to regenerate. If a number of these pile up it can severely slow down +regeneration of non-Maven repos. The ``max_repo_tasks_maven`` limits how many +Maven repos kojira will attempt to regenerate at once. + +Also the following kojira option has been removed: + +:: + + prune_batch_size diff --git a/docs/source/migrations/migrating_to_1.9.rst b/docs/source/migrations/migrating_to_1.9.rst new file mode 100644 index 0000000..fbfb27c --- /dev/null +++ b/docs/source/migrations/migrating_to_1.9.rst @@ -0,0 +1,86 @@ +Migrating to Koji 1.9 +===================== + +.. reStructured Text formatted + +The 1.9 release of Koji includes a few changes that you should consider when +migrating. + +DB Updates +---------- + +ImageFactory support introduced some new archive types. These have been added to +the ``archivetypes`` table. The inaccurate ``vmx`` entry has been removed. + +As in previous releases, we provide a migration script that updates the +database. + +:: + + # psql koji koji [ ...] + (Specify the --help global option for a list of other help options) + + Options: + -h, --help show this help message and exit + +Enhancements to restart-hosts +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/472 + +The `restart-hosts` command is used by admins to safely restart the build hosts +after a configuration change. + +Because multiple restarts can conflict, the command will now exit with a error +if a restart is already underway (can be overridden with --force). + +There are now options to limit the restart to a given channel or arch. + +The command now has a timeout option, which defaults to 24hrs. + +User-Agent header +^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/393 +| Issue: https://pagure.io/koji/issue/392 + +Previously the Koji client library reported a confusingly out-of-date value +in the ``User-Agent`` header. Now it simply reports the major version. + +raise error on non-existing profile +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/375 +| Issue: https://pagure.io/koji/issue/370 + +If the requested client profile is not configured, the library will raise an +error, rather than proceeding with default values. + +See also: :doc:`../profiles` + + +Changes to the Web interface +---------------------------- + +Build Log Display +^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/471 + +The build info pages now display the log files for a build (instead of linking +directly to the directory on the download server). This works for all builds, +including those imported by content generators. + + +Builder changes +--------------- + +Configuring mock chroot behavior +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/400 +| Issue: https://pagure.io/koji/issue/398 + +Koji now supports using mock's --new-chroot option on a per-tag basis. +For details see: :ref:`tuning-mock-per-tag` + +pre/postSCMCheckout callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The callback interface is used by plugins to hook into various Koji operations. +With this release we have added callbacks in the builder daemon for before and +after source checkout: ``preSCMCheckout`` and ``postSCMCheckout``. + +Extended allowed_scms format +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/421 + +The allowed_scms option now accepts entries like: + +:: + + !host:repository + +to explicitly block a host:repository pattern. + +See also: :ref:`scm-config` + + +System changes +-------------- + +mod_auth_gssapi required +^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/444 + +On modern platforms, both koji-hub and koji-web now require +mod_auth_gssapi instead of mod_auth_kerb. + + +Longer tag names +^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/388 +| Issue: https://pagure.io/koji/issue/369 + +Previously, tag names were limited to 50 characters. They are now limited +to 256 characters. diff --git a/docs/source/release_notes/release_notes_1.14.rst b/docs/source/release_notes/release_notes_1.14.rst new file mode 100644 index 0000000..6fa2fb6 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.14.rst @@ -0,0 +1,310 @@ +Koji 1.14 Release Notes +======================= + +Migrating from Koji 1.13 +------------------------ + +For details on migrating see :doc:`../migrations/migrating_to_1.14` + + +Client Changes +-------------- + + +Fail fast option for builds +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/432 + + +When builders are configured with ``build_arch_can_fail = True`` then the +failure of a single buildArch task does not immediately cause the build +to fail. Instead, the remaining buildArch tasks are allowed to complete, +at which point the build will still fail. + +Sometimes developers would rather a build fail immediately, so we have added +the ``--fail-fast`` option to the build command, which overrides this setting. +The option only has an effect if the builders are configured to fail slow. + + +Custom Lorax templates +^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/419 + +Koji now supports custom Lorax templates for the ``spin-livemedia`` command. +The command accepts two new options: + +.. code-block:: text + + --lorax_url=URL The URL to the SCM containing any custom lorax + templates that are to be used to override the default + templates. + --lorax_dir=DIR The relative path to the lorax templates directory + within the checkout of "lorax_url". + + +The Lorax templates must come from an SCM, and the ``allowed_scms`` rules +apply. + +When these options are used, the templates will be fetched and an appropriate +``--lorax-templates`` option will be passed to the underlying livemedia-creator +command. + + +Allow profiles to request a specific python version +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/566 + +On platforms with python3 available, the Koji client is built to execute +with the python3 binary. However, there are a few client features that do not +work under python3, notably old-style (non-gssapi) Kerberos authentication. + +If this issue is affecting you, you can set ``pyver=2`` in your Koji +configuration. This can be done per profile. When Koji sees this setting +at startup, it will re-execute itself under the requested python binary. + + +New list-builds command +^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/526 + +The command line now has a ``list-builds`` command that has similar +functionality to the builds tab of the web interface. + +.. code-block:: text + + Usage: koji list-builds [options] + (Specify the --help global option for a list of other help options) + + Options: + -h, --help show this help message and exit + --package=PACKAGE List builds for this package + --buildid=BUILDID List specific build from ID or nvr + --before=BEFORE List builds built before this time + --after=AFTER List builds built after this time + --state=STATE List builds in this state + --type=TYPE List builds of this type. + --prefix=PREFIX Only list packages starting with this prefix + --owner=OWNER List builds built by this owner + --volume=VOLUME List builds by volume ID + -k FIELD, --sort-key=FIELD + Sort the list by the named field + -r, --reverse Print the list in reverse order + --quiet Do not print the header information + + +New block-group command +^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/509 + +The ``block-group`` command allows admins to block package group entries +without having to resort to the ``call`` command. + +.. code-block:: text + + Usage: koji block-group + (Specify the --help global option for a list of other help options) + + Options: + -h, --help show this help message and exit + + +Exit codes for some commands +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/558 +| PR: https://pagure.io/koji/pull-request/559 + +Several more commands will now return a non-zero exit code +when an error occurs: + + * the various image building commands + * the ``save-failed-tree`` command (provided by a plugin) + + +Easier for scripts to use activate_session +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/493 + +In Koji 1.13.0, it became possible for scripts to ``import koji_cli.lib`` and +gain access to the ``activate_session`` function that the command line tool +uses to authenticate. + +In this release, this function has been made easier for scripts to use: + + * the options argument can now be a dictionary + * less options need to be specified + + +Builder changes +--------------- + + +Normalize paths for scms +^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/591 + + +For many years, kojid has supported the ``allowed_scms`` option +(see: :ref:`scm-config`) for controlling which scms can be used for building. +In 1.13.0, Koji added the ability to explicitly block a host:path pattern. + +Unfortunately, 1.13.0 did not normalize the path before checking the pattern, +making it possible for users to use equivalent paths to route around the +block patterns. + +Now, Koji will normalize these paths before the ``allowed_scms`` check. + + +Graceful reload +^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/565 + + +For a long time kojid has handled the USR1 signal by initiating a graceful restart. +This change exposes that in the systemd service config (and the init script +on older platforms). + +Now, ``service kojid reload`` will trigger the same sort of restart that the +``restart-hosts`` command accomplishes, but only for the build host you run it +on. When this happens, kojid will: + + * stop taking new tasks + * wait for current tasks to finish + * restart itself once all its tasks are completed + + +Friendlier runroot configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/539 +| PR: https://pagure.io/koji/pull-request/528 + +Two changes make it easier to write a configuration for the runroot plugin. + +The ``path_subs`` option is now more forgiving about whitespace: + + * leading and trailing whitespace is ignored for each line + * blank lines are ignored + +The ``[pathNN]`` sections are no longer required to have sequential numbers. +Previously, the plugin expected a sequence like ``[path0]``, ``[path1]``, +``[path2]``, etc, and would stop looking for entries if the next number +was missing. Now, any set of distinct numbers is valid and all ``[pathNN]`` +sections will be processed. + + +System changes +-------------- + +Deprecations +^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/554 +| PR: https://pagure.io/koji/pull-request/597 + +The following features are deprecated and will be removed in a future release: + + * the ``importBuildInPlace`` rpc call + * the ``use_old_ssl`` client configuration option (and the underlying + ``koji.compatrequests`` library) + + +Removed calls +^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/497 +| PR: https://pagure.io/koji/pull-request/507 + +The deprecated ``buildFromCVS`` hub call has been removed. It was replaced +by the ``buildSRPMFromCVS`` call many years ago and has been deprecated since +version 1.6.0. + +The ``add_db_logger`` function has been removed from the koji library, along +with the ``log_messages`` table in the db. This extraneous call has never been +used in Koji. + + +Dropped mod_python support +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/508 + + +Koji no longer supports mod_python. This option has been deprecated since +mod_wsgi support was added in version 1.7.0. + +See also: :doc:`../migrations/migrating_to_1.7` + + +Large integer support +^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/571 + + +Koji uses xmlrpc for communications with the hub, and unfortunately the +baseline xmlrpc standard only supports 32-bit signed integers. This +results in errors when larger integers are encountered, typically +when a file is larger than 2 GiB. + +Starting with version 1.14.0, Koji will emit ``i8`` tags when encoding +large integers for xmlrpc. Integers below the limit are still encoded +with the standard ``int`` tag. The only time this makes a difference +is when Koji would previously have raised an ``OverflowError``. + +The ``i8`` tag comes from the +`ws-xmlrpc `__ +spec. Python's xmlrpc decoder has +for many years accepted and understood this tag, even though its encoder +would not emit it. + +Previous versions of Koji worked around such size issues by converting +large integers to strings in a few targeted places. Those targeted +workarounds have been left in place on the hub for the sake of backward +compatibility. + + +Test mode for protonmsg plugin +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/538 + +The ``protonmsg`` plugin now accepts a boolean ``test_mode`` configuration +option. When this option is enabled, the plugin will not actually +send messages, but will instead log them (at the DEBUG level). + +This option allows testing environments to run with the plugin enabled, but +without requiring a message bus to be set up for that environment. + + +Handling of debugsource rpms +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/524 + +Koji will now treat rpms ending in ``-debugsource`` the same way that it does +other debuginfo rpms. Such rpms are: + + * omitted from Koji's normal yum repos + * listed separately when displaying builds + * not downloaded by default in the ``download-build`` command + + +Added kojifile component type for content generators +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/506 + +Content generator imports now accept entries with type equal to ``kojifile`` +in the component lists for buildroots and images/archives. This type provides +a more reliable way to reference archive that come from Koji. + +See: :ref:`Example metadata `. diff --git a/docs/source/release_notes/release_notes_1.15.1.rst b/docs/source/release_notes/release_notes_1.15.1.rst new file mode 100644 index 0000000..1805df4 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.15.1.rst @@ -0,0 +1,34 @@ +Koji 1.15.1 Release Notes +========================= + +Koji 1.15.1 is a bugfix release for Koji 1.15. The most important change +is the fix for :doc:`../CVEs/CVE-2018-1002150`. + +Please see: :doc:`release_notes_1.15` + +Issues fixed in 1.15.1 +---------------------- + +- `Issue 850 `_ -- + CVE-2018-1002150 + +- `Issue 846 `_ -- + error occurs in SCM.get_source since subprocess.check_output is not supported by python 2.6- + +- `Issue 724 `_ -- + buildNotification of wrapperRPM fails because of task["label"] is None + +- `Issue 786 `_ -- + buildSRPMFromSCM tasks fail on koji 1.15 + +- `Issue 803 `_ -- + Email notifications makes build tasks fail with "KeyError: 'users_usertype'" + +- `Issue 742 `_ -- + dict key access fail in koji_cli.commands._build_image + +- `Issue 811 `_ -- + AttributeError: 'dict' object has no attribute 'hub.checked_md5' + +- `Issue 813 `_ -- + cg imports fail with "Unsupported checksum type" diff --git a/docs/source/release_notes/release_notes_1.15.rst b/docs/source/release_notes/release_notes_1.15.rst new file mode 100644 index 0000000..5debfb3 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.15.rst @@ -0,0 +1,277 @@ +Koji 1.15 Release Notes +======================= + +Updates +------- + +- :doc:`Koji 1.15.1 ` is a security update for Koji 1.15 + +Migrating from the previous release +----------------------------------- + +For details on migrating see :doc:`../migrations/migrating_to_1.15` + + +Client Changes +-------------- + + +Display license Info +^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/686 + + +The ``rpminfo`` command now displays the ``License`` field from the rpm. + + +Keytabs for GSSAPI authentication +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/708 + +Previously keytabs were only supported by the older kerberos auth method, which +is not available on Python 3. Now the gssapi method supports them as well. + + +Add krb_canon_host option +^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/653 + +This release adds a ``krb_canon_host`` option that tells Koji clients +to use the dns canonical hostname for kerberos auth. + +This option allows kerberos authentication to work in situations where +the hub is accessed via a cname, but the hub's credentials are under +its canonical hostname. + +If specified, this option takes precedence over the older +option named ``krb_rdns``. That option caused Koji clients to perform a +reverse name lookup for kerberos auth. + +When configuring kojiweb (in web.conf), the option is named ``KrbCanonHost``. + +Both options only affect the older kerberos authentication path, and not +gssapi. + + +Watch-task return code +^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/703 + +Previously, the ``watch-task`` command would return a non-zero exit status +if any subtask failed, even if this did not cause the parent task to fail. + +Now that we have cases where subtasks are optional, this no longer makes sense. +The exit code is now based solely on the results of +the top level tasks it is asked to watch. + + +New runroot options +^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/633 + +The ``runroot`` command now supports options similar to the various build commands. These new +options are: + + +.. code-block:: text + + --nowait Do not wait on task + --watch Watch task instead of printing runroot.log + --quiet Do not print the task information + + +New watch-logs options +^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/625 + +The ``watch-logs`` command now supports the following new options: + +.. code-block:: text + + --mine Watch logs for all your tasks + --follow Follow spawned child tasks + + +Web UI changes +-------------- + +Archive component display +^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/610 + +Previously, the web UI only displayed component lists for image builds. +However, new build types can also have component lists. + +Now the interface will display components for any archive that has them. + + +Display license Info +^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/686 + + +The ``rpminfo`` page now displays the ``License`` field from the rpm. + + +Show suid bit +^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/617 + +The web UI will now display the setuid bit when displaying rpm/archive file contents. + + + + +Builder changes +--------------- + + +Alternate tmpdir for mock chroots +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/602 + + +Recent versions of mock (1.4+) default to ``use_nspawn=True``, which results +in /tmp being a fresh tmpfs mount on every run. This means the /tmp +directory no longer persists outside of the mock invocation. + +Now, the builder will use /builddir/tmp instead of /tmp for persistent data. + + +Store git commit hash +^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/674 + +In Koji, for builds from an SCM, the source is specified as an +scm url. +For git urls, the revision in that url can be anything that git +will recognize, including: + + - a sha1 ref + - an abbreviated sha1 ref + - a branch name + - a tag + - HEAD + +With this change: + + * the revision is replaced with the full sha1 ref for git urls + * the scm url is stored in build.source + * the original scm url is saved in build.extra + +Previously, this source url was not properly stored for rpm builds. It +appeared in the task parameters, but the build.source field remained blank. +If a symbolic git ref (e.g. HEAD) was given in the url, the underlying +sha1 value was only recorded in the task logs. + + + +System changes +-------------- + + +Volume policy support +^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/622 + +Koji has for many years had the ability to split its storage across multiple +volumes. However, there is no automatic process for placing builds onto +volumes other than the primary. To do so often requires a lot of manual work +from an admin. + +This feature: + + * adds a volume policy check to the key import pathways + * adds an applyVolumePolicy call to apply the policy to existing builds + +The hub consults the volume policy at various points to +determine where a build should live. This allows admins to make rules like: + + - all kernel builds go to the volume named kstore + - all builds built from the epel-7-build tag go to the volume named epel7 + - all builds from the osbs content generator go to the volume named osbs + +The default policy places all builds on the default volume. + +See also: :doc:`../volumes` + +Messagebus plugin changes +^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/537 + +There are two notable changes to the messagebus plugin this release: + + +Deferred sending +"""""""""""""""" + +Similar to the current behavior of the protonmsg plugin, messages are queued +up during hub calls and only sent out during the ``postCommit`` callback. + +This avoids sending messages about failed calls, which can be confusing to +message consumers (e.g. build state change messages about a build that does +not exist because it failed to import). + +Test mode +""""""""" + +The plugin now looks for a boolean ``test_mode`` option. If it is true, then +the messages are still queued up, but not actually sent. This makes it +possible to enable the plugin in test environments without having to set up a +separate message bus. + + +Protonmsg plugin changes +^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/657 +| PR: https://pagure.io/koji/pull-request/651 + +There are two changes to how the protonmsg plugin handles rpmsign events: + + 1. The arch of the rpm is included in messages + 2. The message are omitted when the sigkey is empty + + + +No notifications for disabled users or hosts +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/615 + + +Koji will no longer send out email notifications to disabled users or +to users corresponding to a host. + + +Replace pycurl with requests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/601 + +All uses of the pycurl library have been replaced with calls +to python-requests, so pycurl is no longer required. + + +Drop importBuildInPlace call +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +| PR: https://pagure.io/koji/pull-request/606 + +The deprecated ``importBuildInPlace`` call has been dropped. + +This call was an artifact of a particular bootstrap event that happened a long +time ago. It was never really documented or recommended for use. diff --git a/docs/source/release_notes/release_notes_1.16.1.rst b/docs/source/release_notes/release_notes_1.16.1.rst new file mode 100644 index 0000000..eb9f42b --- /dev/null +++ b/docs/source/release_notes/release_notes_1.16.1.rst @@ -0,0 +1,58 @@ +Koji 1.16.1 Release Notes +========================= + +Koji 1.16.1 is a point release for Koji 1.16. The major changes include: + +- Allow target info to be read for different type tasks in channel policy. +- Create symlinks for builds imported onto non-default volumes. +- Fix RPMdiff issues found in Koji 1.16.0. + +Please see: :doc:`release_notes_1.16` + +Issues fixed in 1.16.1 +---------------------- + +- `Issue 847 `_ -- + spin-livecd failed with "Could not resolve host" + +- `Issue 932 `_ -- + Fix use_host_resolv with new mock version + +- `Issue 1010 `_ -- + koji fails runroot because of `UnicodeDecodeError` + +- `Issue 998 `_ -- + cancel build doesn't work for images + +- `Issue 994 `_ -- + rpmdiff calculate wrong results + +- `Issue 1025 `_ -- + missing default volume symlink for imported builds affected by volume policy + +- `Issue 1007 `_ -- + decode_args() might result in --package parameter missing in runroot command + +- `Issue 150 `_ -- + no target info in channel policy for non-rpm tasks + +- `PR: 973 `_ -- + Check empty arches before spawning dist-repo + +- `Issue 958 `_ -- + Notification for tagBuildBypass is writing message untagged from, expected message tagged into + +- `Issue 968 `_ -- + Default enable python3 on RHEL8 + +- `Issue 916 `_ -- + `clone-tag` doesn't preserve tagging order + +- `Issue 949 `_ -- + cli: [rpminfo] KeyError: 'license' for external RPM + +- `Issue 876 `_ -- + koji clone-tag raises "UnboundLocalError" + +- `Issue 945 `_ -- + Koji build fail due to ambiguous python shebang diff --git a/docs/source/release_notes/release_notes_1.16.2.rst b/docs/source/release_notes/release_notes_1.16.2.rst new file mode 100644 index 0000000..8d0c6b3 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.16.2.rst @@ -0,0 +1,18 @@ +Koji 1.16.2 Release Notes +========================= + +Koji 1.16.2 is a bugfix release for Koji 1.16. +The purpose of this release is address :doc:`../CVEs/CVE-2018-1002161`. + +See also: + +- :doc:`release_notes_1.16.1` + +- :doc:`release_notes_1.16` + + +Issues fixed in 1.16.2 +---------------------- + +- `Issue 1183 `_ -- + CVE-2018-1002161 diff --git a/docs/source/release_notes/release_notes_1.16.rst b/docs/source/release_notes/release_notes_1.16.rst new file mode 100644 index 0000000..e2db029 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.16.rst @@ -0,0 +1,249 @@ +Koji 1.16.0 Release notes +========================= + + +Migrating from Koji 1.15 +------------------------ + +For details on migrating see :doc:`../migrations/migrating_to_1.16` + + + +Security Fixes +-------------- + +**CVE-2018-1002150 - distRepoMove missing access check** + +This release includes the fix for :doc:`../CVEs/CVE-2018-1002150`. + + +Client Changes +-------------- + +**CLI commands to manage notifications** + +| PR: https://pagure.io/koji/pull-request/688 + +The change adds new cli sub-commands: + + - list-notifications + - add-notification + - remove-notification + - edit-notification + +Previously this functionality was only available through the web ui or +by making direct api calls. + + +**Add --old-chroot option to runroot command** + +| PR: https://pagure.io/koji/pull-request/823 + +This option causes the runroot handler to pass the same-named option +to the mock command. This complements the existing ``--new-chroot`` +option. + +If neither ``--old-chroot`` or ``--new-chroot`` is given, then mock will +follow its default behavior. This default varies across mock versions. +For newer versions of mock, ``--new-chroot`` is the default (uses a +systemd nspawn container). + + +**Fix runroot output on py3** + +| PR: https://pagure.io/koji/pull-request/828 + +The runroot command should now work under python3. + + +**Honor runroot --quiet** + +| PR: https://pagure.io/koji/pull-request/806 + +The ``--quiet`` option was added to the runroot command in version 1.15, +but it only took effect when the ``--watch`` option was given. Now it is +honored in all cases. + + +**Drop old ssl code** + +| PR: https://pagure.io/koji/pull-request/498 + +The old ``koji.ssl`` module has been removed, and the ``use_old_ssl`` option +has been removed from client code. + +Because these files (which were originally from +`Plague `_) were the only parts +of Koji that were licensed as GPLv2+, Koji is now simply licensed as +LGPLv2. + + +Builder Changes +--------------- + +**Configure install timeout for imagefactory** + +| PR: https://pagure.io/koji/pull-request/841 + +Previously the install timeout parameter for imagefactory was set +to a fixed value of 7200 by Koji. Now it can be controlled by +setting the ``oz_install_timeout`` option in ``kojid.conf``. + +A value of ``0`` will disable the timeout. + + +**Record log timestamps** + +| PR: https://pagure.io/koji/pull-request/777 + +If the ``log_timestamps`` option is enabled in ``kojid.conf``, then +the builder will record a separate timestamp file for each log file +in a build. + +The filename for the timestamp file is generated by taking the name +of the log file and appending ``-ts.log``. So ``build.log`` will have +timestamp data in ``build.log-ts.log``. + +The format of the timestamp log is plain text with each line showing +a numeric timestamp and a line offset. + + +**Builder option: chroot_tmpdir** + +| PR: https://pagure.io/koji/pull-request/787 + +The new ``chroot_tmpdir`` option controls which directory within buildroots +is used for various temporary data by the Koji builder daemon. +Previously this was hardcoded to ``/builddir/tmp``, which created problems +with modern versions of mock. + +The default value is ``/chroot_tmpdir``. + + +**Add internal_dev_setup option to runroot config** + +| PR: https://pagure.io/koji/pull-request/824 + +The ``internal_dev_setup`` config option for the runroot builder plugin +controls whether the mock option of the same name is set for runroot +tasks. + + + +System Changes +-------------- + + +**Add option to configure DB port** + +| PR: https://pagure.io/koji/pull-request/884 + +The hub now accepts a ``DBPort`` option in ``hub.conf``, which specifies +which port the hub should use when connecting to the database. + + +**Split debuginfo for dist repos** + +| PR: https://pagure.io/koji/pull-request/914 + +Dist repos can now be generated with debuginfo files split into a separate +repo. The behavior is controlled by passing the ``--split-debuginfo`` option +to the ``dist-repo`` subcommand. + +When this option is in effect, the main repo will be in the normal location. +The debuginfo repo will be in the ``debug`` subdirectory. So, you will +see a directory structure like: + +.. code-block:: text + + + Packages/ + repodata/ + debug/ + debug/repodata + +Regardless of the split, all the rpms are located in the top level +``Packages`` directory. + + +**Notifications in [un]tagBuildBypass** + +| PR: https://pagure.io/koji/pull-request/691 + +Previously the ``tagBuildBypass`` and ``untagBuildBypass`` calls did not trigger +notifications. Now they will do so by default. The call now accepts a +``notify`` option (defaults to True) which controls the behavior. + + +**Track history for host data** + +| PR: https://pagure.io/koji/pull-request/778 + +Koji now tracks changes to host data similarly to the way it tracks +changes for other data. This includes + + - enabled state + - arches + - capacity + - description & comment + - channels + +The ``list-history`` cli command now supports ``--host`` and ``--channel`` +options to select history entries for a host or channel. + +The versioned host data is stored in the ``host_config`` and ``host_channels`` +tables. + + +**Fix block-group functionality** + +| PR: https://pagure.io/koji/pull-request/678 + +The ``block-group`` command and its underlying api call now actually work. + + +**Strict option for archive listing calls** + +| PR: https://pagure.io/koji/pull-request/734 +| PR: https://pagure.io/koji/pull-request/748 + +The ``list_archives``, ``get_archive_file()``, and ``list_archive_files()`` +hub functions now accept a strict option, which defaults to False. When +the option is True, the call will raise an exception if there is no +match. + + +**Search build by source** + +| PR: https://pagure.io/koji/pull-request/765 + +The ``listBuilds()`` api call now supports a source option. This is +treated as a glob pattern and matched against the ``source`` field of the build. + + +**Option to ignore tags in kojira** + +| PR: https://pagure.io/koji/pull-request/695 + +Kojira now supports an ``ignore_tags`` option. This is treated as a +space-separated list of glob patterns. Tags that match are ignored +by kojira (it will not generate newRepo tasks for them). + + +**Improve kojira throughput** + +| PR: https://pagure.io/koji/pull-request/797 + +Kojira should be much more responsive in triggering ``newRepo`` tasks. + + +**Drop migrateImage call** + +| PR: https://pagure.io/koji/pull-request/632 + +The ``migrateImage`` call hub call has been removed. + +This call was added in version 1.8 (April 2013) +as a one-time tool for migrating images from the old model (no build entry) +to the new model (image build type). It was only available if the +EnableImageMigration option was set on the hub. diff --git a/docs/source/release_notes/release_notes_1.17.rst b/docs/source/release_notes/release_notes_1.17.rst new file mode 100644 index 0000000..348b5e2 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.17.rst @@ -0,0 +1,242 @@ +Koji 1.17.0 Release notes +========================= + + +Migrating from Koji 1.16 +------------------------ + +For details on migrating see :doc:`../migrations/migrating_to_1.17` + + + +Security Fixes +-------------- + +**CVE-2018-1002161 - SQL injection in multiple remote calls** + +| PR: https://pagure.io/koji/pull-request/1274 + +This release includes the fix for :doc:`../CVEs/CVE-2018-1002161` + + +Client Changes +-------------- + +**Volume id option for livemedia and livecd tasks** + +| PR: https://pagure.io/koji/pull-request/1227 + +The ``spin-livecd`` and ``spin-livemedia`` commands now accept a ``--volid`` +argument to specify the volume id for the media. If unspecified, the +volume id is chosen via the same heuristic as before. + +Volume ids must be 32 characters or less. + + + +**Build order preserved by clone-tag** + +| PR: https://pagure.io/koji/pull-request/1014 + +This is an improvement to the ``clone-tag`` command. Previously, when the +command was used without the ``--latest-only`` option, it could get the +ordering of builds wrong in the destination tag. Now, the order will +match the source tag. + + + +**Configurable authentication timeout** + +| PR: https://pagure.io/koji/pull-request/1172 + +Previously, the network timeout during authentication was hard coded to +60 seconds. It is now configurable via the ``auth_timeout`` configuration +option. + + +**Additional information from list-channels command** + +| PR: https://pagure.io/koji/pull-request/940 + +The ``list-channels`` command now shows three separate host counts for +each channel: + +- the number of enabled hosts in the channel +- the number of ready hosts in the channel +- the number of disabled hosts in the channel + + +**The free-task command requires at least one task-id** + +| PR: https://pagure.io/koji/pull-request/1045 + +Previously this command was a no-op when given no arguments. Now it will return an +error. + + + +Library Changes +--------------- + +**Drop encode_int function** + +| PR: https://pagure.io/koji/pull-request/852 + +This is a follow up to the large integer support that we added in version 1.14 + +See also: :doc:`release_notes_1.14` + +The ``encode_int`` function is no longer used +and has been dropped from the library. + +Because we no longer call ``encode_int``, the hub will now always use i8 tags +when returning large integers, rather than returning them as strings in some +cases. + + +**Use custom Kerberos context with krb_login** + +| PR: https://pagure.io/koji/pull-request/1187 + +Clients can now pass in their own Kerberos context to +``ClientSession.krb_login()`` using +the ``ctx`` parameter. This is intended for multi-threaded clients. + + +**Custom keyboard interrupt handling in watch_tasks** + +| PR: https://pagure.io/koji/pull-request/981 + +The new ``ki_handler`` option for the ``koji_cli.lib.watch_tasks()`` function +allows other cli tools to set their own handler for keyboard interrupts. +If specified, the value should be callable and will be called when a +keyboard interrupt is encountered. +If unspecified, the original behavior is retained. + + +**_unique_path() -> unique_path** + +| PR: https://pagure.io/koji/pull-request/980 + +The ``_unique_path`` function is deprecated. It has been replaced +by ``unique_path``. + + +Web UI Changes +-------------- + +**Additional info on builders in channelinfo page** + +| PR: https://pagure.io/koji/pull-request/989 + +The channelinfo page now shows enabled/ready status for each host and a count +for each. + + + +Builder Changes +--------------- + +**Builder task_avail_delay check** + +| PR: https://pagure.io/koji/pull-request/1176 + +This delay works around a deficiency in task scheduling. The default +delay is 300 seconds and can be adjusted with the ``task_avail_delay`` +option to kojid. However, it is unlikely that admins will need to +adjust this setting. + +Despite the name, this does not introduce any new delay compared to the +old behavior. The setting controls how long a host will wait before taking +a task in a given channel-arch "bin" when that host has an available +capacity lower than the median for that bin. Previously, such hosts +could wait forever. + + + +System Changes +-------------- + + +**Python 3 Support** + +| PR: https://pagure.io/koji/pull-request/1117 +| PR: https://pagure.io/koji/pull-request/891 +| PR: https://pagure.io/koji/pull-request/921 +| PR: https://pagure.io/koji/pull-request/1184 +| PR: https://pagure.io/koji/pull-request/1019 +| PR: https://pagure.io/koji/pull-request/685 +| ...and many fixes + +Support for Python 3 has been extended to all components of Koji. Including: + +- Hub +- Builder +- Web UI +- Utils + + + +**No more messagebus plugin** + +| PR: https://pagure.io/koji/pull-request/1043 + +The messagebus plugin has been dropped. The protonmsg plugin is still +available. + + + +**Simple mode for mergerepos** + +| PR: https://pagure.io/koji/pull-request/1066 + +External repos now have a ``merge_mode`` option. Valid values are +either ``koji`` (the old way) or ``simple`` (a new alternative). This +option can be set with the ``--mode`` option to the ``add-external-repo`` +or ``edit-external-repo`` commands. + +When an external repo is merged with simple mode, a number of the complex +filters that Koji normally applies are skipped. This mode still honors +the block list from Koji and ignores duplicate NVRAs, but otherwise +it simply merges the repo in. + +Multiple merge modes cannot be combined in a single tag. If a tag +has two external repos with different modes, then the repo will +fail to generate. + + +**Avoid "unknown task" errors in Kojira** + +| PR: https://pagure.io/koji/pull-request/1175 + +This is a bug fix for a minor race condition in Kojira that could cause +errors in the log and redundant repo regens. + + + +**Full filename display for kojifiles directory indexes** + +| PR: https://pagure.io/koji/pull-request/1156 + +This is simply a change to the default httpd configuration for serving +/mnt/koji. It adds ``NameWidth=*`` to ``IndexOptions`` so that long filenames +are fully displayed. + + + +**Broader support for target/source/scratch tests in channel policy** + +| PR: https://pagure.io/koji/pull-request/962 + +It is now possible to write channel policy rules based on +build target, source, and scratch options for task types other +than ``build``. + + + +**Longer Build Target names** + +| PR: https://pagure.io/koji/pull-request/925 + +Build target names can now be up to 256 characters, the same length +restriction as for tag names. diff --git a/docs/source/release_notes/release_notes_1.18.1.rst b/docs/source/release_notes/release_notes_1.18.1.rst new file mode 100644 index 0000000..7f310c3 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.18.1.rst @@ -0,0 +1,12 @@ +Koji 1.18.1 Release Notes +========================= + +Koji 1.18.1 is a bugfix release for Koji 1.18. +The purpose of this release is address :doc:`../CVEs/CVE-2019-17109`. + + +Issues fixed in 1.18.1 +---------------------- + +- `Issue 1634 `_ -- + possible to upload file to a path other than work directory diff --git a/docs/source/release_notes/release_notes_1.18.rst b/docs/source/release_notes/release_notes_1.18.rst new file mode 100644 index 0000000..32da4c3 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.18.rst @@ -0,0 +1,378 @@ +Koji 1.18.0 Release notes +========================= + + +Migrating from Koji 1.17 +------------------------ + +For details on migrating see :doc:`../migrations/migrating_to_1.18` + + + +Security Fixes +-------------- + + + +Client Changes +-------------- + +**Add option for custom cert location** + +| PR: https://pagure.io/koji/pull-request/1253 + +The CLI now has an option for setting a custom SSL certificate, similar to the +options for Kerberos authentication. + + +**Load client plugins from ~/.koji/plugins** + +| PR: https://pagure.io/koji/pull-request/892 + + +This change allows users to load their own cli plugins from ``~/.koji/plugins`` +or from another location by using the ``plugin_paths`` setting. + + +**Show load/capacity in list-channels** + +| PR: https://pagure.io/koji/pull-request/1449 + +The ``list-channels`` display has been expanded to show overall totals for load +and capacity. + + +**Allow taginfo cli to use tag IDs** + +| PR: https://pagure.io/koji/pull-request/1476 + +The ``taginfo`` command can now accept a numeric tag id on the command line. + + +**Add option to show channels in list-hosts** + +| PR: https://pagure.io/koji/pull-request/1425 + +The ``list-hosts`` command will now display channel subscriptions if the +``--show-channels`` option is given. + + +**Remove merge option from edit-external-repo** + +| PR: https://pagure.io/koji/pull-request/1499 + +This option was mistakenly added to the command and never did anything. +It is gone now. + + +**Honor mock.package_manager tag setting in mock-config cli** + +| PR: https://pagure.io/koji/pull-request/1374 + +The ``mock-config`` command will now honor this setting just as ``kojid`` does. + + + + +Library Changes +--------------- + +**New multicall interface** + +| PR: https://pagure.io/koji/pull-request/957 + +This feature implements a new and much better way to use multicall in the Koji +library. +These changes create a new implementation outside of ClientSession. +The old way will still work. + +With this new implementation: + +* a multicall is tracked as an instance of `MultiCallSession` +* the original session is unaffected +* multiple multicalls can be managed in parallel, if desired +* `MultiCallSession` behaves more or less like a session in multicall mode +* method calls return a `VirtualCall` instance that can later be used to access the result +* `MultiCallSession` can be used as a context manager, ensuring that the calls are executed + +Usage examples can be found in the :doc:`Writing Koji Code <../writing_koji_code>` +document. + + + + +Web UI Changes +-------------- + +**Retain old search pattern in web ui** + +| PR: https://pagure.io/koji/pull-request/1258 + +The search results page of the web ui now retains a search form with the +current search pre-filled. +This makes it easier for users to refine their searches. + + +**Display task durations in webui** + +| PR: https://pagure.io/koji/pull-request/1383 + + +The ``taskinfo`` page in the web ui now shows task durations in addition to +timestamps. + + + +Builder Changes +--------------- + +**Rebuild SRPMS before building** + +| PR: https://pagure.io/koji/pull-request/1462 + +For rpm builds from an uploaded srpm, Koji will now rebuild the srpm in the +build environment first. +This ensures that the NVR is correct for the resulting build. + +The old behavior can be requested by setting ``rebuild_srpm=False`` in the tag +extra data for the build tag in question. + + +**User createrepo_c by default** + +| PR: https://pagure.io/koji/pull-request/1278 + + +The ``use_createrepo_c`` configuration option for ``kojid`` now defaults to True. + + +**Use createrepo update option even for first repo run** + +| PR: https://pagure.io/koji/pull-request/1363 + +If there is no older repo for a tag, Koji will now attempt to find +a related repo to use ``createrepo --update`` with. +This will speed up first-time repo generations for tags that +predominantly inherit their content from another build tag. + + +**Scale task_avail_delay based on bin rank** + +| PR: https://pagure.io/koji/pull-request/1386 + +This is an adjustment to Koji's decentralized scheduling algorithm. +It should result in better utilization of host capacity, particularly when +a channel has hosts that are very heterogeneous in capacity. + +The meaning of the ``task_avail_delay`` setting is different now. +Within a channel-arch bin, the hosts with highest capacity will take the task +immediately, while hosts lower down will have a delay proportional to their +rank. +The "rank" here is a float between 0.0 and 1.0 used as a multiplier. +So ``task_avail_delay`` is the maximum time that any host will wait to +take a task. + +Hosts with higher available capacity will be more likely to claim a +task, resulting in better utilization of the highest capacity hosts. + + +**Use RawConfigParser for kojid** + +| PR: https://pagure.io/koji/pull-request/1544 + +The use of percent signs is common in ``kojid.conf`` because of the +``host_principal_format`` setting. +This causes an error in python3 if ``SafeConfigParser`` is used, so we use +``RawConfigParser`` instead. + + +**Handle bare merge mode** + +| PR: https://pagure.io/koji/pull-request/1411 +| PR: https://pagure.io/koji/pull-request/1516 +| PR: https://pagure.io/koji/pull-request/1502 + + +This feature adds a new merge mode for external repos named ``bare``. +This mode is intended for use with modularity. + +Use of this mode requires createrepo_c version 0.14.0 or later on the builders +that handle the createrepo tasks. + + + + +System Changes +-------------- + + +**API for reserving NVRs for content generators** + +| PR: https://pagure.io/koji/pull-request/1464 +| PR: https://pagure.io/koji/pull-request/1597 +| PR: https://pagure.io/koji/pull-request/1601 +| PR: https://pagure.io/koji/pull-request/1602 +| PR: https://pagure.io/koji/pull-request/1606 + +This feature allows content generators to reserve NVRs earlier in the build +process similar to builds performed by ``kojid``. The NVR is reserved by +calling ``CGInitBuild()`` and finalized by the ``CGImport()`` call. + + + +**Per-tag configuration of rpm macros** + +| PR: https://pagure.io/koji/pull-request/898 + +This feature allows setting rpm macros via the tag extra field. These macros +will be added to the mock configuration for the buildroot. The system +looks for extra values of the form ``rpm.macro.NAME``. + +For example, to set the dist tag for a given tag, you could use a command like: + +:: + + $ koji edit-tag f30-build -x rpm.macro.dist=MYDISTTAG + + + +**Per-tag configuration for module_hotfixes setting** + +| PR: https://pagure.io/koji/pull-request/1524 +| PR: https://pagure.io/koji/pull-request/1578 + +Koji now handles the field ``mock.yum.module_hotfixes`` in the tag extra. +When set, kojid will set ``module_hotfixes=0/1`` in the yum portion of the +mock configuration for a buildroot. + + +**Allow users to opt out of notifications** + +| PR: https://pagure.io/koji/pull-request/1417 +| PR: https://pagure.io/koji/pull-request/1580 + +This feature lets users opt out of notifications that they would otherwise +automatically recieve, such as build and tag notifications for: + +- the build owner (the user who submitted the build) +- the package owner within the given tag + +These opt-outs are user controlled and can be managed with the new +``block-notification`` and ``unblock-notificiation`` commands. + + +**Allow hub policy to match version and release** + +| PR: https://pagure.io/koji/pull-request/1513 + + +This feature adds new policy tests to match ``version`` and ``release``. +This tests are glob pattern matches. + + +**Allow hub policy to match build type** + +| PR: https://pagure.io/koji/pull-request/1415 + + +Koji added btypes in version 1.11 along with content generators. +Now, all builds have one or more btypes. + +This change allows policies to check the btype value using the ``buildtype`` test. + + + +**More granular admin permissions** + +| PR: https://pagure.io/koji/pull-request/1454 + +A number of actions that were previously admin-only are now governed by +separate permissions: + + ``host`` + This permission governs most host management operations, such as + adding, editing, enabling/disabling, and restarting. + + ``tag`` + This permission governs adding, editing, and deleting tags. + + ``target`` + This permission governs adding, editing, and deleting targets. + +Koji administrators may want to consider reducing the number of users with +full ``admin`` permission. + + +**Option to generate separate source repo** + +| PR: https://pagure.io/koji/pull-request/1273 + +The (non-dist) yum repos that Koji generates for building normally don't +include srpms. +An old option allowed them to be included in some cases, but they were simply +added to each repo. +Newer options have been added that instruct Koji to include them as a separate +src repo. + +In the cli, the ``regen-repo`` command now accepts a ``--separate-source`` +option that triggers this behavior. + +In ``kojira``, the ``separate_source_tags`` option is a list of tag patterns. +Build tags that match any of these patterns will have their repos generated +with a separate src repo. + + + +**Add volume option for dist-repo** + +| PR: https://pagure.io/koji/pull-request/1327 + +Dist repos can now be generated on volumes other than the main one. +Use the ``--volume`` option to the ``dist-repo`` command to do so. + +Generally you want the repo to be on the same volume as the rpms it will +contain. +Dist repos hard link (same volume) or copy (different volume) their rpms into +place. +Using the appropriate volume can drastically improve the efficiency, both in +generation time and space consumption. + + +**Minor gc optimizations** + +| PR: https://pagure.io/koji/pull-request/1337 +| PR: https://pagure.io/koji/pull-request/1442 +| PR: https://pagure.io/koji/pull-request/1437 + +This change speeds up portions of garbage collection by making the +``build_references`` check lazy by default. + + + +**Rollback errors in multiCall** + +| PR: https://pagure.io/koji/pull-request/1358 + +If one of the calls in a multicall raises an error, then the transaction will +be rolled back to the start of that call before Koji proceeds to the next call. +This matches the behavior of normal calls more closely. + +Multicalls are still handled within single database transaction. + + + +**Support tilde in search** + +| PR: https://pagure.io/koji/pull-request/1297 + + +The tilde character is no longer prohibited in search terms. + + + +**Remove 'keepalive' option** + +| PR: https://pagure.io/koji/pull-request/1277 + +The ``keepalive`` setting is no longer used anywhere in koji. +It has been removed. diff --git a/docs/source/release_notes/release_notes_1.19.rst b/docs/source/release_notes/release_notes_1.19.rst new file mode 100644 index 0000000..84a8d3a --- /dev/null +++ b/docs/source/release_notes/release_notes_1.19.rst @@ -0,0 +1,416 @@ +Koji 1.19.0 Release notes +========================= + + +Migrating from Koji 1.18 +------------------------ + +For details on migrating see :doc:`../migrations/migrating_to_1.19` + + + +Security Fixes +-------------- + +**GSSAPI authentication checks kerberos principal** + +| PR: https://pagure.io/koji/pull-request/1419 + +When using GSSAPI authentication the user's kerberos principal will be checked +for their username to avoid a potential username and kerberos principal mismatch. + + + +Client Changes +-------------- + +**Add user edit** + +| PR: https://pagure.io/koji/pull-request/902 +| PR: https://pagure.io/koji/pull-request/1701 +| PR: https://pagure.io/koji/pull-request/1713 + +A new ``edit-user`` command and API call was added, allowing for user rename, +and changing, adding, or removing the kerberos principal of a user. + + +**Add remove group** + +| PR: https://pagure.io/koji/pull-request/923 + +A new ``remove-group`` command was added, allowing the removal of a group +from a tag. It uses the existing ``groupListRemove`` API call. + + +**Query builds per chunks in prune-signed-builds** + +| PR: https://pagure.io/koji/pull-request/1589 + +For bigger installations querying all builds can cause the hub to run out of memory. +``prune-signed-builds`` now queries these in 50k chunks. + + +**Show inheritance flags in list-tag-inheritance output** + +| PR: https://pagure.io/koji/pull-request/1120 + +While not often used, tag inheritance can be modified with a few different options (e.g. maxdepth). +These options are shown in the ``taginfo`` display, but not the ``list-tag-inheritance`` display. +This change adds basic indicators to the latter. + + +**Return usage information in make-task** + +| PR: https://pagure.io/koji/pull-request/1157 + +``make-task`` now returns usage information if no arguments are provided. + + +**Clarify clone-tag usage** + +| PR: https://pagure.io/koji/pull-request/1623 + +The ``clone-tag`` help text now clarifies that the destination tag will be created +if it does not already exist. + + +**Add option check for list-signed** + +| PR: https://pagure.io/koji/pull-request/1631 + +The ``list-signed`` command will now fail if no options are provided. + + + +Library Changes +--------------- + +**Consolidate config reading style** + +| PR: https://pagure.io/koji/pull-request/1296 + +Changes have been made to make configuration handling more consistent. + +With this new implementation: + +* ``read_config_files`` is extended with a strict option and directory support +* ``ConfigParser`` is used for all invokings except kojixmlrpc and ``kojid`` +* ``RawConfigParser`` is used for ``kojid`` + + +**list_archive_files handles multi-type builds** + +| PR: https://pagure.io/koji/pull-request/1508 + +If ``list_archive_files`` is provided a build with multiple archive types it now correctly +handles them instead of failing. + + +**Disallow archive imports that don't match build type** + +| PR: https://pagure.io/koji/pull-request/1627 +| PR: https://pagure.io/koji/pull-request/1633 + +The ``importArchive`` call now refuses to proceed if the build does not have the given type. + + +**Add listCG RPC** + +| PR: https://pagure.io/koji/pull-request/1160 + +``listCGs`` has been added to list new content generator records. + +The purpose of this change is to make it easier for administrators to determine what +content generators are present and what user accounts have access to those. + + +**Add method to cancel CG reservations** + +| PR: https://pagure.io/koji/pull-request/1662 + +The new ``CGRefundBuild`` call allows CGs to cancel build reservations, such as in the case +of a failing build. + + +**Allow ClientSession objects to get cleaned up by the garbage collector** + +| PR: https://pagure.io/koji/pull-request/1653 + +This change ensures ``koji.ClientSession`` objects are destroyed once their requests are complete. + + +**Add missing package list check** + +| PR: https://pagure.io/koji/pull-request/1244 +| PR: https://pagure.io/koji/pull-request/1702 + +The ``host.tagBuild`` method was missing a check to ensure the package was actually listed in the +destination tag. This should now be checked as expected. + + +**Increase buildReferences SQL performance** + +| PR: https://pagure.io/koji/pull-request/1675 + +The performance for ``build_references`` has been improved. + + +**ensuredir does not duplicate directories** + +| PR: https://pagure.io/koji/pull-request/1197 + +``koji.ensuredir`` no longer creates duplicate directories if provided a path ending in a +forward slash. + + +**Warn users if buildroot uses yum instead of dnf** + +| PR: https://pagure.io/koji/pull-request/1595 + +This change sets the mock config ``dnf_warning`` to True for buildroots using yum. + + +**Tag permission can be used for tagBuildBypass and untagBuildBypass** + +| PR: https://pagure.io/koji/pull-request/1685 + +The ``tag`` permission can now be used in place of admin to call ``tagBuildBypass`` +and ``untagBuildBypass``. Admin is still required to use the ``--force`` option. + + +**Rework update of reserved builds** + +| PR: https://pagure.io/koji/pull-request/1621 + +This change reworks and simplifies the code that updates reserved build entries for cg imports. +It removes redundancy with checks in ``prep_build`` and avoids duplicate ``*BuildStateChange`` +callbacks. + + +**Use correct top limit for randint** + +| PR: https://pagure.io/koji/pull-request/1612 + +The top limit for ``randint`` has been set to 255 from 256 to prevent ``generate_token`` from +creating unneccesarily long tokens. + + +**Add strict option to getRPMFile** + +| PR: https://pagure.io/koji/pull-request/1068 + +``getRPMFile`` now has a ``strict`` option, failing when the RPM or filename does not exist. + + +**Stricter groupListRemove** + +| PR: https://pagure.io/koji/pull-request/1173 +| PR: https://pagure.io/koji/pull-request/1678 + +``groupListRemove`` now returns an error if the provided group does not exist for the tag. + + +**Clarified docs for build.extra.source** + +| PR: https://pagure.io/koji/pull-request/1677 + +The usage for ``build.extra.source`` has now been clarified in the ``getBuild`` call. + + +**Use bytes for debug string** + +| PR: https://pagure.io/koji/pull-request/1657 + +This change fixes debug output for Python 3. + + +**Removed host.repoAddRPM call** + +| PR: https://pagure.io/koji/pull-request/1680 + +The ``host.repoAddRPM`` call has been removed because it was unused and broken. + + + +Web UI Changes +-------------- + +**Made difference between Builds and Tags sections more clear** + +| PR: https://pagure.io/koji/pull-request/1676 + +The search page results for packages now has a clearer delineation between builds and tags. + + + +Builder Changes +--------------- + +**Use preferred arch when builder provides multiple** + +| PR: https://pagure.io/koji/pull-request/1684 + +When using ExclusiveArch for noarch builds the build task will now use the +arch specified instead of randomly picking from the arches the builder provides. + +This change adds a ``preferred_arch`` parameter to ``find_arch``. + + +**Log insufficient disk space location** + +| PR: https://pagure.io/koji/pull-request/1523 + +When ``kojid`` fails due to insufficient disk space, the directory which needs more +disk space is now included as part of the log message. + + +**Allow builder to attempt krb if gssapi is available** + +| PR: https://pagure.io/koji/pull-request/1613 + +``kojid`` will now use ``requests_kebreros`` for kerberos authentication when available. + + +**Add support for new mock exit codes** + +| PR: https://pagure.io/koji/pull-request/1682 + +``kojid`` now expects mock exit code 10 for failed builds (previously 1). + + +**Fix kickstart uploads for Python 3** + +| PR: https://pagure.io/koji/pull-request/1618 + +This change fixes the file handling of kickstarts for Python 3. + + + +System Changes +-------------- + +**Package ownership changes do not trigger repo regens** + +| PR: https://pagure.io/koji/pull-request/1473 +| PR: https://pagure.io/koji/pull-request/1643 + +Changing tag or package owners no longer cause repo regeneration. A new +``tag_package_owners`` table has been added for this purpose. + + +**Support multiple realms by kerberos auth** + +| PR: https://pagure.io/koji/pull-request/1648 +| PR: https://pagure.io/koji/pull-request/1696 +| PR: https://pagure.io/koji/pull-request/1701 + +This change adds a new table ``user_krb_principals`` which tracks a list of ``krb_principals`` +for each user instead of the previous one-to-one mapping. In addition: + +* all APIs related to user or krb principals are changed +* ``userinfo`` of ``getUser`` will contain a new list ``krb_principals`` + * ``krb_principals`` will contain all available principals if ``krb_princs=True`` +* there is a new hub option ``AllowedKrbRealms`` to indicate which realms are allowed +* there is a new client option ``krb_server_realm`` to allow krbV login to set server realm + * Previously same as client principal realm before, supported by all clients +* ``QueryProcessor`` has a new queryOpt ``group``, which is used to generate ``GROUP BY`` section + * By default, this feature is disabled by arg ``enable_group=False`` + + +**Added cronjob for sessions table maintenance** + +| PR: https://pagure.io/koji/pull-request/1492 + +The sessions table is now periodically cleaned up via script (handled by cron by default). +Without this the sessions table can grow large enough to affect Koji performance. + + +**Added basic email template for koji-gc** + +| PR: https://pagure.io/koji/pull-request/1430 + +The email message koji-gc uses has been moved to ``/etc/koji-gc/email.tpl`` for +easier customization. + + +**Add all permissions to database** + +| PR: https://pagure.io/koji/pull-request/1681 + +Permissions previously missing from schema have been added, including ``dist-repo``, ``host``, +``image-import``, ``sign``, ``tag``, and ``target``. + + +**Add new CoreOS artifact types** + +| PR: https://pagure.io/koji/pull-request/1616 + +This change adds the new CoreOS artifact types ``iso-compressed``, ``vhd-compressed``, +``vhdx-compressed``, and ``vmdk-compressed`` to the database. + + +**Enforce unique content generator names in database** + +| PR: https://pagure.io/koji/pull-request/1159 + +Set a uniqueness constraint on the content generator name in the database. +Prior to this change, we were only enforcing this in the hub application layer. +Configure this in postgres for safety. + + +**Fix typo preventing VM builds** + +| PR: https://pagure.io/koji/pull-request/1666 + +This change fixes the options passed to ``verifyChecksum`` which was preventing VM builds. + + +**Fix verifyChecksum for non-output files** + +| PR: https://pagure.io/koji/pull-request/1670 + +``verifyChecksum`` now accepts files under the build requires path as well as the output path. +Other paths can be added as needed. + + +**Set f30+ python-devel default** + +| PR: https://pagure.io/koji/pull-request/1683 + +When installed on a Fedora 30+ host with Python 2 support, Koji will now require +``python2-devel`` instead of ``python-devel``. + + +**Handle sys.exc_clear for Python 3** + +| PR: https://pagure.io/koji/pull-request/1642 + +The method ``sys.exc_clear`` does not exist in Python 3, so it has been escaped for those instances. + + +**Remove deprecated koji.util.relpath** + +| PR: https://pagure.io/koji/pull-request/1458 + +``koji.util.relpath`` was deprecated in 1.16 and has been removed from 1.19. + + +**Remove deprecated BuildRoot.uploadDir** + +| PR: https://pagure.io/koji/pull-request/1511 + +``BuildRoot.uploadDir`` was deprecated in 1.18 and has been removed from 1.19. + + +**Remove deprecated koji_cli.lib_unique_path** + +| PR: https://pagure.io/koji/pull-request/1512 + +``koji_cli.lib_unique_path`` was deprecated in 1.17 and has been removed from 1.19. + + +**Deprecation of sha1_constructor and md5_constructor** + +| PR: https://pagure.io/koji/pull-request/1490 + +``sha1_constructor`` and ``md5_constructor`` have been deprecated in favor of ``hashlib``. diff --git a/docs/source/release_notes_1.13.rst b/docs/source/release_notes_1.13.rst deleted file mode 100644 index bf0b58d..0000000 --- a/docs/source/release_notes_1.13.rst +++ /dev/null @@ -1,173 +0,0 @@ -Koji 1.13 Release Notes -======================= - -Migrating from Koji 1.12 ------------------------- - -For details on migrating see :doc:`migrating_to_1.13` - - -Client Changes --------------- - -Python 3 client support -^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/417 - -The koji command and core library now support Python 3 (as well as 2). The -default spec now produces both `python2-koji` and `python3-koji` -subpackages. The `koji` package still contains the (now much smaller) -``/usr/bin/koji`` file. - -Some older features are not supported by the Python 3 client - - * the `use_old_ssl` option is not supported, python-requests must be used - * the old kerberos auth mechanism is not supported, use gssapi instead - -CLI Plugins -^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/199 - -The command line interface now has basic plugin support. The primary use case -is for plugins to be able to add new subcommands. -For details see: :ref:`plugin-cli-command` - -list-channels CLI command -^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/442 - -The new `list-channels` command lists the known channels for the system. - -.. code-block:: text - - Usage: koji list-channels - (Specify the --help global option for a list of other help options) - - Options: - -h, --help show this help message and exit - --quiet Do not print header information - -hostinfo CLI command -^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/399 -| Issue: https://pagure.io/koji/issue/364 - -The new ``hostinfo`` command shows basic information about a build host, -similar to the web interface. - -.. code-block:: text - - Usage: koji hostinfo [options] [ ...] - (Specify the --help global option for a list of other help options) - - Options: - -h, --help show this help message and exit - -Enhancements to restart-hosts -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/472 - -The `restart-hosts` command is used by admins to safely restart the build hosts -after a configuration change. - -Because multiple restarts can conflict, the command will now exit with a error -if a restart is already underway (can be overridden with --force). - -There are now options to limit the restart to a given channel or arch. - -The command now has a timeout option, which defaults to 24hrs. - -User-Agent header -^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/393 -| Issue: https://pagure.io/koji/issue/392 - -Previously the Koji client library reported a confusingly out-of-date value -in the ``User-Agent`` header. Now it simply reports the major version. - -raise error on non-existing profile -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/375 -| Issue: https://pagure.io/koji/issue/370 - -If the requested client profile is not configured, the library will raise an -error, rather than proceeding with default values. - -See also: :doc:`profiles` - - -Changes to the Web interface ----------------------------- - -Build Log Display -^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/471 - -The build info pages now display the log files for a build (instead of linking -directly to the directory on the download server). This works for all builds, -including those imported by content generators. - - -Builder changes ---------------- - -Configuring mock chroot behavior -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/400 -| Issue: https://pagure.io/koji/issue/398 - -Koji now supports using mock's --new-chroot option on a per-tag basis. -For details see: :ref:`tuning-mock-per-tag` - -pre/postSCMCheckout callbacks -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The callback interface is used by plugins to hook into various Koji operations. -With this release we have added callbacks in the builder daemon for before and -after source checkout: ``preSCMCheckout`` and ``postSCMCheckout``. - -Extended allowed_scms format -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/421 - -The allowed_scms option now accepts entries like: - -:: - - !host:repository - -to explicitly block a host:repository pattern. - -See also: :ref:`scm-config` - - -System changes --------------- - -mod_auth_gssapi required -^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/444 - -On modern platforms, both koji-hub and koji-web now require -mod_auth_gssapi instead of mod_auth_kerb. - - -Longer tag names -^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/388 -| Issue: https://pagure.io/koji/issue/369 - -Previously, tag names were limited to 50 characters. They are now limited -to 256 characters. diff --git a/docs/source/release_notes_1.14.rst b/docs/source/release_notes_1.14.rst deleted file mode 100644 index 6e1be87..0000000 --- a/docs/source/release_notes_1.14.rst +++ /dev/null @@ -1,310 +0,0 @@ -Koji 1.14 Release Notes -======================= - -Migrating from Koji 1.13 ------------------------- - -For details on migrating see :doc:`migrating_to_1.14` - - -Client Changes --------------- - - -Fail fast option for builds -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/432 - - -When builders are configured with ``build_arch_can_fail = True`` then the -failure of a single buildArch task does not immediately cause the build -to fail. Instead, the remaining buildArch tasks are allowed to complete, -at which point the build will still fail. - -Sometimes developers would rather a build fail immediately, so we have added -the ``--fail-fast`` option to the build command, which overrides this setting. -The option only has an effect if the builders are configured to fail slow. - - -Custom Lorax templates -^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/419 - -Koji now supports custom Lorax templates for the ``spin-livemedia`` command. -The command accepts two new options: - -.. code-block:: text - - --lorax_url=URL The URL to the SCM containing any custom lorax - templates that are to be used to override the default - templates. - --lorax_dir=DIR The relative path to the lorax templates directory - within the checkout of "lorax_url". - - -The Lorax templates must come from an SCM, and the ``allowed_scms`` rules -apply. - -When these options are used, the templates will be fetched and an appropriate -``--lorax-templates`` option will be passed to the underlying livemedia-creator -command. - - -Allow profiles to request a specific python version -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/566 - -On platforms with python3 available, the Koji client is built to execute -with the python3 binary. However, there are a few client features that do not -work under python3, notably old-style (non-gssapi) Kerberos authentication. - -If this issue is affecting you, you can set ``pyver=2`` in your Koji -configuration. This can be done per profile. When Koji sees this setting -at startup, it will re-execute itself under the requested python binary. - - -New list-builds command -^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/526 - -The command line now has a ``list-builds`` command that has similar -functionality to the builds tab of the web interface. - -.. code-block:: text - - Usage: koji list-builds [options] - (Specify the --help global option for a list of other help options) - - Options: - -h, --help show this help message and exit - --package=PACKAGE List builds for this package - --buildid=BUILDID List specific build from ID or nvr - --before=BEFORE List builds built before this time - --after=AFTER List builds built after this time - --state=STATE List builds in this state - --type=TYPE List builds of this type. - --prefix=PREFIX Only list packages starting with this prefix - --owner=OWNER List builds built by this owner - --volume=VOLUME List builds by volume ID - -k FIELD, --sort-key=FIELD - Sort the list by the named field - -r, --reverse Print the list in reverse order - --quiet Do not print the header information - - -New block-group command -^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/509 - -The ``block-group`` command allows admins to block package group entries -without having to resort to the ``call`` command. - -.. code-block:: text - - Usage: koji block-group - (Specify the --help global option for a list of other help options) - - Options: - -h, --help show this help message and exit - - -Exit codes for some commands -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/558 -| PR: https://pagure.io/koji/pull-request/559 - -Several more commands will now return a non-zero exit code -when an error occurs: - - * the various image building commands - * the ``save-failed-tree`` command (provided by a plugin) - - -Easier for scripts to use activate_session -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/493 - -In Koji 1.13.0, it became possible for scripts to ``import koji_cli.lib`` and -gain access to the ``activate_session`` function that the command line tool -uses to authenticate. - -In this release, this function has been made easier for scripts to use: - - * the options argument can now be a dictionary - * less options need to be specified - - -Builder changes ---------------- - - -Normalize paths for scms -^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/591 - - -For many years, kojid has supported the ``allowed_scms`` option -(see: :ref:`scm-config`) for controlling which scms can be used for building. -In 1.13.0, Koji added the ability to explicitly block a host:path pattern. - -Unfortunately, 1.13.0 did not normalize the path before checking the pattern, -making it possible for users to use equivalent paths to route around the -block patterns. - -Now, Koji will normalize these paths before the ``allowed_scms`` check. - - -Graceful reload -^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/565 - - -For a long time kojid has handled the USR1 signal by initiating a graceful restart. -This change exposes that in the systemd service config (and the init script -on older platforms). - -Now, ``service kojid reload`` will trigger the same sort of restart that the -``restart-hosts`` command accomplishes, but only for the build host you run it -on. When this happens, kojid will: - - * stop taking new tasks - * wait for current tasks to finish - * restart itself once all its tasks are completed - - -Friendlier runroot configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/539 -| PR: https://pagure.io/koji/pull-request/528 - -Two changes make it easier to write a configuration for the runroot plugin. - -The ``path_subs`` option is now more forgiving about whitespace: - - * leading and trailing whitespace is ignored for each line - * blank lines are ignored - -The ``[pathNN]`` sections are no longer required to have sequential numbers. -Previously, the plugin expected a sequence like ``[path0]``, ``[path1]``, -``[path2]``, etc, and would stop looking for entries if the next number -was missing. Now, any set of distinct numbers is valid and all ``[pathNN]`` -sections will be processed. - - -System changes --------------- - -Deprecations -^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/554 -| PR: https://pagure.io/koji/pull-request/597 - -The following features are deprecated and will be removed in a future release: - - * the ``importBuildInPlace`` rpc call - * the ``use_old_ssl`` client configuration option (and the underlying - ``koji.compatrequests`` library) - - -Removed calls -^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/497 -| PR: https://pagure.io/koji/pull-request/507 - -The deprecated ``buildFromCVS`` hub call has been removed. It was replaced -by the ``buildSRPMFromCVS`` call many years ago and has been deprecated since -version 1.6.0. - -The ``add_db_logger`` function has been removed from the koji library, along -with the ``log_messages`` table in the db. This extraneous call has never been -used in Koji. - - -Dropped mod_python support -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/508 - - -Koji no longer supports mod_python. This option has been deprecated since -mod_wsgi support was added in version 1.7.0. - -See also: :doc:`migrating_to_1.7` - - -Large integer support -^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/571 - - -Koji uses xmlrpc for communications with the hub, and unfortunately the -baseline xmlrpc standard only supports 32-bit signed integers. This -results in errors when larger integers are encountered, typically -when a file is larger than 2 GiB. - -Starting with version 1.14.0, Koji will emit ``i8`` tags when encoding -large integers for xmlrpc. Integers below the limit are still encoded -with the standard ``int`` tag. The only time this makes a difference -is when Koji would previously have raised an ``OverflowError``. - -The ``i8`` tag comes from the -`ws-xmlrpc `__ -spec. Python's xmlrpc decoder has -for many years accepted and understood this tag, even though its encoder -would not emit it. - -Previous versions of Koji worked around such size issues by converting -large integers to strings in a few targeted places. Those targeted -workarounds have been left in place on the hub for the sake of backward -compatibility. - - -Test mode for protonmsg plugin -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/538 - -The ``protonmsg`` plugin now accepts a boolean ``test_mode`` configuration -option. When this option is enabled, the plugin will not actually -send messages, but will instead log them (at the DEBUG level). - -This option allows testing environments to run with the plugin enabled, but -without requiring a message bus to be set up for that environment. - - -Handling of debugsource rpms -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/524 - -Koji will now treat rpms ending in ``-debugsource`` the same way that it does -other debuginfo rpms. Such rpms are: - - * omitted from Koji's normal yum repos - * listed separately when displaying builds - * not downloaded by default in the ``download-build`` command - - -Added kojifile component type for content generators -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/506 - -Content generator imports now accept entries with type equal to ``kojifile`` -in the component lists for buildroots and images/archives. This type provides -a more reliable way to reference archive that come from Koji. - -See: :ref:`Example metadata `. diff --git a/docs/source/release_notes_1.15.1.rst b/docs/source/release_notes_1.15.1.rst deleted file mode 100644 index dc803c9..0000000 --- a/docs/source/release_notes_1.15.1.rst +++ /dev/null @@ -1,34 +0,0 @@ -Koji 1.15.1 Release Notes -========================= - -Koji 1.15.1 is a bugfix release for Koji 1.15. The most important change -is the fix for :doc:`CVE-2018-1002150`. - -Please see: :doc:`release_notes_1.15` - -Issues fixed in 1.15.1 ----------------------- - -- `Issue 850 `_ -- - CVE-2018-1002150 - -- `Issue 846 `_ -- - error occurs in SCM.get_source since subprocess.check_output is not supported by python 2.6- - -- `Issue 724 `_ -- - buildNotification of wrapperRPM fails because of task["label"] is None - -- `Issue 786 `_ -- - buildSRPMFromSCM tasks fail on koji 1.15 - -- `Issue 803 `_ -- - Email notifications makes build tasks fail with "KeyError: 'users_usertype'" - -- `Issue 742 `_ -- - dict key access fail in koji_cli.commands._build_image - -- `Issue 811 `_ -- - AttributeError: 'dict' object has no attribute 'hub.checked_md5' - -- `Issue 813 `_ -- - cg imports fail with "Unsupported checksum type" diff --git a/docs/source/release_notes_1.15.rst b/docs/source/release_notes_1.15.rst deleted file mode 100644 index a6164b8..0000000 --- a/docs/source/release_notes_1.15.rst +++ /dev/null @@ -1,279 +0,0 @@ -Koji 1.15 Release Notes -======================= - -Updates -------- - -- :doc:`Koji 1.15.1 ` is a security update for Koji 1.15 - -Migrating from the previous release ------------------------------------ - -For details on migrating see :doc:`migrating_to_1.15` - - -Client Changes --------------- - - -Display license Info -^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/686 - - -The ``rpminfo`` command now displays the ``License`` field from the rpm. - - -Keytabs for GSSAPI authentication -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/708 - -Previously keytabs were only supported by the older kerberos auth method, which -is not available on Python 3. Now the gssapi method supports them as well. - - -Add krb_canon_host option -^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/653 - -This release adds a ``krb_canon_host`` option that tells Koji clients -to use the dns canonical hostname for kerberos auth. - -This option allows kerberos authentication to work in situations where -the hub is accessed via a cname, but the hub's credentials are under -its canonical hostname. - -If specified, this option takes precedence over the older -option named ``krb_rdns``. That option caused Koji clients to perform a -reverse name lookup for kerberos auth. - -When configuring kojiweb (in web.conf), the option is named ``KrbCanonHost``. - -Both options only affect the older kerberos authentication path, and not -gssapi. - - -Watch-task return code -^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/703 - -Previously, the ``watch-task`` command would return a non-zero exit status -if any subtask failed, even if this did not cause the parent task to fail. - -Now that we have cases where subtasks are optional, this no longer makes sense. -The exit code is now based solely on the results of -the top level tasks it is asked to watch. - - -New runroot options -^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/633 - -The ``runroot`` command now supports options similar to the various build commands. These new -options are: - - -.. code-block:: text - - --nowait Do not wait on task - --watch Watch task instead of printing runroot.log - --quiet Do not print the task information - - -New watch-logs options -^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/625 - -The ``watch-logs`` command now supports the following new options: - -.. code-block:: text - - --mine Watch logs for all your tasks - --follow Follow spawned child tasks - - -Web UI changes --------------- - -Archive component display -^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/610 - -Previously, the web UI only displayed component lists for image builds. -However, new build types can also have component lists. - -Now the interface will display components for any archive that has them. - - -Display license Info -^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/686 - - -The ``rpminfo`` page now displays the ``License`` field from the rpm. - - -Show suid bit -^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/617 - -The web UI will now display the setuid bit when displaying rpm/archive file contents. - - - - -Builder changes ---------------- - - -Alternate tmpdir for mock chroots -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/602 - - -Recent versions of mock (1.4+) default to ``use_nspawn=True``, which results -in /tmp being a fresh tmpfs mount on every run. This means the /tmp -directory no longer persists outside of the mock invocation. - -Now, the builder will use /builddir/tmp instead of /tmp for persistent data. - - -Store git commit hash -^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/674 - -In Koji, for builds from an SCM, the source is specified as an -scm url. -For git urls, the revision in that url can be anything that git -will recognize, including: - - - a sha1 ref - - an abbreviated sha1 ref - - a branch name - - a tag - - HEAD - -With this change: - - * the revision is replaced with the full sha1 ref for git urls - * the scm url is stored in build.source - * the original scm url is saved in build.extra - -Previously, this source url was not properly stored for rpm builds. It -appeared in the task parameters, but the build.source field remained blank. -If a symbolic git ref (e.g. HEAD) was given in the url, the underlying -sha1 value was only recorded in the task logs. - - - -System changes --------------- - - -Volume policy support -^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/622 - -Koji has for many years had the ability to split its storage across multiple -volumes. However, there is no automatic process for placing builds onto -volumes other than the primary. To do so often requires a lot of manual work -from an admin. - -This feature: - - * adds a volume policy check to the key import pathways - * adds an applyVolumePolicy call to apply the policy to existing builds - -The hub consults the volume policy at various points to -determine where a build should live. This allows admins to make rules like: - - - all kernel builds go to the volume named kstore - - all builds built from the epel-7-build tag go to the volume named epel7 - - all builds from the osbs content generator go to the volume named osbs - -The default policy places all builds on the default volume. - -See also: :doc:`volumes` - -Messagebus plugin changes -^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/537 - -There are two notable changes to the messagebus plugin this release: - - -Deferred sending -"""""""""""""""" - -Similar to the current behavior of the protonmsg plugin, messages are queued -up during hub calls and only sent out during the ``postCommit`` callback. - -This avoids sending messages about failed calls, which can be confusing to -message consumers (e.g. build state change messages about a build that does -not exist because it failed to import). - -Test mode -""""""""" - -The plugin now looks for a boolean ``test_mode`` option. If it is true, then -the messages are still queued up, but not actually sent. This makes it -possible to enable the plugin in test environments without having to set up a -separate message bus. - - -Protonmsg plugin changes -^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/657 -| PR: https://pagure.io/koji/pull-request/651 - -There are two changes to how the protonmsg plugin handles rpmsign events: - - 1. The arch of the rpm is included in messages - 2. The message are omitted when the sigkey is empty - - - -No notifications for disabled users or hosts -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/615 - - -Koji will no longer send out email notifications to disabled users or -to users corresponding to a host. - - -Replace pycurl with requests -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/601 - -All uses of the pycurl library have been replaced with calls -to python-requests, so pycurl is no longer required. - - -Drop importBuildInPlace call -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -| PR: https://pagure.io/koji/pull-request/606 - -The deprecated ``importBuildInPlace`` call has been dropped. - -This call was an artifact of a particular bootstrap event that happened a long -time ago. It was never really documented or recommended for use. - - diff --git a/docs/source/release_notes_1.16.1.rst b/docs/source/release_notes_1.16.1.rst deleted file mode 100644 index eb2f5b3..0000000 --- a/docs/source/release_notes_1.16.1.rst +++ /dev/null @@ -1,59 +0,0 @@ -Koji 1.16.1 Release Notes -========================= - -Koji 1.16.1 is a point release for Koji 1.16. The major changes include: - -- Allow target info to be read for different type tasks in channel policy. -- Create symlinks for builds imported onto non-default volumes. -- Fix RPMdiff issues found in Koji 1.16.0. - -Please see: :doc:`release_notes_1.16` - -Issues fixed in 1.16.1 ----------------------- - -- `Issue 847 `_ -- - spin-livecd failed with "Could not resolve host" - -- `Issue 932 `_ -- - Fix use_host_resolv with new mock version - -- `Issue 1010 `_ -- - koji fails runroot because of `UnicodeDecodeError` - -- `Issue 998 `_ -- - cancel build doesn't work for images - -- `Issue 994 `_ -- - rpmdiff calculate wrong results - -- `Issue 1025 `_ -- - missing default volume symlink for imported builds affected by volume policy - -- `Issue 1007 `_ -- - decode_args() might result in --package parameter missing in runroot command - -- `Issue 150 `_ -- - no target info in channel policy for non-rpm tasks - -- `PR: 973 `_ -- - Check empty arches before spawning dist-repo - -- `Issue 958 `_ -- - Notification for tagBuildBypass is writing message untagged from, expected message tagged into - -- `Issue 968 `_ -- - Default enable python3 on RHEL8 - -- `Issue 916 `_ -- - `clone-tag` doesn't preserve tagging order - -- `Issue 949 `_ -- - cli: [rpminfo] KeyError: 'license' for external RPM - -- `Issue 876 `_ -- - koji clone-tag raises "UnboundLocalError" - -- `Issue 945 `_ -- - Koji build fail due to ambiguous python shebang - diff --git a/docs/source/release_notes_1.16.2.rst b/docs/source/release_notes_1.16.2.rst deleted file mode 100644 index 432b932..0000000 --- a/docs/source/release_notes_1.16.2.rst +++ /dev/null @@ -1,18 +0,0 @@ -Koji 1.16.2 Release Notes -========================= - -Koji 1.16.2 is a bugfix release for Koji 1.16. -The purpose of this release is address :doc:`CVE-2018-1002161`. - -See also: - -- :doc:`release_notes_1.16.1` - -- :doc:`release_notes_1.16` - - -Issues fixed in 1.16.2 ----------------------- - -- `Issue 1183 `_ -- - CVE-2018-1002161 diff --git a/docs/source/release_notes_1.16.rst b/docs/source/release_notes_1.16.rst deleted file mode 100644 index 36e332b..0000000 --- a/docs/source/release_notes_1.16.rst +++ /dev/null @@ -1,249 +0,0 @@ -Koji 1.16.0 Release notes -========================= - - -Migrating from Koji 1.15 ------------------------- - -For details on migrating see :doc:`migrating_to_1.16` - - - -Security Fixes --------------- - -**CVE-2018-1002150 - distRepoMove missing access check** - -This release includes the fix for :doc:`CVE-2018-1002150`. - - -Client Changes --------------- - -**CLI commands to manage notifications** - -| PR: https://pagure.io/koji/pull-request/688 - -The change adds new cli sub-commands: - - - list-notifications - - add-notification - - remove-notification - - edit-notification - -Previously this functionality was only available through the web ui or -by making direct api calls. - - -**Add --old-chroot option to runroot command** - -| PR: https://pagure.io/koji/pull-request/823 - -This option causes the runroot handler to pass the same-named option -to the mock command. This complements the existing ``--new-chroot`` -option. - -If neither ``--old-chroot`` or ``--new-chroot`` is given, then mock will -follow its default behavior. This default varies across mock versions. -For newer versions of mock, ``--new-chroot`` is the default (uses a -systemd nspawn container). - - -**Fix runroot output on py3** - -| PR: https://pagure.io/koji/pull-request/828 - -The runroot command should now work under python3. - - -**Honor runroot --quiet** - -| PR: https://pagure.io/koji/pull-request/806 - -The ``--quiet`` option was added to the runroot command in version 1.15, -but it only took effect when the ``--watch`` option was given. Now it is -honored in all cases. - - -**Drop old ssl code** - -| PR: https://pagure.io/koji/pull-request/498 - -The old ``koji.ssl`` module has been removed, and the ``use_old_ssl`` option -has been removed from client code. - -Because these files (which were originally from -`Plague `_) were the only parts -of Koji that were licensed as GPLv2+, Koji is now simply licensed as -LGPLv2. - - -Builder Changes ---------------- - -**Configure install timeout for imagefactory** - -| PR: https://pagure.io/koji/pull-request/841 - -Previously the install timeout parameter for imagefactory was set -to a fixed value of 7200 by Koji. Now it can be controlled by -setting the ``oz_install_timeout`` option in ``kojid.conf``. - -A value of ``0`` will disable the timeout. - - -**Record log timestamps** - -| PR: https://pagure.io/koji/pull-request/777 - -If the ``log_timestamps`` option is enabled in ``kojid.conf``, then -the builder will record a separate timestamp file for each log file -in a build. - -The filename for the timestamp file is generated by taking the name -of the log file and appending ``-ts.log``. So ``build.log`` will have -timestamp data in ``build.log-ts.log``. - -The format of the timestamp log is plain text with each line showing -a numeric timestamp and a line offset. - - -**Builder option: chroot_tmpdir** - -| PR: https://pagure.io/koji/pull-request/787 - -The new ``chroot_tmpdir`` option controls which directory within buildroots -is used for various temporary data by the Koji builder daemon. -Previously this was hardcoded to ``/builddir/tmp``, which created problems -with modern versions of mock. - -The default value is ``/chroot_tmpdir``. - - -**Add internal_dev_setup option to runroot config** - -| PR: https://pagure.io/koji/pull-request/824 - -The ``internal_dev_setup`` config option for the runroot builder plugin -controls whether the mock option of the same name is set for runroot -tasks. - - - -System Changes --------------- - - -**Add option to configure DB port** - -| PR: https://pagure.io/koji/pull-request/884 - -The hub now accepts a ``DBPort`` option in ``hub.conf``, which specifies -which port the hub should use when connecting to the database. - - -**Split debuginfo for dist repos** - -| PR: https://pagure.io/koji/pull-request/914 - -Dist repos can now be generated with debuginfo files split into a separate -repo. The behavior is controlled by passing the ``--split-debuginfo`` option -to the ``dist-repo`` subcommand. - -When this option is in effect, the main repo will be in the normal location. -The debuginfo repo will be in the ``debug`` subdirectory. So, you will -see a directory structure like: - -.. code-block:: text - - - Packages/ - repodata/ - debug/ - debug/repodata - -Regardless of the split, all the rpms are located in the top level -``Packages`` directory. - - -**Notifications in [un]tagBuildBypass** - -| PR: https://pagure.io/koji/pull-request/691 - -Previously the ``tagBuildBypass`` and ``untagBuildBypass`` calls did not trigger -notifications. Now they will do so by default. The call now accepts a -``notify`` option (defaults to True) which controls the behavior. - - -**Track history for host data** - -| PR: https://pagure.io/koji/pull-request/778 - -Koji now tracks changes to host data similarly to the way it tracks -changes for other data. This includes - - - enabled state - - arches - - capacity - - description & comment - - channels - -The ``list-history`` cli command now supports ``--host`` and ``--channel`` -options to select history entries for a host or channel. - -The versioned host data is stored in the ``host_config`` and ``host_channels`` -tables. - - -**Fix block-group functionality** - -| PR: https://pagure.io/koji/pull-request/678 - -The ``block-group`` command and its underlying api call now actually work. - - -**Strict option for archive listing calls** - -| PR: https://pagure.io/koji/pull-request/734 -| PR: https://pagure.io/koji/pull-request/748 - -The ``list_archives``, ``get_archive_file()``, and ``list_archive_files()`` -hub functions now accept a strict option, which defaults to False. When -the option is True, the call will raise an exception if there is no -match. - - -**Search build by source** - -| PR: https://pagure.io/koji/pull-request/765 - -The ``listBuilds()`` api call now supports a source option. This is -treated as a glob pattern and matched against the ``source`` field of the build. - - -**Option to ignore tags in kojira** - -| PR: https://pagure.io/koji/pull-request/695 - -Kojira now supports an ``ignore_tags`` option. This is treated as a -space-separated list of glob patterns. Tags that match are ignored -by kojira (it will not generate newRepo tasks for them). - - -**Improve kojira throughput** - -| PR: https://pagure.io/koji/pull-request/797 - -Kojira should be much more responsive in triggering ``newRepo`` tasks. - - -**Drop migrateImage call** - -| PR: https://pagure.io/koji/pull-request/632 - -The ``migrateImage`` call hub call has been removed. - -This call was added in version 1.8 (April 2013) -as a one-time tool for migrating images from the old model (no build entry) -to the new model (image build type). It was only available if the -EnableImageMigration option was set on the hub. diff --git a/docs/source/release_notes_1.17.rst b/docs/source/release_notes_1.17.rst deleted file mode 100644 index ac80b0d..0000000 --- a/docs/source/release_notes_1.17.rst +++ /dev/null @@ -1,242 +0,0 @@ -Koji 1.17.0 Release notes -========================= - - -Migrating from Koji 1.16 ------------------------- - -For details on migrating see :doc:`migrating_to_1.17` - - - -Security Fixes --------------- - -**CVE-2018-1002161 - SQL injection in multiple remote calls** - -| PR: https://pagure.io/koji/pull-request/1274 - -This release includes the fix for :doc:`CVE-2018-1002161` - - -Client Changes --------------- - -**Volume id option for livemedia and livecd tasks** - -| PR: https://pagure.io/koji/pull-request/1227 - -The ``spin-livecd`` and ``spin-livemedia`` commands now accept a ``--volid`` -argument to specify the volume id for the media. If unspecified, the -volume id is chosen via the same heuristic as before. - -Volume ids must be 32 characters or less. - - - -**Build order preserved by clone-tag** - -| PR: https://pagure.io/koji/pull-request/1014 - -This is an improvement to the ``clone-tag`` command. Previously, when the -command was used without the ``--latest-only`` option, it could get the -ordering of builds wrong in the destination tag. Now, the order will -match the source tag. - - - -**Configurable authentication timeout** - -| PR: https://pagure.io/koji/pull-request/1172 - -Previously, the network timeout during authentication was hard coded to -60 seconds. It is now configurable via the ``auth_timeout`` configuration -option. - - -**Additional information from list-channels command** - -| PR: https://pagure.io/koji/pull-request/940 - -The ``list-channels`` command now shows three separate host counts for -each channel: - -- the number of enabled hosts in the channel -- the number of ready hosts in the channel -- the number of disabled hosts in the channel - - -**The free-task command requires at least one task-id** - -| PR: https://pagure.io/koji/pull-request/1045 - -Previously this command was a no-op when given no arguments. Now it will return an -error. - - - -Library Changes ---------------- - -**Drop encode_int function** - -| PR: https://pagure.io/koji/pull-request/852 - -This is a follow up to the large integer support that we added in version 1.14 - -See also: :doc:`release_notes_1.14` - -The ``encode_int`` function is no longer used -and has been dropped from the library. - -Because we no longer call ``encode_int``, the hub will now always use i8 tags -when returning large integers, rather than returning them as strings in some -cases. - - -**Use custom Kerberos context with krb_login** - -| PR: https://pagure.io/koji/pull-request/1187 - -Clients can now pass in their own Kerberos context to -``ClientSession.krb_login()`` using -the ``ctx`` parameter. This is intended for multi-threaded clients. - - -**Custom keyboard interrupt handling in watch_tasks** - -| PR: https://pagure.io/koji/pull-request/981 - -The new ``ki_handler`` option for the ``koji_cli.lib.watch_tasks()`` function -allows other cli tools to set their own handler for keyboard interrupts. -If specified, the value should be callable and will be called when a -keyboard interrupt is encountered. -If unspecified, the original behavior is retained. - - -**_unique_path() -> unique_path** - -| PR: https://pagure.io/koji/pull-request/980 - -The ``_unique_path`` function is deprecated. It has been replaced -by ``unique_path``. - - -Web UI Changes --------------- - -**Additional info on builders in channelinfo page** - -| PR: https://pagure.io/koji/pull-request/989 - -The channelinfo page now shows enabled/ready status for each host and a count -for each. - - - -Builder Changes ---------------- - -**Builder task_avail_delay check** - -| PR: https://pagure.io/koji/pull-request/1176 - -This delay works around a deficiency in task scheduling. The default -delay is 300 seconds and can be adjusted with the ``task_avail_delay`` -option to kojid. However, it is unlikely that admins will need to -adjust this setting. - -Despite the name, this does not introduce any new delay compared to the -old behavior. The setting controls how long a host will wait before taking -a task in a given channel-arch "bin" when that host has an available -capacity lower than the median for that bin. Previously, such hosts -could wait forever. - - - -System Changes --------------- - - -**Python 3 Support** - -| PR: https://pagure.io/koji/pull-request/1117 -| PR: https://pagure.io/koji/pull-request/891 -| PR: https://pagure.io/koji/pull-request/921 -| PR: https://pagure.io/koji/pull-request/1184 -| PR: https://pagure.io/koji/pull-request/1019 -| PR: https://pagure.io/koji/pull-request/685 -| ...and many fixes - -Support for Python 3 has been extended to all components of Koji. Including: - -- Hub -- Builder -- Web UI -- Utils - - - -**No more messagebus plugin** - -| PR: https://pagure.io/koji/pull-request/1043 - -The messagebus plugin has been dropped. The protonmsg plugin is still -available. - - - -**Simple mode for mergerepos** - -| PR: https://pagure.io/koji/pull-request/1066 - -External repos now have a ``merge_mode`` option. Valid values are -either ``koji`` (the old way) or ``simple`` (a new alternative). This -option can be set with the ``--mode`` option to the ``add-external-repo`` -or ``edit-external-repo`` commands. - -When an external repo is merged with simple mode, a number of the complex -filters that Koji normally applies are skipped. This mode still honors -the block list from Koji and ignores duplicate NVRAs, but otherwise -it simply merges the repo in. - -Multiple merge modes cannot be combined in a single tag. If a tag -has two external repos with different modes, then the repo will -fail to generate. - - -**Avoid "unknown task" errors in Kojira** - -| PR: https://pagure.io/koji/pull-request/1175 - -This is a bug fix for a minor race condition in Kojira that could cause -errors in the log and redundant repo regens. - - - -**Full filename display for kojifiles directory indexes** - -| PR: https://pagure.io/koji/pull-request/1156 - -This is simply a change to the default httpd configuration for serving -/mnt/koji. It adds ``NameWidth=*`` to ``IndexOptions`` so that long filenames -are fully displayed. - - - -**Broader support for target/source/scratch tests in channel policy** - -| PR: https://pagure.io/koji/pull-request/962 - -It is now possible to write channel policy rules based on -build target, source, and scratch options for task types other -than ``build``. - - - -**Longer Build Target names** - -| PR: https://pagure.io/koji/pull-request/925 - -Build target names can now be up to 256 characters, the same length -restriction as for tag names. diff --git a/docs/source/release_notes_1.18.1.rst b/docs/source/release_notes_1.18.1.rst deleted file mode 100644 index 94e6b6b..0000000 --- a/docs/source/release_notes_1.18.1.rst +++ /dev/null @@ -1,12 +0,0 @@ -Koji 1.18.1 Release Notes -========================= - -Koji 1.18.1 is a bugfix release for Koji 1.18. -The purpose of this release is address :doc:`CVE-2019-17109`. - - -Issues fixed in 1.18.1 ----------------------- - -- `Issue 1634 `_ -- - possible to upload file to a path other than work directory diff --git a/docs/source/release_notes_1.18.rst b/docs/source/release_notes_1.18.rst deleted file mode 100644 index cd1d15e..0000000 --- a/docs/source/release_notes_1.18.rst +++ /dev/null @@ -1,378 +0,0 @@ -Koji 1.18.0 Release notes -========================= - - -Migrating from Koji 1.17 ------------------------- - -For details on migrating see :doc:`migrating_to_1.18` - - - -Security Fixes --------------- - - - -Client Changes --------------- - -**Add option for custom cert location** - -| PR: https://pagure.io/koji/pull-request/1253 - -The CLI now has an option for setting a custom SSL certificate, similar to the -options for Kerberos authentication. - - -**Load client plugins from ~/.koji/plugins** - -| PR: https://pagure.io/koji/pull-request/892 - - -This change allows users to load their own cli plugins from ``~/.koji/plugins`` -or from another location by using the ``plugin_paths`` setting. - - -**Show load/capacity in list-channels** - -| PR: https://pagure.io/koji/pull-request/1449 - -The ``list-channels`` display has been expanded to show overall totals for load -and capacity. - - -**Allow taginfo cli to use tag IDs** - -| PR: https://pagure.io/koji/pull-request/1476 - -The ``taginfo`` command can now accept a numeric tag id on the command line. - - -**Add option to show channels in list-hosts** - -| PR: https://pagure.io/koji/pull-request/1425 - -The ``list-hosts`` command will now display channel subscriptions if the -``--show-channels`` option is given. - - -**Remove merge option from edit-external-repo** - -| PR: https://pagure.io/koji/pull-request/1499 - -This option was mistakenly added to the command and never did anything. -It is gone now. - - -**Honor mock.package_manager tag setting in mock-config cli** - -| PR: https://pagure.io/koji/pull-request/1374 - -The ``mock-config`` command will now honor this setting just as ``kojid`` does. - - - - -Library Changes ---------------- - -**New multicall interface** - -| PR: https://pagure.io/koji/pull-request/957 - -This feature implements a new and much better way to use multicall in the Koji -library. -These changes create a new implementation outside of ClientSession. -The old way will still work. - -With this new implementation: - -* a multicall is tracked as an instance of `MultiCallSession` -* the original session is unaffected -* multiple multicalls can be managed in parallel, if desired -* `MultiCallSession` behaves more or less like a session in multicall mode -* method calls return a `VirtualCall` instance that can later be used to access the result -* `MultiCallSession` can be used as a context manager, ensuring that the calls are executed - -Usage examples can be found in the :doc:`Writing Koji Code ` -document. - - - - -Web UI Changes --------------- - -**Retain old search pattern in web ui** - -| PR: https://pagure.io/koji/pull-request/1258 - -The search results page of the web ui now retains a search form with the -current search pre-filled. -This makes it easier for users to refine their searches. - - -**Display task durations in webui** - -| PR: https://pagure.io/koji/pull-request/1383 - - -The ``taskinfo`` page in the web ui now shows task durations in addition to -timestamps. - - - -Builder Changes ---------------- - -**Rebuild SRPMS before building** - -| PR: https://pagure.io/koji/pull-request/1462 - -For rpm builds from an uploaded srpm, Koji will now rebuild the srpm in the -build environment first. -This ensures that the NVR is correct for the resulting build. - -The old behavior can be requested by setting ``rebuild_srpm=False`` in the tag -extra data for the build tag in question. - - -**User createrepo_c by default** - -| PR: https://pagure.io/koji/pull-request/1278 - - -The ``use_createrepo_c`` configuration option for ``kojid`` now defaults to True. - - -**Use createrepo update option even for first repo run** - -| PR: https://pagure.io/koji/pull-request/1363 - -If there is no older repo for a tag, Koji will now attempt to find -a related repo to use ``createrepo --update`` with. -This will speed up first-time repo generations for tags that -predominantly inherit their content from another build tag. - - -**Scale task_avail_delay based on bin rank** - -| PR: https://pagure.io/koji/pull-request/1386 - -This is an adjustment to Koji's decentralized scheduling algorithm. -It should result in better utilization of host capacity, particularly when -a channel has hosts that are very heterogeneous in capacity. - -The meaning of the ``task_avail_delay`` setting is different now. -Within a channel-arch bin, the hosts with highest capacity will take the task -immediately, while hosts lower down will have a delay proportional to their -rank. -The "rank" here is a float between 0.0 and 1.0 used as a multiplier. -So ``task_avail_delay`` is the maximum time that any host will wait to -take a task. - -Hosts with higher available capacity will be more likely to claim a -task, resulting in better utilization of the highest capacity hosts. - - -**Use RawConfigParser for kojid** - -| PR: https://pagure.io/koji/pull-request/1544 - -The use of percent signs is common in ``kojid.conf`` because of the -``host_principal_format`` setting. -This causes an error in python3 if ``SafeConfigParser`` is used, so we use -``RawConfigParser`` instead. - - -**Handle bare merge mode** - -| PR: https://pagure.io/koji/pull-request/1411 -| PR: https://pagure.io/koji/pull-request/1516 -| PR: https://pagure.io/koji/pull-request/1502 - - -This feature adds a new merge mode for external repos named ``bare``. -This mode is intended for use with modularity. - -Use of this mode requires createrepo_c version 0.14.0 or later on the builders -that handle the createrepo tasks. - - - - -System Changes --------------- - - -**API for reserving NVRs for content generators** - -| PR: https://pagure.io/koji/pull-request/1464 -| PR: https://pagure.io/koji/pull-request/1597 -| PR: https://pagure.io/koji/pull-request/1601 -| PR: https://pagure.io/koji/pull-request/1602 -| PR: https://pagure.io/koji/pull-request/1606 - -This feature allows content generators to reserve NVRs earlier in the build -process similar to builds performed by ``kojid``. The NVR is reserved by -calling ``CGInitBuild()`` and finalized by the ``CGImport()`` call. - - - -**Per-tag configuration of rpm macros** - -| PR: https://pagure.io/koji/pull-request/898 - -This feature allows setting rpm macros via the tag extra field. These macros -will be added to the mock configuration for the buildroot. The system -looks for extra values of the form ``rpm.macro.NAME``. - -For example, to set the dist tag for a given tag, you could use a command like: - -:: - - $ koji edit-tag f30-build -x rpm.macro.dist=MYDISTTAG - - - -**Per-tag configuration for module_hotfixes setting** - -| PR: https://pagure.io/koji/pull-request/1524 -| PR: https://pagure.io/koji/pull-request/1578 - -Koji now handles the field ``mock.yum.module_hotfixes`` in the tag extra. -When set, kojid will set ``module_hotfixes=0/1`` in the yum portion of the -mock configuration for a buildroot. - - -**Allow users to opt out of notifications** - -| PR: https://pagure.io/koji/pull-request/1417 -| PR: https://pagure.io/koji/pull-request/1580 - -This feature lets users opt out of notifications that they would otherwise -automatically recieve, such as build and tag notifications for: - -- the build owner (the user who submitted the build) -- the package owner within the given tag - -These opt-outs are user controlled and can be managed with the new -``block-notification`` and ``unblock-notificiation`` commands. - - -**Allow hub policy to match version and release** - -| PR: https://pagure.io/koji/pull-request/1513 - - -This feature adds new policy tests to match ``version`` and ``release``. -This tests are glob pattern matches. - - -**Allow hub policy to match build type** - -| PR: https://pagure.io/koji/pull-request/1415 - - -Koji added btypes in version 1.11 along with content generators. -Now, all builds have one or more btypes. - -This change allows policies to check the btype value using the ``buildtype`` test. - - - -**More granular admin permissions** - -| PR: https://pagure.io/koji/pull-request/1454 - -A number of actions that were previously admin-only are now governed by -separate permissions: - - ``host`` - This permission governs most host management operations, such as - adding, editing, enabling/disabling, and restarting. - - ``tag`` - This permission governs adding, editing, and deleting tags. - - ``target`` - This permission governs adding, editing, and deleting targets. - -Koji administrators may want to consider reducing the number of users with -full ``admin`` permission. - - -**Option to generate separate source repo** - -| PR: https://pagure.io/koji/pull-request/1273 - -The (non-dist) yum repos that Koji generates for building normally don't -include srpms. -An old option allowed them to be included in some cases, but they were simply -added to each repo. -Newer options have been added that instruct Koji to include them as a separate -src repo. - -In the cli, the ``regen-repo`` command now accepts a ``--separate-source`` -option that triggers this behavior. - -In ``kojira``, the ``separate_source_tags`` option is a list of tag patterns. -Build tags that match any of these patterns will have their repos generated -with a separate src repo. - - - -**Add volume option for dist-repo** - -| PR: https://pagure.io/koji/pull-request/1327 - -Dist repos can now be generated on volumes other than the main one. -Use the ``--volume`` option to the ``dist-repo`` command to do so. - -Generally you want the repo to be on the same volume as the rpms it will -contain. -Dist repos hard link (same volume) or copy (different volume) their rpms into -place. -Using the appropriate volume can drastically improve the efficiency, both in -generation time and space consumption. - - -**Minor gc optimizations** - -| PR: https://pagure.io/koji/pull-request/1337 -| PR: https://pagure.io/koji/pull-request/1442 -| PR: https://pagure.io/koji/pull-request/1437 - -This change speeds up portions of garbage collection by making the -``build_references`` check lazy by default. - - - -**Rollback errors in multiCall** - -| PR: https://pagure.io/koji/pull-request/1358 - -If one of the calls in a multicall raises an error, then the transaction will -be rolled back to the start of that call before Koji proceeds to the next call. -This matches the behavior of normal calls more closely. - -Multicalls are still handled within single database transaction. - - - -**Support tilde in search** - -| PR: https://pagure.io/koji/pull-request/1297 - - -The tilde character is no longer prohibited in search terms. - - - -**Remove 'keepalive' option** - -| PR: https://pagure.io/koji/pull-request/1277 - -The ``keepalive`` setting is no longer used anywhere in koji. -It has been removed. diff --git a/docs/source/release_notes_1.19.rst b/docs/source/release_notes_1.19.rst deleted file mode 100644 index bd68cc0..0000000 --- a/docs/source/release_notes_1.19.rst +++ /dev/null @@ -1,416 +0,0 @@ -Koji 1.19.0 Release notes -========================= - - -Migrating from Koji 1.18 ------------------------- - -For details on migrating see :doc:`migrating_to_1.19` - - - -Security Fixes --------------- - -**GSSAPI authentication checks kerberos principal** - -| PR: https://pagure.io/koji/pull-request/1419 - -When using GSSAPI authentication the user's kerberos principal will be checked -for their username to avoid a potential username and kerberos principal mismatch. - - - -Client Changes --------------- - -**Add user edit** - -| PR: https://pagure.io/koji/pull-request/902 -| PR: https://pagure.io/koji/pull-request/1701 -| PR: https://pagure.io/koji/pull-request/1713 - -A new ``edit-user`` command and API call was added, allowing for user rename, -and changing, adding, or removing the kerberos principal of a user. - - -**Add remove group** - -| PR: https://pagure.io/koji/pull-request/923 - -A new ``remove-group`` command was added, allowing the removal of a group -from a tag. It uses the existing ``groupListRemove`` API call. - - -**Query builds per chunks in prune-signed-builds** - -| PR: https://pagure.io/koji/pull-request/1589 - -For bigger installations querying all builds can cause the hub to run out of memory. -``prune-signed-builds`` now queries these in 50k chunks. - - -**Show inheritance flags in list-tag-inheritance output** - -| PR: https://pagure.io/koji/pull-request/1120 - -While not often used, tag inheritance can be modified with a few different options (e.g. maxdepth). -These options are shown in the ``taginfo`` display, but not the ``list-tag-inheritance`` display. -This change adds basic indicators to the latter. - - -**Return usage information in make-task** - -| PR: https://pagure.io/koji/pull-request/1157 - -``make-task`` now returns usage information if no arguments are provided. - - -**Clarify clone-tag usage** - -| PR: https://pagure.io/koji/pull-request/1623 - -The ``clone-tag`` help text now clarifies that the destination tag will be created -if it does not already exist. - - -**Add option check for list-signed** - -| PR: https://pagure.io/koji/pull-request/1631 - -The ``list-signed`` command will now fail if no options are provided. - - - -Library Changes ---------------- - -**Consolidate config reading style** - -| PR: https://pagure.io/koji/pull-request/1296 - -Changes have been made to make configuration handling more consistent. - -With this new implementation: - -* ``read_config_files`` is extended with a strict option and directory support -* ``ConfigParser`` is used for all invokings except kojixmlrpc and ``kojid`` -* ``RawConfigParser`` is used for ``kojid`` - - -**list_archive_files handles multi-type builds** - -| PR: https://pagure.io/koji/pull-request/1508 - -If ``list_archive_files`` is provided a build with multiple archive types it now correctly -handles them instead of failing. - - -**Disallow archive imports that don't match build type** - -| PR: https://pagure.io/koji/pull-request/1627 -| PR: https://pagure.io/koji/pull-request/1633 - -The ``importArchive`` call now refuses to proceed if the build does not have the given type. - - -**Add listCG RPC** - -| PR: https://pagure.io/koji/pull-request/1160 - -``listCGs`` has been added to list new content generator records. - -The purpose of this change is to make it easier for administrators to determine what -content generators are present and what user accounts have access to those. - - -**Add method to cancel CG reservations** - -| PR: https://pagure.io/koji/pull-request/1662 - -The new ``CGRefundBuild`` call allows CGs to cancel build reservations, such as in the case -of a failing build. - - -**Allow ClientSession objects to get cleaned up by the garbage collector** - -| PR: https://pagure.io/koji/pull-request/1653 - -This change ensures ``koji.ClientSession`` objects are destroyed once their requests are complete. - - -**Add missing package list check** - -| PR: https://pagure.io/koji/pull-request/1244 -| PR: https://pagure.io/koji/pull-request/1702 - -The ``host.tagBuild`` method was missing a check to ensure the package was actually listed in the -destination tag. This should now be checked as expected. - - -**Increase buildReferences SQL performance** - -| PR: https://pagure.io/koji/pull-request/1675 - -The performance for ``build_references`` has been improved. - - -**ensuredir does not duplicate directories** - -| PR: https://pagure.io/koji/pull-request/1197 - -``koji.ensuredir`` no longer creates duplicate directories if provided a path ending in a -forward slash. - - -**Warn users if buildroot uses yum instead of dnf** - -| PR: https://pagure.io/koji/pull-request/1595 - -This change sets the mock config ``dnf_warning`` to True for buildroots using yum. - - -**Tag permission can be used for tagBuildBypass and untagBuildBypass** - -| PR: https://pagure.io/koji/pull-request/1685 - -The ``tag`` permission can now be used in place of admin to call ``tagBuildBypass`` -and ``untagBuildBypass``. Admin is still required to use the ``--force`` option. - - -**Rework update of reserved builds** - -| PR: https://pagure.io/koji/pull-request/1621 - -This change reworks and simplifies the code that updates reserved build entries for cg imports. -It removes redundancy with checks in ``prep_build`` and avoids duplicate ``*BuildStateChange`` -callbacks. - - -**Use correct top limit for randint** - -| PR: https://pagure.io/koji/pull-request/1612 - -The top limit for ``randint`` has been set to 255 from 256 to prevent ``generate_token`` from -creating unneccesarily long tokens. - - -**Add strict option to getRPMFile** - -| PR: https://pagure.io/koji/pull-request/1068 - -``getRPMFile`` now has a ``strict`` option, failing when the RPM or filename does not exist. - - -**Stricter groupListRemove** - -| PR: https://pagure.io/koji/pull-request/1173 -| PR: https://pagure.io/koji/pull-request/1678 - -``groupListRemove`` now returns an error if the provided group does not exist for the tag. - - -**Clarified docs for build.extra.source** - -| PR: https://pagure.io/koji/pull-request/1677 - -The usage for ``build.extra.source`` has now been clarified in the ``getBuild`` call. - - -**Use bytes for debug string** - -| PR: https://pagure.io/koji/pull-request/1657 - -This change fixes debug output for Python 3. - - -**Removed host.repoAddRPM call** - -| PR: https://pagure.io/koji/pull-request/1680 - -The ``host.repoAddRPM`` call has been removed because it was unused and broken. - - - -Web UI Changes --------------- - -**Made difference between Builds and Tags sections more clear** - -| PR: https://pagure.io/koji/pull-request/1676 - -The search page results for packages now has a clearer delineation between builds and tags. - - - -Builder Changes ---------------- - -**Use preferred arch when builder provides multiple** - -| PR: https://pagure.io/koji/pull-request/1684 - -When using ExclusiveArch for noarch builds the build task will now use the -arch specified instead of randomly picking from the arches the builder provides. - -This change adds a ``preferred_arch`` parameter to ``find_arch``. - - -**Log insufficient disk space location** - -| PR: https://pagure.io/koji/pull-request/1523 - -When ``kojid`` fails due to insufficient disk space, the directory which needs more -disk space is now included as part of the log message. - - -**Allow builder to attempt krb if gssapi is available** - -| PR: https://pagure.io/koji/pull-request/1613 - -``kojid`` will now use ``requests_kebreros`` for kerberos authentication when available. - - -**Add support for new mock exit codes** - -| PR: https://pagure.io/koji/pull-request/1682 - -``kojid`` now expects mock exit code 10 for failed builds (previously 1). - - -**Fix kickstart uploads for Python 3** - -| PR: https://pagure.io/koji/pull-request/1618 - -This change fixes the file handling of kickstarts for Python 3. - - - -System Changes --------------- - -**Package ownership changes do not trigger repo regens** - -| PR: https://pagure.io/koji/pull-request/1473 -| PR: https://pagure.io/koji/pull-request/1643 - -Changing tag or package owners no longer cause repo regeneration. A new -``tag_package_owners`` table has been added for this purpose. - - -**Support multiple realms by kerberos auth** - -| PR: https://pagure.io/koji/pull-request/1648 -| PR: https://pagure.io/koji/pull-request/1696 -| PR: https://pagure.io/koji/pull-request/1701 - -This change adds a new table ``user_krb_principals`` which tracks a list of ``krb_principals`` -for each user instead of the previous one-to-one mapping. In addition: - -* all APIs related to user or krb principals are changed -* ``userinfo`` of ``getUser`` will contain a new list ``krb_principals`` - * ``krb_principals`` will contain all available principals if ``krb_princs=True`` -* there is a new hub option ``AllowedKrbRealms`` to indicate which realms are allowed -* there is a new client option ``krb_server_realm`` to allow krbV login to set server realm - * Previously same as client principal realm before, supported by all clients -* ``QueryProcessor`` has a new queryOpt ``group``, which is used to generate ``GROUP BY`` section - * By default, this feature is disabled by arg ``enable_group=False`` - - -**Added cronjob for sessions table maintenance** - -| PR: https://pagure.io/koji/pull-request/1492 - -The sessions table is now periodically cleaned up via script (handled by cron by default). -Without this the sessions table can grow large enough to affect Koji performance. - - -**Added basic email template for koji-gc** - -| PR: https://pagure.io/koji/pull-request/1430 - -The email message koji-gc uses has been moved to ``/etc/koji-gc/email.tpl`` for -easier customization. - - -**Add all permissions to database** - -| PR: https://pagure.io/koji/pull-request/1681 - -Permissions previously missing from schema have been added, including ``dist-repo``, ``host``, -``image-import``, ``sign``, ``tag``, and ``target``. - - -**Add new CoreOS artifact types** - -| PR: https://pagure.io/koji/pull-request/1616 - -This change adds the new CoreOS artifact types ``iso-compressed``, ``vhd-compressed``, -``vhdx-compressed``, and ``vmdk-compressed`` to the database. - - -**Enforce unique content generator names in database** - -| PR: https://pagure.io/koji/pull-request/1159 - -Set a uniqueness constraint on the content generator name in the database. -Prior to this change, we were only enforcing this in the hub application layer. -Configure this in postgres for safety. - - -**Fix typo preventing VM builds** - -| PR: https://pagure.io/koji/pull-request/1666 - -This change fixes the options passed to ``verifyChecksum`` which was preventing VM builds. - - -**Fix verifyChecksum for non-output files** - -| PR: https://pagure.io/koji/pull-request/1670 - -``verifyChecksum`` now accepts files under the build requires path as well as the output path. -Other paths can be added as needed. - - -**Set f30+ python-devel default** - -| PR: https://pagure.io/koji/pull-request/1683 - -When installed on a Fedora 30+ host with Python 2 support, Koji will now require -``python2-devel`` instead of ``python-devel``. - - -**Handle sys.exc_clear for Python 3** - -| PR: https://pagure.io/koji/pull-request/1642 - -The method ``sys.exc_clear`` does not exist in Python 3, so it has been escaped for those instances. - - -**Remove deprecated koji.util.relpath** - -| PR: https://pagure.io/koji/pull-request/1458 - -``koji.util.relpath`` was deprecated in 1.16 and has been removed from 1.19. - - -**Remove deprecated BuildRoot.uploadDir** - -| PR: https://pagure.io/koji/pull-request/1511 - -``BuildRoot.uploadDir`` was deprecated in 1.18 and has been removed from 1.19. - - -**Remove deprecated koji_cli.lib_unique_path** - -| PR: https://pagure.io/koji/pull-request/1512 - -``koji_cli.lib_unique_path`` was deprecated in 1.17 and has been removed from 1.19. - - -**Deprecation of sha1_constructor and md5_constructor** - -| PR: https://pagure.io/koji/pull-request/1490 - -``sha1_constructor`` and ``md5_constructor`` have been deprecated in favor of ``hashlib``.