#52 Find docker images to rebuild which contains a RPM signed in Brew
Merged 8 years ago by cqi. Opened 8 years ago by cqi.
cqi/freshmaker find-images-internal  into  master

file modified
+3
@@ -25,8 +25,11 @@ 

      ]

  

      HANDLERS = [

+         'freshmaker.handlers.brew:BrewSignRPMHanlder',

      ]

  

+     KOJI_PROFILE = 'brew'

+ 

      # LightBlue server URL, e.g. http://localhost/

      LIGHTBLUE_SERVER_URL = ''  # replace with default server url

      LIGHTBLUE_VERIFY_SSL = True

file modified
+1 -1
@@ -104,7 +104,7 @@ 

          :return: True if all builds in advisory are signed.

          :rtype: bool

          """

-         builds_per_product = self._errata_rest_get(

+         builds_per_product = self._errata_http_get(

              "advisory/%s/builds.json" % str(errata_id))

  

          # Store NVRs of all builds in advisory to nvrs set.

@@ -0,0 +1,22 @@ 

+ # -*- coding: utf-8 -*-

+ # Copyright (c) 2017  Red Hat, Inc.

+ #

+ # Permission is hereby granted, free of charge, to any person obtaining a copy

+ # of this software and associated documentation files (the "Software"), to deal

+ # in the Software without restriction, including without limitation the rights

+ # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

+ # copies of the Software, and to permit persons to whom the Software is

+ # furnished to do so, subject to the following conditions:

+ #

+ # The above copyright notice and this permission notice shall be included in all

+ # copies or substantial portions of the Software.

+ #

+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

+ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

+ # SOFTWARE.

+ 

+ from .sign_rpm import BrewSignRPMHanlder  # noqa

@@ -0,0 +1,119 @@ 

+ # -*- coding: utf-8 -*-

+ # Copyright (c) 2016  Red Hat, Inc.

+ #

+ # Permission is hereby granted, free of charge, to any person obtaining a copy

+ # of this software and associated documentation files (the "Software"), to deal

+ # in the Software without restriction, including without limitation the rights

+ # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

+ # copies of the Software, and to permit persons to whom the Software is

+ # furnished to do so, subject to the following conditions:

+ #

+ # The above copyright notice and this permission notice shall be included in all

+ # copies or substantial portions of the Software.

+ #

+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

+ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

+ # SOFTWARE.

+ #

+ # Written by Chenxiong Qi <cqi@redhat.com>

+ 

+ from itertools import chain

+ 

+ from freshmaker import conf

+ from freshmaker import log

+ from freshmaker.events import BrewSignRPMEvent

+ from freshmaker.handlers import BaseHandler

+ from freshmaker.kojiservice import koji_service

+ from freshmaker.lightblue import LightBlue

+ from freshmaker.pulp import Pulp

+ from freshmaker.errata import Errata

+ 

+ 

+ class BrewSignRPMHanlder(BaseHandler):

+     """Rebuild docker images when a RPM is signed in Brew"""

+ 

+     name = 'BrewSignRPMHandler'

+ 

+     def can_handle(self, event):

+         return isinstance(event, BrewSignRPMEvent)

+ 

+     def handle(self, event):

+         """Rebuild docker images which contains this signed RPM

+ 

+         Before rebuilding docker images, freshmaker has to find which docker

+         images includes the signed RPM. As of writing this feature, this

+         information is stored in LightBlue, and need to use content_sets to

+         find out those images.

+ 

+         There are several external services taking part in the process of

+         rebuilding docker images.

+ 

+         * Errata Tool: get which advisories contains the signed RPM, and Pulp

+           repositories the signed RPM will end up eventually when shipped.

+         * Pulp: query content set with repositories got from Errata Tool.

+         * LightBlue: this is where to query docker images that contains RPMs

+           from those content sets.

+         """

+ 

+         images = self._find_images_to_rebuild(event)

+ 

+         if not images:

+             log.info('Not find docker images to rebuild.')

+             return []

+ 

+         log.info('Found docker images to rebuild: %s', images)

+ 

+         # TODO: build yum repo to contain that signed RPM and start to rebuild

+ 

+         return []

+ 

+     def _find_images_to_rebuild(self, event):

+         # When get a signed RPM, first step is to find out advisories

+         # containing that RPM and has to ensure all builds are signed.

+         errata = Errata(conf.errata_tool_server_url)

+         advisories = errata.advisories_from_event(event)

+ 

+         if not all((errata.builds_signed(advisory.errata_id)

+                     for advisory in advisories)):

+             log.info('Not all builds in %s are signed. Do not rebuild any '

+                      'docker image until signed.', advisories)

+             return []

+ 

+         # Use the advisories to find out Pulp repository IDs from Errata Tool

+         # and furthermore get content_sets from Pulp where signed RPM will end

+         # up eventually when advisories are shipped.

+         pulp_repo_ids = list(set(chain(

+             *[errata.get_pulp_repository_ids(advisory.errata_id)

+               for advisory in advisories]

+         )))

+ 

+         pulp = Pulp(server_url=conf.pulp_server_url,

+                     username=conf.pulp_username,

+                     password=conf.pulp_password)

+         content_sets = pulp.get_content_set_by_repo_ids(pulp_repo_ids)

+ 

+         log.info('RPM will end up within content sets %s', content_sets)

+ 

+         # Query images from LightBlue by signed RPM's srpm name and found

+         # content sets

+         lb = LightBlue(server_url=conf.lightblue_server_url,

+                        cert=conf.lightblue_certificate,

+                        private_key=conf.lightblue_private_key)

+ 

+         srpm_name = self._find_build_srpm_name(event.nvr)

+         return lb.find_images_with_package_from_content_set(srpm_name,

+                                                             content_sets)

+ 

+     def _find_build_srpm_name(self, build_nvr):

+         """Find srpm name from a build"""

+         with koji_service(conf.koji_profile, log) as session:

+             rpm_infos = session.get_build_rpms(build_nvr, arches='src')

+             if not rpm_infos:

+                 raise ValueError(

+                     'Build {} does not have a SRPM, although this should not '

+                     'happen in practice.'.format(build_nvr))

+             return rpm_infos[0]['name']

file modified
+3 -2
@@ -95,9 +95,10 @@ 

  

          return task_id

  

-     def get_build_rpms(self, build_nvr):

+     def get_build_rpms(self, build_nvr, arches=None):

          build_info = self.session.getBuild(build_nvr)

-         return self.session.listRPMs(buildID=build_info['id'])

+         return self.session.listRPMs(buildID=build_info['id'],

+                                      arches=arches)

  

  

  @contextlib.contextmanager

file modified
+3 -3
@@ -127,7 +127,7 @@ 

              will be used on each entity.

          """

          self.server_url = server_url.rstrip('/')

-         self.api_root = '{}/rest/data'.format(server_url)

+         self.api_root = '{}/rest/data'.format(self.server_url)

          if verify_ssl is None:

              self.verify_ssl = True

          else:
@@ -375,7 +375,6 @@ 

                      break

  

              for rpm in image["parsed_data"]["rpm_manifest"]:

-                 print(rpm)

                  if rpm["srpm_name"] == srpm_name:

                      srpm_nevra = rpm['srpm_nevra']

                      break
@@ -385,5 +384,6 @@ 

              reponame = reponame.replace("/plain/Dockerfile", "")

              commits.append({"repository": reponame,

                              "commit": commit,

-                             "srpm_nevra": srpm_nevra})

+                             "srpm_nevra": srpm_nevra,

+                             "brew": image["brew"]})

          return commits

file modified
+24 -6
@@ -21,6 +21,7 @@ 

  #

  # Written by Chenxiong Qi <cqi@redhat.com>

  

+ import json

  import requests

  

  
@@ -33,12 +34,29 @@ 

          self.server_url = server_url

          self.rest_api_root = '{0}/pulp/api/v2/'.format(self.server_url.rstrip('/'))

  

-     def _rest_get(self, endpoint):

-         r = requests.get('{0}{1}'.format(self.rest_api_root, endpoint.lstrip('/')),

-                          auth=(self.username, self.password))

+     def _rest_post(self, endpoint, post_data):

+         r = requests.post(

+             '{0}{1}'.format(self.rest_api_root, endpoint.lstrip('/')),

+             post_data,

+             auth=(self.username, self.password))

          r.raise_for_status()

          return r.json()

  

-     def get_content_set_by_repo_id(self, repo_id):

-         data = self._rest_get('repositories/{0}/'.format(repo_id))

-         return data['notes']['content_set']

+     def get_content_set_by_repo_ids(self, repo_ids):

+         """Get content_sets by repository IDs

+ 

+         :param list repo_ids: list of repository IDs.

+         :return: list of names of content_sets.

+         :rtype: list

+         """

+         query_data = {

+             'criteria': {

+                 'filters': {

+                     'id': {'$in': repo_ids},

+                 },

+                 'fields': ['notes.content_set'],

+             }

+         }

+         repos = self._rest_post('repositories/search/', json.dumps(query_data))

+         return [repo['notes']['content_set'] for repo in repos

+                 if 'content_set' in repo['notes']]

@@ -0,0 +1,80 @@ 

+ # -*- coding: utf-8 -*-

+ # Copyright (c) 2017  Red Hat, Inc.

+ #

+ # Permission is hereby granted, free of charge, to any person obtaining a copy

+ # of this software and associated documentation files (the "Software"), to deal

+ # in the Software without restriction, including without limitation the rights

+ # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

+ # copies of the Software, and to permit persons to whom the Software is

+ # furnished to do so, subject to the following conditions:

+ #

+ # The above copyright notice and this permission notice shall be included in

+ # all copies or substantial portions of the Software.

+ #

+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

+ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

+ # SOFTWARE.

+ #

+ # Written by Chenxiong Qi <cqi@redhat.com>

+ 

+ import six

+ import pytest

+ import unittest

+ 

+ from mock import patch

+ 

+ from freshmaker.handlers.brew.sign_rpm import BrewSignRPMHanlder

+ 

+ 

+ @pytest.mark.skipif(six.PY3, reason='koji does not work in Python 3')

+ class TestFindBuildSrpmName(unittest.TestCase):

+     """Test BrewSignRPMHanlder._find_build_srpm_name"""

+ 

+     @patch('koji.ClientSession')

+     def test_find_srpm_name(self, ClientSession):

+         session = ClientSession.return_value

+         session.getBuild.return_value = {

+             'build_id': 439408,

+             'id': 439408,

+             'name': 'bind-dyndb-ldap',

+             'nvr': 'bind-dyndb-ldap-2.3-8.el6',

+         }

+         session.listRPMs.return_value = [{

+             'arch': 'src',

+             'name': 'bind-dyndb-ldap',

+             'nvr': 'bind-dyndb-ldap-2.3-8.el6',

+         }]

+ 

+         handler = BrewSignRPMHanlder()

+         srpm_name = handler._find_build_srpm_name('bind-dyndb-ldap-2.3-8.el6')

+ 

+         session.getBuild.assert_called_once_with('bind-dyndb-ldap-2.3-8.el6')

+         session.listRPMs.assert_called_once_with(buildID=439408, arches='src')

+         self.assertEqual('bind-dyndb-ldap', srpm_name)

+ 

+     @patch('koji.ClientSession')

+     def test_error_if_no_srpm_in_build(self, ClientSession):

+         session = ClientSession.return_value

+         session.getBuild.return_value = {

+             'build_id': 439408,

+             'id': 439408,

+             'name': 'bind-dyndb-ldap',

+             'nvr': 'bind-dyndb-ldap-2.3-8.el6',

+         }

+         session.listRPMs.return_value = []

+ 

+         handler = BrewSignRPMHanlder()

+ 

+         self.assertRaisesRegexp(

+             ValueError,

+             'Build bind-dyndb-ldap-2.3-8.el6 does not have a SRPM',

+             handler._find_build_srpm_name,

+             'bind-dyndb-ldap-2.3-8.el6',

+         )

+ 

+         session.getBuild.assert_called_once_with('bind-dyndb-ldap-2.3-8.el6')

+         session.listRPMs.assert_called_once_with(buildID=439408, arches='src')

file modified
+20 -13
@@ -28,12 +28,14 @@ 

  from freshmaker.events import BrewSignRPMEvent, GitRPMSpecChangeEvent

  

  

- class MockedErrataRESTAPI(object):

+ class MockedErrataAPI(object):

      """

      Class mocking methods accessing Errata API in Errata class.

      """

-     def __init__(self, errata_rest_get):

+     def __init__(self, errata_rest_get, errata_http_get=None):

          errata_rest_get.side_effect = (self.errata_rest_get)

+         if errata_http_get:

+             errata_http_get.side_effect = self.errata_http_get

  

          self.builds_json = {

              "PRODUCT1": [
@@ -67,12 +69,14 @@ 

              "rpms_signed": True}

  

      def errata_rest_get(self, endpoint):

-         if endpoint.endswith("builds.json"):

-             return self.builds_json

-         elif endpoint.find("build/") != -1:

+         if endpoint.find("build/") != -1:

              nvr = endpoint.split("/")[-1]

              return self.builds[nvr]

  

+     def errata_http_get(self, endpoint):

+         if endpoint.endswith("builds.json"):

+             return self.builds_json

+ 

  

  class TestErrata(unittest.TestCase):

      def setUp(self):
@@ -80,7 +84,7 @@ 

  

      @patch.object(Errata, "_errata_rest_get")

      def test_advisories_from_event(self, errata_rest_get):

-         MockedErrataRESTAPI(errata_rest_get)

+         MockedErrataAPI(errata_rest_get)

          event = BrewSignRPMEvent("msgid", "libntirpc-1.4.3-4.el7rhgs")

          advisories = self.errata.advisories_from_event(event)

          self.assertEqual(len(advisories), 1)
@@ -88,7 +92,7 @@ 

  

      @patch.object(Errata, "_errata_rest_get")

      def test_advisories_from_event_missing_all_errata(self, errata_rest_get):

-         mocked_errata = MockedErrataRESTAPI(errata_rest_get)

+         mocked_errata = MockedErrataAPI(errata_rest_get)

          del mocked_errata.builds["libntirpc-1.4.3-4.el7rhgs"]["all_errata"]

  

          event = BrewSignRPMEvent("msgid", "libntirpc-1.4.3-4.el7rhgs")
@@ -101,19 +105,22 @@ 

              self.errata.advisories_from_event(event)

  

      @patch.object(Errata, "_errata_rest_get")

-     def test_builds_signed_all_signed(self, errata_rest_get):

-         MockedErrataRESTAPI(errata_rest_get)

+     @patch.object(Errata, "_errata_http_get")

+     def test_builds_signed_all_signed(self, errata_http_get, errata_rest_get):

+         MockedErrataAPI(errata_rest_get, errata_http_get)

          self.assertTrue(self.errata.builds_signed(28484))

  

      @patch.object(Errata, "_errata_rest_get")

-     def test_builds_signed_some_unsigned(self, errata_rest_get):

-         mocked_errata = MockedErrataRESTAPI(errata_rest_get)

+     @patch.object(Errata, "_errata_http_get")

+     def test_builds_signed_some_unsigned(self, errata_http_get, errata_rest_get):

+         mocked_errata = MockedErrataAPI(errata_rest_get, errata_http_get)

          mocked_errata.builds["libntirpc-1.4.3-4.el7rhgs"]["rpms_signed"] = False

          self.assertFalse(self.errata.builds_signed(28484))

  

      @patch.object(Errata, "_errata_rest_get")

-     def test_builds_signed_missing_data(self, errata_rest_get):

-         mocked_errata = MockedErrataRESTAPI(errata_rest_get)

+     @patch.object(Errata, "_errata_http_get")

+     def test_builds_signed_missing_data(self, errata_http_get, errata_rest_get):

+         mocked_errata = MockedErrataAPI(errata_rest_get, errata_http_get)

          mocked_errata.builds["libntirpc-1.4.3-4.el7rhgs"] = {}

          self.assertFalse(self.errata.builds_signed(28484))

  

file modified
+12 -2
@@ -519,12 +519,22 @@ 

                               {

                                   "repository": "rpms/repo-1",

                                   "commit": "commit_hash1",

-                                  "srpm_nevra": "openssl-0:1.2.3-1.src"

+                                  "srpm_nevra": "openssl-0:1.2.3-1.src",

+                                  "brew": {

+                                      "completion_date": u"20170421T04:27:51.000-0400",

+                                      "build": "package-name-1-4-12.10",

+                                      "package": "package-name-1"

+                                  }

                               },

                               {

                                   "repository": "ns/repo-2",

                                   "commit": "commit_hash2",

-                                  "srpm_nevra": "openssl-1:1.2.3-1.src"

+                                  "srpm_nevra": "openssl-1:1.2.3-1.src",

+                                  "brew": {

+                                      "completion_date": u"20170421T04:27:51.000-0400",

+                                      "build": "package-name-2-4-12.10",

+                                      "package": "package-name-2"

+                                  }

                               }

                           ])

  

file modified
+113 -23
@@ -21,6 +21,7 @@ 

  #

  # Written by Chenxiong Qi <cqi@redhat.com>

  

+ import json

  import unittest

  

  from mock import patch
@@ -36,34 +37,123 @@ 

          self.username = 'qa'

          self.password = 'qa'

  

-     @patch('freshmaker.pulp.requests.get')

-     def test_query_content_set_by_repo_id(self, get):

-         get.return_value.json.return_value = {

-             'id': 'rhel-7-desktop-debug-rpms__7Client__x86_64',

-             '_ns': 'repos',

-             'content_unit_counts': {

-                 'erratum': 1420,

-                 'rpm': 4773

+     @patch('freshmaker.pulp.requests.post')

+     def test_query_content_set_by_repo_ids(self, post):

+         post.return_value.json.return_value = [

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-workstation-rpms__7Workstation__x86_64/',

+                 '_id':

+                 {

+                     '$oid': '53853a247bc9f61b85909cfe'

+                 },

+                 'id': 'rhel-7-workstation-rpms__7Workstation__x86_64',

+                 'notes':

+                 {

+                     'content_set': 'rhel-7-workstation-rpms',

+                 },

              },

-             '_id': {

-                 '$oid': '5384ee687bc9f619942a8b47'

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-hpc-node-rpms__7ComputeNode__x86_64/',

+                 '_id': {

+                     '$oid': '5384ee7c7bc9f619942a8f89',

+                 },

+                 'id': 'rhel-7-hpc-node-rpms__7ComputeNode__x86_64',

+                 'notes': {

+                     'content_set': 'rhel-7-hpc-node-rpms'

+                 },

              },

-             'last_unit_added': '2017-06-02T15:01:06Z',

-             'notes': {

-                 'arch': 'x86_64',

-                 'content_set': 'rhel-7-desktop-debug-rpms',

-                 'platform_full_version': '7',

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-desktop-rpms__7Client__x86_64/',

+                 '_id': {

+                     '$oid': '5384ee6a7bc9f619942a8bca',

+                 },

+                 'id': 'rhel-7-desktop-rpms__7Client__x86_64',

+                 'notes': {

+                     'content_set': 'rhel-7-desktop-rpms',

+                 }

+             }

+         ]

+ 

+         pulp = Pulp(self.server_url, username=self.username, password=self.password)

+         repo_ids = [

+             'rhel-7-hpc-node-rpms__7ComputeNode__x86_64',

+             'rhel-7-workstation-rpms__7Workstation__x86_64',

+             'rhel-7-desktop-rpms__7Client__x86_64',

+         ]

+         content_sets = pulp.get_content_set_by_repo_ids(repo_ids)

+ 

+         post.assert_called_once_with(

+             '{}pulp/api/v2/repositories/search/'.format(self.server_url),

+             json.dumps({

+                 'criteria': {

+                     'filters': {

+                         'id': {'$in': repo_ids},

+                     },

+                     'fields': ['notes.content_set'],

+                 }

+             }),

+             auth=(self.username, self.password))

+ 

+         self.assertEqual(

+             ['rhel-7-workstation-rpms',

+              'rhel-7-hpc-node-rpms',

+              'rhel-7-desktop-rpms'],

+             content_sets)

+ 

+     @patch('freshmaker.pulp.requests.post')

+     def test_get_content_sets_by_ignoring_nonexisting_ones(self, post):

+         post.return_value.json.return_value = [

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-workstation-rpms__7Workstation__x86_64/',

+                 '_id':

+                 {

+                     '$oid': '53853a247bc9f61b85909cfe'

+                 },

+                 'id': 'rhel-7-workstation-rpms__7Workstation__x86_64',

+                 'notes':

+                 {

+                     'content_set': 'rhel-7-workstation-rpms',

+                 },

+             },

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-hpc-node-rpms__7ComputeNode__x86_64/',

+                 '_id': {

+                     '$oid': '5384ee7c7bc9f619942a8f89',

+                 },

+                 'id': 'rhel-7-hpc-node-rpms__7ComputeNode__x86_64',

+                 'notes': {},

              },

-             'display_name': 'rhel-7-desktop-debug-rpms__7Client__x86_64',

-             'description': None,

-         }

+             {

+                 '_href': '/pulp/api/v2/repositories/rhel-7-desktop-rpms__7Client__x86_64/',

+                 '_id': {

+                     '$oid': '5384ee6a7bc9f619942a8bca',

+                 },

+                 'id': 'rhel-7-desktop-rpms__7Client__x86_64',

+                 'notes': {

+                     'content_set': 'rhel-7-desktop-rpms',

+                 }

+             }

+         ]

  

          pulp = Pulp(self.server_url, username=self.username, password=self.password)

-         repo_id = 'rhel-7-desktop-debug-rpms__7Client__x86_64'

-         content_set = pulp.get_content_set_by_repo_id(repo_id)

+         repo_ids = [

+             'rhel-7-hpc-node-rpms__7ComputeNode__x86_64',

+             'rhel-7-workstation-rpms__7Workstation__x86_64',

+             'rhel-7-desktop-rpms__7Client__x86_64',

+         ]

+         content_sets = pulp.get_content_set_by_repo_ids(repo_ids)

  

-         get.assert_called_once_with(

-             '{}pulp/api/v2/repositories/{}/'.format(self.server_url, repo_id),

+         post.assert_called_once_with(

+             '{}pulp/api/v2/repositories/search/'.format(self.server_url),

+             json.dumps({

+                 'criteria': {

+                     'filters': {

+                         'id': {'$in': repo_ids},

+                     },

+                     'fields': ['notes.content_set'],

+                 }

+             }),

              auth=(self.username, self.password))

  

-         self.assertEqual('rhel-7-desktop-debug-rpms', content_set)

+         self.assertEqual(['rhel-7-workstation-rpms', 'rhel-7-desktop-rpms'],

+                          content_sets)

This patch is for finding docker images to rebuild by using existing
Errata Tool, Pulp and LightBlue APIs.

Signed-off-by: Chenxiong Qi cqi@redhat.com

Demo: https://fedorapeople.org/groups/factory2/sprint-032/cqi-freshmaker-find-docker-images-with-brew-and-lightblue.ogv

Nothing wrong about this - but are you planning to use this later? it doesn't seem to be used now

Perhaps not necessary ATM, but we might want to do a logical "OR" query on Pulp instead to speed things up (i.e. one query instead of one query for pulp repo id)

Generally looks good, will wait for final version for deeper review.

Need to get build NVR in order to get build target and extract branch name from that target.

Good point. I'll learn Pulp API deeply. Or, any hint at this moment? :)

It's basically just:
"filters": {
"$or": [
{"notes.content_set": "content-set-1"},
{"notes.content_set": "content-set-2"}
]
}

rebased

8 years ago

Major updates:

  • Fixed Errata.build_signed. advisory/errata_id/builds.json is a legacy API, should use _errata_http_get.
  • Rewrite Pulp.get_content_set_by_repo_ids to use Pulp search API to query content sets with multiple repository IDs in one request.
  • Update tests for above changes,
  • Add missing KOJI_PROFILE and handler in internal specific configuration.

Please review.

I am fairly certain not all returned will always have content_set in notes - so you could end up with KeyError here

What Stano wrote, otherwise +1 from me.

rebased

8 years ago

Fixed issue mentioned by @sochotni

Pull-Request has been merged by cqi

8 years ago