PR#9284: Update critpath.py for f32 branched - releng

releng

#9284 Update critpath.py for f32 branched

Merged 4 years ago by mohanboddu. Opened 4 years ago by mohanboddu.

mohanboddu/releng f32-critpath-branching into master

Move update-critpath to critpath.py

Mohan Boddu • 4 years ago

14e6e61

Critpath changes

Mohan Boddu • 4 years ago

a658a43

scripts/critpath.py

file modified

+2 -2

		`@@ -1,4 +1,4 @@`
		`- #!/usr/bin/python -tt`
		`+ #!/usr/bin/python3`
		`#`
		`# Copyright (C) 2013 Red Hat Inc,`
		`# SPDX-License-Identifier: GPL-2.0+`
		`@@ -48,7 +48,7 @@`
		`updatepath[r] = 'updates/%s/$basearch/' % r`

		`# Branched Fedora goes here`
		`- branched = '31'`
		`+ branched = '32'`
		`releasepath['branched'] = 'development/%s/Everything/$basearch/os' % branched`
		`updatepath['branched'] = ''`

scripts/update-critpath.py ~~scripts/update-critpath~~

file renamed

+20 -7

		`@@ -34,7 +34,6 @@`

		`# Here, 'fedora' is the name of the production instance, configured in`
		`# /etc/pdc.d/. See the PDC SOP for more information on authenticating.`
		`- pdc = pdc_client.PDCClient('fedora')`

		`# This is an on-disk cache to avoid hitting PDC over and over.`
		`cache_filename = "/var/tmp/pdc-branch-cache.dbm"`
		`@@ -64,6 +63,9 @@`
		`parser.add_argument(`
		`'txtlist',`
		`help="Filename containing list of packages to critpath")`
		`+ parser.add_argument(`
		`+ 'token',`
		`+ help="PDC Token")`

		`return parser`

		`@@ -77,7 +79,7 @@`
		`:return: a set of package names with PDC branch IDs`
		`:rtype: set`
		`'''`
		`- if not isinstance(branch, basestring):`
		`+ if not isinstance(branch, str):`
		`raise TypeError('branch must be a string')`

		`LOG.debug("Retrieving existing critpath list from PDC.")`
		`@@ -100,14 +102,20 @@`
		`@cache.cache_on_arguments()`
		`def _get_pdc_branch_id(**args):`
		`results = list(pdc.get_paged(endpoint, **args))`
		`- assert len(results) == 1`
		`- return results[0]`
		`+ #critpath.py generates subpackages as well.`
		`+ #As there wont be any pdc entries for subpackages`
		`+ #we need to skip them`
		`+ if(len(results) == 1):`
		`+ return results[0]`
		`+ else:`
		`+ return None`

		`for i, component in enumerate(global_components):`
		`args = {'name': branch, 'type': 'rpm', 'global_component': component}`
		`result = _get_pdc_branch_id(**args)`

		`- pkgs.add("{id}:{global_component}".format(**result))`
		`+ if result:`
		`+ pkgs.add("{id}:{global_component}".format(**result))`
		`LOG.debug("Done finding PDC branch IDs for supplied critpath list.")`
		`return pkgs`

		`@@ -143,10 +151,12 @@`
		`# Parse the commandline`
		`try:`
		`arg = parser.parse_args()`
		`- except argparse.ArgumentTypeError, err:`
		`+ except argparse.ArgumentTypeError as err:`
		`print("\nError: {0}".format(err))`
		`return 2`

		`+ token = arg.token`
		`+`
		`if arg.debug:`
		`LOG.setLevel(logging.DEBUG)`

		`@@ -154,6 +164,9 @@`
		`global pdc`
		`LOG.info("Using staging environment")`
		`pdc = pdc_client.PDCClient('staging')`
		`+ else:`
		`+ pdc = pdc_client.PDCClient('fedora', token)`
		`+`


		`LOG.debug("Using pdc branch cache %r" % cache_filename)`
		`@@ -182,7 +195,7 @@`
		`except KeyboardInterrupt:`
		`LOG.info("\nInterrupted by user.")`
		`return_code = 1`
		`- except Exception, err:`
		`+ except Exception as err:`
		`LOG.exception("Caught generic error")`
		`return_code = 2`

mohanboddu commented 4 years ago

Signed-off-by: Mohan Boddu mboddu@bhujji.com

churchyard commented on line 12 of scripts/critpath.py 4 years ago

why is this getting lower?

kevin commented 4 years ago

@churchyard because 32 hasn't been released yet, so it has no release/32 directory, it's under development still.

We are debugging this in #fedora-releng if you want to drop by. :)

churchyard commented 4 years ago

Unfortunately, this is too late for me. This has been a long day :(

kevin commented 4 years ago

No problem. @mohanboddu got the lists and updated rawhide and branched and will try and do the others tonight.

I got:

Traceback (most recent call last):
File "./critpath.py", line 341, in <module>
f.write(packagename + b'\n')
TypeError: can only concatenate str (not "bytes") to str

at the end for some reason. ;(

churchyard commented 4 years ago

A wild guess:

        if args.nvr:
            critpath.update([nvr(p).encode('utf8') for p in pkgs])
        elif args.srpm:
            critpath.update([get_source(p.sourcerpm) for p in pkgs])
        else:
            critpath.update([p.name.encode('utf8') for p in pkgs])

This seem to put bytes, str, bytes into the set.
Ideally, this should all work just with str, but as a short term fix, I suggest trying to add encode call to the elif branch.

(I might be completely wrong.)

mohanboddu commented 4 years ago

FYI, WIP, please dont merge it.

rebased onto 54a47c5ca65e3887cbb0e89652e4de8f293275f1

4 years ago

mohanboddu commented 4 years ago

Okay, its ready to merge now.

kevin commented 4 years ago

+1 to merge if this is the working version...

rebased onto a658a43

4 years ago

Pull-Request has been merged by mohanboddu