PR#7930: Orphans: Use pagination to query pagure - releng

releng

#7930 Orphans: Use pagination to query pagure

Merged 5 years ago by humaton. Opened 5 years ago by till.

till/releng pagure_pagination into master

find_unblocked_orphans: Reduce pagure cache expiration

Till Maas • 5 years ago

9f35940

Orphans: Use pagination to query pagure

Till Maas • 5 years ago

e7985ab

scripts/find_unblocked_orphans.py

file modified

+16 -3

		`@@ -39,11 +39,12 @@`

		`cache = dogpile.cache.make_region().configure(`
		`'dogpile.cache.dbm',`
		`- expiration_time=187000,`
		`+ expiration_time=86400,`
		`arguments=dict(`
		`filename=os.path.expanduser('~/.cache/dist-git-orphans-cache.dbm')),`
		`)`
		`PAGURE_URL = 'https://src.fedoraproject.org'`
		`+ PAGURE_MAX_ENTRIES_PER_PAGE = 100`


		`EPEL6_RELEASE = dict(`
		`@@ -221,13 +222,25 @@`

		`@cache.cache_on_arguments()`
		`def orphan_packages(namespace='rpms'):`
		`+ pkgs, pages = get_pagure_orphans(namespace)`
		`+ for page in range(2, pages + 1):`
		`+ new_pkgs, _ = get_pagure_orphans(namespace, page)`
		`+ pkgs.update(new_pkgs)`
		`+ return pkgs`
		`+`
		`+`
		`+ @cache.cache_on_arguments()`
		`+ def get_pagure_orphans(namespace, page=1):`
		`url = PAGURE_URL + '/api/0/projects'`
		`- params = dict(owner=ORPHAN_UID, namespace=namespace)`
		`+ params = dict(owner=ORPHAN_UID, namespace=namespace,`
		`+ page=page,`
		`+ per_page=PAGURE_MAX_ENTRIES_PER_PAGE)`
		`response = requests.get(url, params=params)`
		`if not bool(response):`
		`raise IOError("%r gave %r" % (response.request.url, response))`
		`pkgs = response.json()['projects']`
		`- return dict([(p['name'], p) for p in pkgs])`
		`+ pages = response.json()['pagination']['pages']`
		`+ return dict([(p['name'], p) for p in pkgs]), pages`


		`def unblocked_packages(packages, tagID=RAWHIDE_RELEASE["tag"]):`

till commented 5 years ago

Signed-off-by: Till Maas opensource@till.name

churchyard commented 5 years ago

Thank you.

churchyard commented 5 years ago

I still only get this:

Contacting pagure for list of orphans...done
Getting builds from koji...done
Setting up dependency checker...done
Calculating dependencies...Got info for adonthell on master, todo: 0
Getting packages depending on: adonthell
to_check (1): ['adonthell']
to_check (1): ['wastesedge']
Waiting for (co)maintainer information...Got info for wastesedge on master, todo: 0
done
done
fedretire --orphan-dependent adonthell --branch master -- wastesedge
fedretire --orphan --branch master -- adonthell
Addresses (0): 
The following packages are orphaned and will be retired when they
are orphaned for six weeks, unless someone adopts them. If you know for sure
that the package should be retired, please do so now with a proper reason:
https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life

Note: If you received this mail directly you (co)maintain one of the affected
packages or a package that depends on one. Please adopt the affected package or
retire your depending package to avoid broken dependencies, otherwise your
package will be retired when the affected package gets retired.

adonthell orphan 31 weeks ago


The following packages require above mentioned packages:
Depending on: adonthell (1), status change: 2018-04-24 (31 weeks ago)
    wastesedge (maintained by: orphan)
        wastesedge-0.3.7-5.fc29.noarch requires adonthell = 0.3.6-10.fc29
        wastesedge-0.3.7-5.fc29.src requires adonthell = 0.3.6-10.fc29

Affected (co)maintainers

Orphans (1): adonthell


Orphans (dependend on) (1): adonthell


Orphans (rawhide) for at least 6 weeks (dependend on) (1): adonthell


Orphans  (rawhide)(not depended on) (0):


Orphans (rawhide) for at least 6 weeks (not dependend on) (0):


Depending packages (rawhide) (1): wastesedge


Packages depending on packages orphaned (rawhide) for more than 6
    weeks (1): wastesedge

-- 
The script creating this output is run and developed by Fedora
Release Engineering. Please report issues at its pagure instance:
https://pagure.io/releng/
The sources of this script can be found at:
https://pagure.io/releng/blob/master/f/scripts/find_unblocked_orphans.py

till commented 5 years ago

There is a cache for pagure replies that can be cleared manually with rm ~/.cache/dist-git-orphans-cache.dbm* - it also expires after some time - it seems that the support for pagure changed it to more than two days. I guess one day would be enough.

1 new commit added