PR#8174: Adding mass branching for modules - releng

releng

#8174 Adding mass branching for modules

Merged 5 years ago by mohanboddu. Opened 5 years ago by mohanboddu.

mohanboddu/releng branching-modules into master

Adding mass branching for modules

Mohan Boddu • 5 years ago

c6a4995

scripts/mass-rebuild-modules.py

file modified

+23 -2

		`@@ -65,10 +65,14 @@`

		`parser = argparse.ArgumentParser()`
		`parser.add_argument('token_file', help='MBS token file for authentication.')`
		`+ #During mass rebuild, we need to check if a module has build time dependencies on module_mass_rebuild_platform`
		`+ #During mass branching, we need to check if a module has run time dependencies on module_mass_branching_platform`
		`+ parser.add_argument('process', help='build or branch, build is used during mass rebuild time, branch is used during branching time')`
		`args = parser.parse_args()`

		`if __name__ == '__main__':`
		`token_file = args.token_file`
		`+ process = args.process`
		`with open(token_file, 'r', encoding='utf-8') as f:`
		`token = f.read().strip()`
		`pdc = 'https://pdc.fedoraproject.org/'`
		`@@ -154,7 +158,18 @@`
		`except:`
		`print("Could not able to read the modulemd file")`
		`continue`
		`- needs_building = mmd.build_depends_on_stream('platform', rebuildid)`
		`+ if process == build:`
		`+ platform = massrebuild['module_mass_rebuild_platform']`
		`+ #check if a module has build time dependency on platform`
		`+ needs_building = mmd.build_depends_on_stream('platform', platform)`
		`+ elif process == branch:`
		`+ platform = massrebuild['module_mass_branching_platform']`
		`+ #check if a module has run time dependency on platform`
		`+ needs_building = mmd.depends_on_stream('platform', platform)`
		`+ else:`
		`+ print("Please select either build or branch for the process type")`
		`+ sys.exit(1)`
		`+`

		`if not needs_building:`
		`print("Not required to build module {} for stream {}".format(name,stream))`
		`@@ -172,7 +187,13 @@`
		`continue`

		`# Empty git commit`
		`- commit = ['git', 'commit', '-s', '--allow-empty', '-m', comment]`
		`+ if process == build:`
		`+ commit = ['git', 'commit', '-s', '--allow-empty', '-m', comment]`
		`+ elif process == branch:`
		`+ commit = ['git', 'commit', '-s', '--allow-empty', '-m', 'Branching {} from rawhide'.format(platform)]`
		`+ else:`
		`+ print("Please select either build or branch for the process type")`
		`+ sys.exit(1)`
		`print('Committing changes for %s' % name)`
		`if runme(commit, 'commit', name, enviro,`
		`cwd=os.path.join(workdir, name)):`

scripts/massrebuildsinfo.py

file modified

		`@@ -26,6 +26,8 @@`
		`"buildtag": 'f30-rebuild', # tag to build from`
		`"epoch": '2019-01-31 10:10:00.000000', # rebuild anything not built after this date`
		`"module_epoch": '2019-02-13T18:30:00Z', # rebuild anything not built after this date for modules`
		`+ "module_mass_rebuild_platform": "f30", # rebuild all modules that has build time dependency on this platform, this is used during mass rebuild time`
		`+ "module_mass_branching_platform": "f31", # rebuild all modules that has run time dependency on this platform, this is used during mass branching time`
		`"targets": ['f30-candidate', 'rawhide', 'f30'], # build targets to check for existing builds to skip rebuild`
		`"target": 'f30-rebuild', # target to build into`
		`"desttag": 'f30', # Tag where fixed builds go`

mohanboddu commented 5 years ago

Signed-off-by: Mohan Boddu mboddu@bhujji.com

psabata commented 5 years ago

This should work for now -- if your goal is to build all modules that would require the given platform stream. It will also work with MSE correctly.

However, in the future:

for mass rebuild, get everything that builds on that platform and build that; that's already there
for branching it's a bit more complex:

Get a list of all built modules and check whether they can run on the given platform; if so, tag them directly.
Get a list of source modules and determine whether they could run on the given platform if rebuilt -- the current code works for that.
Remove the modules that you already tagged from the list and build the rest.