PR#898: Add support for tag/target macros for Mageia - koji

koji

#898 Add support for tag/target macros for Mageia

Merged 3 years ago by mikem. Opened 5 years ago by davidlt.

davidlt/koji tag_macros into master

Add support for tag/target macros for Mageia

David Abdurachmanov • 5 years ago

05b88e0

builder/kojid

file modified

		`@@ -251,6 +251,7 @@`
		`opts['package_manager'] = self.config['extra']['mock.package_manager']`
		`if self.internal_dev_setup is not None:`
		`opts['internal_dev_setup'] = bool(self.internal_dev_setup)`
		`+ opts['tag_macros'] = dict([ _ for _ in self.config['extra'].items() if _[0].startswith('%') ])`
		`output = koji.genMockConfig(self.name, self.br_arch, managed=True, **opts)`

		`#write config`

koji/__init__.py

file modified

		`@@ -1517,6 +1517,9 @@`
		`#TODO - track some of these in the db instead?`
		`}`

		`+ # Load tag specific macros, which can override macros above`
		`+ macros.update(opts.get('tag_macros', []))`
		`+`
		`parts = ["""# Auto-generated by the Koji build system`
		`"""]`
		`if managed:`

davidlt commented 5 years ago

Discussions with Son_Goku (from Mageia) on IRC revealed that they need
per tag/target macro support in Koji. We store tag macros in 'Extra Options'
and macro has to start with '%'.

Example:

% koji taginfo f29
Tag: f29 [11]
Arches: riscv64
Groups:
Required permission: 'admin'
Tag options:
  %mymacro : 'value123'
  %packager : 'Fedora Project RISC-V'
  mock.package_manager : 'dnf'
[..]

Tag macros can be added using koji edit-tag.

View from koji-web: https://ibb.co/bLNgYc

The tag macros are allowed to overwrite macros set in koji.genMockConfig.
For example, %packager is set in koji.genMockConfig by default. The mock
configuration generated by kojid was:

[..]
config_opts['macros']['%_host'] = 'riscv64-koji-linux-gnu'
config_opts['macros']['%mymacro'] = 'value123'
config_opts['macros']['%_host_cpu'] = 'riscv64'
config_opts['macros']['%vendor'] = 'Fedora Project'
config_opts['macros']['%distribution'] = 'Fedora Project'
config_opts['macros']['%_topdir'] = '/builddir/build'
config_opts['macros']['%_rpmfilename'] = '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'
config_opts['macros']['%packager'] = 'Fedora Project RISC-V'
[..]

This two-line change was tested on Fedora RISC-V koji instance.

Signed-off-by: David Abdurachmanov david.abdurachmanov@gmail.com

ngompa commented 5 years ago

:thumbsup:

Thanks for this, I greatly appreciate it!

julian8628 commented 5 years ago

IMO, it's better to move macros to a namespace like mock.marcos.*

And, changing '%_topdir' might bring some problems because currently chroothome is hard code

@mikem @tkopecek Any ideas?

Edited 5 years ago by julian8628

tkopecek commented 5 years ago

+1 for moving to subnamespace

Overriding chroothome is ok, if we got this way. Config (file) value will be simply default and it could be overwritten by tag's extras.

What problems are coming:
- genMockConfig is also used in koji mock-config, values needs to be properly queried there, we already have problems with this (#596)
- build reproducibility - we now simply will not be able to run build with same configuration as before (via --repo-id) as generated config could be different than previous time even with same repo. This is qualitatively different from what we have today - even if other things changes (e.g. tag inheritance), repo is same and we get +- same build, that's no more true if are fiddling with rpm macros. solution could be saving generated mock config with tasks/in db, or some other solution. (discussion in #618)

I would like to see fixing mock config issues as a priority for 1.17 with possible buildroot API, but I'm afraid, that it is not going to be fixed in 1.16

ngompa commented 4 years ago

This just came up in #koji today:

[13:44] <stormi> Pharaoh_Atem: can koji handle something similar to how Mageia separates "core" and "tainted" builds, i.e. set a macro depending on the target?
[13:45] <Pharaoh_Atem> there's a PR for this outstanding: https://pagure.io/koji/pull-request/898
[13:46] <Pharaoh_Atem> it was made by davidlt

@mikem, @tkopecek: Can we get some movement on integrating this functionality for Koji 1.17?

mikem commented 4 years ago

This is likely to interact with a larger feature we are planning, so I am cautious to commit to supporting a particular implementation just yet. I should have some design work on that in the next month or so

ngompa commented 4 years ago

@mikem Any progress on that design work? This feature would also be interesting for Fedora for enabling reproducible builds per tag/target, similar to what openSUSE is doing now.

ngompa commented 4 years ago

Any progress...?

ngompa commented 4 years ago

Anything at all about this...?

ngompa commented 3 years ago

ping?

mikem commented 3 years ago

Some updates here: https://github.com/mikem23/koji-playground/commits/pagure/pr/898