#898 Add support for tag/target macros for Mageia
Merged 4 years ago by mikem. Opened 5 years ago by davidlt.
davidlt/koji tag_macros  into  master

file modified
+1
@@ -251,6 +251,7 @@ 

              opts['package_manager'] = self.config['extra']['mock.package_manager']

          if self.internal_dev_setup is not None:

              opts['internal_dev_setup'] = bool(self.internal_dev_setup)

+         opts['tag_macros'] = dict([ _ for _ in self.config['extra'].items() if _[0].startswith('%') ])

          output = koji.genMockConfig(self.name, self.br_arch, managed=True, **opts)

  

          #write config

file modified
+3
@@ -1517,6 +1517,9 @@ 

          #TODO - track some of these in the db instead?

      }

  

+     # Load tag specific macros, which can override macros above

+     macros.update(opts.get('tag_macros', []))

+ 

      parts = ["""# Auto-generated by the Koji build system

  """]

      if managed:

Discussions with Son_Goku (from Mageia) on IRC revealed that they need
per tag/target macro support in Koji. We store tag macros in 'Extra Options'
and macro has to start with '%'.

Example:

% koji taginfo f29
Tag: f29 [11]
Arches: riscv64
Groups:
Required permission: 'admin'
Tag options:
  %mymacro : 'value123'
  %packager : 'Fedora Project RISC-V'
  mock.package_manager : 'dnf'
[..]

Tag macros can be added using koji edit-tag.

View from koji-web: https://ibb.co/bLNgYc

The tag macros are allowed to overwrite macros set in koji.genMockConfig.
For example, %packager is set in koji.genMockConfig by default. The mock
configuration generated by kojid was:

[..]
config_opts['macros']['%_host'] = 'riscv64-koji-linux-gnu'
config_opts['macros']['%mymacro'] = 'value123'
config_opts['macros']['%_host_cpu'] = 'riscv64'
config_opts['macros']['%vendor'] = 'Fedora Project'
config_opts['macros']['%distribution'] = 'Fedora Project'
config_opts['macros']['%_topdir'] = '/builddir/build'
config_opts['macros']['%_rpmfilename'] = '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'
config_opts['macros']['%packager'] = 'Fedora Project RISC-V'
[..]

This two-line change was tested on Fedora RISC-V koji instance.

Signed-off-by: David Abdurachmanov david.abdurachmanov@gmail.com

:thumbsup:

Thanks for this, I greatly appreciate it!

IMO, it's better to move macros to a namespace like mock.marcos.*

And, changing '%_topdir' might bring some problems because currently chroothome is hard code

@mikem @tkopecek Any ideas?

+1 for moving to subnamespace

Overriding chroothome is ok, if we got this way. Config (file) value will be simply default and it could be overwritten by tag's extras.

What problems are coming:
- genMockConfig is also used in koji mock-config, values needs to be properly queried there, we already have problems with this (#596)
- build reproducibility - we now simply will not be able to run build with same configuration as before (via --repo-id) as generated config could be different than previous time even with same repo. This is qualitatively different from what we have today - even if other things changes (e.g. tag inheritance), repo is same and we get +- same build, that's no more true if are fiddling with rpm macros. solution could be saving generated mock config with tasks/in db, or some other solution. (discussion in #618)

I would like to see fixing mock config issues as a priority for 1.17 with possible buildroot API, but I'm afraid, that it is not going to be fixed in 1.16

This just came up in #koji today:

[13:44] <stormi> Pharaoh_Atem: can koji handle something similar to how Mageia separates "core" and "tainted" builds, i.e. set a macro depending on the target?
[13:45] <Pharaoh_Atem> there's a PR for this outstanding: https://pagure.io/koji/pull-request/898
[13:46] <Pharaoh_Atem> it was made by davidlt

@mikem, @tkopecek: Can we get some movement on integrating this functionality for Koji 1.17?

This is likely to interact with a larger feature we are planning, so I am cautious to commit to supporting a particular implementation just yet. I should have some design work on that in the next month or so

@mikem Any progress on that design work? This feature would also be interesting for Fedora for enabling reproducible builds per tag/target, similar to what openSUSE is doing now.

Anything at all about this...?

Some updates here: https://github.com/mikem23/koji-playground/commits/pagure/pr/898

  • some of the macro values set in the code should not be overridden
  • the macro values need a clearer namespace

Using simply "%" for the namespace is too simple. Koji can build more than rpms, but these values are only for rpm builds.

@mikem Those changes look okay to me.

Commit 4414e3a fixes this pull-request

Pull-Request has been merged by mikem

4 years ago