PR#3903: Upload mock config - koji

koji

#3903 Upload mock config

Closed 7 months ago by tkopecek. Opened 8 months ago by tkopecek.

tkopecek/koji issue3875 into master

Upload mock config

Tomas Kopecek • 7 months ago

4a1031a

builder/kojid

file modified

+10 -3

		`@@ -270,11 +270,14 @@`
		`self.internal_dev_setup = internal_dev_setup`
		`self._writeMockConfig()`

		`+ def getConfigPath(self):`
		`+ configdir = '/etc/mock/koji'`
		`+ return "%s/%s.cfg" % (configdir, self.name)`
		`+`
		`def _writeMockConfig(self):`
		`# mock config`
		`- configdir = '/etc/mock/koji'`
		`- configfile = "%s/%s.cfg" % (configdir, self.name)`
		`self.mockcfg = "koji/%s" % self.name`
		`+ configfile = self.getConfigPath()`

		`opts = {}`
		`for k in ('repoid', 'tag_name'):`
		`@@ -1478,13 +1481,16 @@`

		`# extract results`
		`resultdir = broot.resultdir()`
		`+ # copy mock config there to be gathered as log file`
		`+ shutil.copy(broot.getConfigPath(), resultdir)`
		`+ self.logger.debug("Copied mock config %s to resultdir." % broot.getConfigPath())`
		`rpm_files = []`
		`srpm_files = []`
		`log_files = []`
		`unexpected = []`
		`for f in os.listdir(resultdir):`
		`# files here should have one of two extensions: .log and .rpm`
		`- if f[-4:] == ".log":`
		`+ if f[-4:] in (".cfg", ".log"):`
		`log_files.append(f)`
		`elif f[-8:] == ".src.rpm":`
		`srpm_files.append(f)`
		`@@ -1519,6 +1525,7 @@`

		`# upload files to storage server`
		`uploadpath = broot.getUploadPath()`
		`+ self.uploadFile(broot.getConfigPath())`
		`for f in rpm_files:`
		`self.uploadFile("%s/%s" % (resultdir, f))`
		`self.logger.debug("keep srpm %i %s %s" % (self.id, keep_srpm, opts))`

tkopecek commented 8 months ago

It is now just uploaded to buildArch task.
1) Are there other task where it would make sense?
2) I'm not sure if it is worth to propagate it to regular build logs (so it can outlive workdir).

rebased onto bc3bc352bc8c57ed6922d4a8ae24a58ae125f016

8 months ago

mikem commented 8 months ago

The motivating issue refers to logging the macros for auditing purposes, so I think we need to associate these with the build. That is unfortunately more difficult. We likely want to do it for every build that involves a mock config (or at least leave ourselves room to extend it as such).

Slight overlap with part of this koji-devel discussion about the koji-flatpak plugin

Edited 8 months ago by mikem

mikem commented 8 months ago

I looked at this a bit the other month and it looks like for rpm and a few other key build types (maven, wrapperRPM) we could just copy the log into resultdir in order to have the handler code include it in the build.

This would not cover createLiveCD/Media/Appliance which only look for fixed filenames, but those are of less concern.

I note that while buildSRPMFromSCM/rebuildSRPM both return a list of log files, the calling task does nothing with them.

rebased onto 4a1031a

7 months ago

mikem commented 7 months ago

In my above comment, I'd envisioned having the BuildRoot class do this copy (possibly adding a .log to the name) so that the existing code for incremental log uploads would do the uploading and the calling code would grab it like other logs.

Here, we're modifying the buildArch handler instead. This won't help other tasks that use mock. I guess we could modify each mock-based task. However, if we do go though route then the copy might not be necessary, since that code can choose to upload and report whatever files it wants.

I think my inclination would be to build this into the BuildRoot class in a way that all consumers can benefit from it.

tkopecek commented 7 months ago

Yep, I tried that but haven't found the nice place for that. In the moment of writing the config, directories doesn't exist yet, of course. Later it would meand that any call to mock() could do that as BuildRoot can't know when the caller will try to upload it. A bit ugly solution is to check existence of that file in the end of each mock() call and copy it if it doesn't exist. I'm not that happy about it, but probably better than the current PR. Do you've any better idea?