PR#3598: kiwi: upload log for failed tasks - koji

koji

#3598 kiwi: upload log for failed tasks

Merged 2 years ago by tkopecek. Opened 2 years ago by tkopecek.

tkopecek/koji issue3597 into master

fix typos

Tomas Kopecek • 2 years ago

95820af

symlink logs to result dir

Tomas Kopecek • 2 years ago

dfe15d2

kiwi: upload log for failed tasks

Tomas Kopecek • 2 years ago

d5f458b

plugins/builder/kiwi.py

file modified

+13 -8

		`@@ -371,14 +371,19 @@`
		`desc, types = self.prepareDescription(path, name, version, repos, arch)`
		`self.uploadFile(desc)`

		`+ target_dir = '/builddir/result/image'`
		`+ os.symlink( # symlink log to resultdir, so it is incrementally uploaded`
		`+ os.path.join(broot.rootdir(), f'tmp/image-root.{arch}.log'),`
		`+ os.path.join(broot.resultdir(), f'image-root.{arch}.log')`
		`+ )`
		`cmd = ['kiwi-ng']`
		`if self.opts.get('profile'):`
		`cmd.extend(['--profile', self.opts['profile']])`
		`if self.opts.get('type'):`
		`cmd.extend(['--type', self.opts['type']])`
		`- target_dir = '/builddir/result/image'`
		`cmd.extend([`
		`'--kiwi-file', os.path.basename(desc), # global option for image/system commands`
		`+ '--logfile', f'/tmp/image-root.{arch}.log',`
		`'system', 'build',`
		`'--description', os.path.join(os.path.basename(scmsrcdir), base_path),`
		`'--target-dir', target_dir,`
		`@@ -388,8 +393,14 @@`
		`raise koji.GenericError("Kiwi failed")`

		`# rename artifacts accordingly to release`
		`+ os.symlink( # symlink log to resultdir, so it is incrementally uploaded`
		`+ os.path.join(broot.rootdir(), f'/tmp/kiwi-result-bundle.{arch}.log'),`
		`+ os.path.join(broot.resultdir(), f'kiwi-result-bundle.{arch}.log')`
		`+ )`
		`bundle_dir = '/builddir/result/bundle'`
		`- cmd = ['kiwi-ng', 'result', 'bundle',`
		`+ cmd = ['kiwi-ng',`
		`+ '--logfile', f'/tmp/kiwi-result-bundle.{arch}.log',`
		`+ 'result', 'bundle',`
		`'--target-dir', target_dir,`
		`'--bundle-dir', bundle_dir,`
		`'--id', release]`
		`@@ -410,12 +421,6 @@`
		`'files': [],`
		`}`

		`- # TODO: upload detailed log?`
		`- # build/image-root.log`
		`- root_log_path = os.path.join(broot.tmpdir(), target_dir[1:], "build/image-root.log")`
		`- if os.path.exists(root_log_path):`
		`- self.uploadFile(root_log_path, remoteName=f"image-root.{arch}.log")`
		`-`
		`bundle_path = os.path.join(broot.rootdir(), bundle_dir[1:])`
		`for fname in os.listdir(bundle_path):`
		`self.uploadFile(os.path.join(bundle_path, fname), remoteName=fname)`

tkopecek commented 2 years ago

ngompa commented 2 years ago

This doesn't make a streaming log, does it?

tkopecek commented 2 years ago

No, it would need more work (#3600)

ngompa commented 2 years ago

:thumbsup:

rebased onto 8865d34f654cf126839361edc8b58b0e7d980527

2 years ago

tkopecek commented 2 years ago

Updated to streaming log. @mikem can you check added mock option?

ngompa commented 2 years ago

This should also fix it being captured for scratch builds too, right? (Right now, that doesn't happen regardless of success or failure)

Cf. https://cbs.centos.org/koji/taskinfo?taskID=3105781

tkopecek commented 2 years ago

yes, works for scratch builds also

ngompa commented 2 years ago

code lgtm :thumbsup:

ngompa commented 2 years ago

@ignatenkobrain, can you test this in your koji deployment?

tkopecek commented 2 years ago

Wait a bit until @mikem checks on this. I'm not happiest about mock change here.

mikem commented 2 years ago

It seems weird for this option to accept absolute paths. I'd expect any paths specified to be relative to the rootdir.

It looks like the log_timestamps feature will not function for these logs.

Would it make sense to provide additional directories to watch for logs, rather than individual files? If so, I think this might simplify implementation quite a bit.

tkopecek commented 2 years ago

1) makes sense
2) depends of 3
3) It makes sense but doesn't allow renaming of logs. It is similar to ozlog handling, we need to add arch there to not conflict with other subtasks. Maybe just extend ts handling to put -ts.logs to the same directory as original log instead of resultdir?

ngompa commented 2 years ago

You can set the logfile to go somewhere else with the --logfile global option.

The default from kiwi for kiwi system build is that if --logfile isn't passed, it'll write it to that path: https://github.com/OSInside/kiwi/blob/698398cca03f8bf99f26b38a57312b3d430a7022/kiwi/tasks/system_build.py#L142-L145

So you can do kiwi-ng --logfile=/path/to/somewhere/kiwi-system-build-{arch}.log system build to redirect it.

You probably also want a log file for the kiwi-ng result bundle too, like so: kiwi-ng --logfile=/path/to/somewhere/kiwi-result-bundle-{arch}.log result bundle.

ngompa commented 2 years ago

You can also tell kiwi to use an existing Unix domain socket listening for input if you want by using --logsocket instead of --logfile if it helps.

tkopecek commented 2 years ago

Nice, much easier - missed that somehow.

1 new commit added

symlink logs to result dir

2 years ago

tkopecek commented 2 years ago

Anyway, we need to "copy" them outside of chroot. So, I've created symlink from resultdir to chroot/tmp. It works "sufficiently" (few error messages in between symlink creation and target file creation)

ngompa commented on line 27 of plugins/builder/kiwi.py 2 years ago

Missing f prefix here.

ngompa commented on line 28 of plugins/builder/kiwi.py 2 years ago

String quoting inconsistency here.

1 new commit added