Learn more about these different git repos.
Other Git URLs
I hit an error composing the F29-updates-testing repo in staging:
https://kojipkgs.stg.fedoraproject.org/compose/updates/Fedora-29-updates-testing-20190506.4/logs/global/traceback.global.log
Traceback (most recent call last): File "/usr/bin/pungi-koji", line 482, in <module> main() File "/usr/bin/pungi-koji", line 261, in main run_compose(compose, create_latest_link=create_latest_link, latest_link_status=latest_link_status) File "/usr/bin/pungi-koji", line 381, in run_compose essentials_phase.stop() File "/usr/lib/python3.7/site-packages/pungi/phases/weaver.py", line 53, in stop self.pool.stop() File "/usr/lib/python3.7/site-packages/kobo/threads.py", line 120, in stop six.reraise(exc_info[0], exc_info[1], exc_info[2]) File "/usr/lib/python3.7/site-packages/six.py", line 693, in reraise raise value File "/usr/lib/python3.7/site-packages/kobo/threads.py", line 67, in run self.process(item, num) File "/usr/lib/python3.7/site-packages/pungi/phases/weaver.py", line 69, in process phase.start() File "/usr/lib/python3.7/site-packages/pungi/phases/base.py", line 64, in start self.run() File "/usr/lib/python3.7/site-packages/pungi/phases/gather/__init__.py", line 111, in run manifest=self.manifest) File "/usr/lib/python3.7/site-packages/pungi/phases/gather/link.py", line 122, in link_files pool.stop() File "/usr/lib/python3.7/site-packages/kobo/threads.py", line 120, in stop six.reraise(exc_info[0], exc_info[1], exc_info[2]) File "/usr/lib/python3.7/site-packages/six.py", line 693, in reraise raise value File "/usr/lib/python3.7/site-packages/kobo/threads.py", line 67, in run self.process(item, num) File "/usr/lib/python3.7/site-packages/pungi/linker.py", line 63, in process self.pool.linker.link(src, dst, link_type=self.pool.link_type) File "/usr/lib/python3.7/site-packages/pungi/linker.py", line 218, in link raise OSError(errno.EEXIST, "File exists") FileExistsError: [Errno 17] File exists Frame link in /usr/lib/python3.7/site-packages/pungi/linker.py at line 218 <CODE> 211 def link(self, src, dst, link_type="hardlink-or-copy"): 212 """Link directories recursively.""" 213 if os.path.isfile(src) or os.path.islink(src): 214 self._link_file(src, dst, link_type) 215 return 216 217 if os.path.isfile(dst): --> 218 raise OSError(errno.EEXIST, "File exists") 219 220 if not self.test: 221 if not os.path.exists(dst): 222 makedirs(dst) 223 shutil.copystat(src, dst) 224 </CODE> <LOCALS> dst = '/mnt/koji/compose/updates/Fedora-29-updates-testing-20190506.4/compose/Everything/source/tree/Packages/r/rdopkg-0.49.0-1.fc29.src.rpm' link_type = 'hardlink-or-copy' self = <pungi.linker.Linker object at 0x7f1e9fe28e80> self._inode_map = {(48, 27531575): '/mnt/koji/compose/updates/Fedora-29-updates-testing-20190506.4/compose/Everything/source/tree/Packages/d/delve-1.2.0-1.fc29.src.rpm', (48, 38070092): '/mnt/koji/compose/updates/Fedor self._logger = None self.always_copy = [] self.test = False src = '/mnt/koji/vol/prod/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm' </LOCALS> Frame process in /usr/lib/python3.7/site-packages/pungi/linker.py at line 63 <CODE> 56 src, dst = item 57 58 if (num % 100 == 0) or (num == self.pool.queue_total): 59 self.pool.log_debug("Linked %s out of %s packages" % (num, self.pool.queue_total)) 60 61 directory = os.path.dirname(dst) 62 makedirs(directory) --> 63 self.pool.linker.link(src, dst, link_type=self.pool.link_type) 64 65 66 class Linker(kobo.log.LoggingBase): 67 def __init__(self, always_copy=None, test=False, logger=None): 68 kobo.log.LoggingBase.__init__(self, logger=logger) 69 self.always_copy = always_copy or [] </CODE> <LOCALS> directory = '/mnt/koji/compose/updates/Fedora-29-updates-testing-20190506.4/compose/Everything/source/tree/Packages/r' dst = '/mnt/koji/compose/updates/Fedora-29-updates-testing-20190506.4/compose/Everything/source/tree/Packages/r/rdopkg-0.49.0-1.fc29.src.rpm' item = <ERROR WHILE CONVERTING VALUE TO STRING> num = 818 self = <LinkerThread(Thread-102, stopped 139769471821568)> self._args = <ERROR WHILE CONVERTING VALUE TO STRING> self._daemonic = False self._ident = 139769471821568 self._initialized = True self._is_stopped = True self._kwargs = {} self._name = 'Thread-102' self._started = <threading.Event object at 0x7f1e9fe28a90> self._stderr = <_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> self._target = None self._tstate_lock = None self.daemon = False self.failed = True self.get_timeout = 1 self.ident = 139769471821568 self.kill = True self.name = 'Thread-102' self.pool = <pungi.linker.LinkerPool object at 0x7f1e9fe28e10> self.running = False src = '/mnt/koji/vol/prod/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm' </LOCALS>
I think this is not a bug in Pungi, but instead a rather unfortunate race condition.
The error happened when packages from Koji volume were hardlinked into the compose for Everything.x86_64. The problematic file was vol/prod/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm.
vol/prod/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm
In order for this exception to be raised, multiple conditions have to be met: the target path must exist and not be a symlink (which was true here since the file was hardlinked for another architecture already).
Another condition is that the source file must not be a file (because the same function can work with entire directories). In this case I think the source file did not exist.
At least checking from stage composer machine I'm getting this:
$ ls /mnt/fedora_koji_prod/koji/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm ls: cannot access '/mnt/fedora_koji_prod/koji/packages/rdopkg/0.49.0/1.fc29/data/signed/429476b4/src/rdopkg-0.49.0-1.fc29.src.rpm': No such file or directory
There is a sigcache for this key. I think what happened here was that the signed copy was garbage collected while the compose was running.
On Tue, 2019-05-07 at 07:20 +0000, Lubom=C3=ADr Sedl=C3=A1=C5=99 wrote:
I think what happened here was that the signed copy was garbage collected while the compose was running.
Oh interesting. Thanks for the analysis!
Metadata Update from @lsedlar: - Issue close_status updated to: Invalid - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.