This has been reported in [[ https://bugzilla.redhat.com/show_bug.cgi?id=1456869 | Bugzilla ]]. Prioritizing, because not only sudo support seems not to be working in certain locales, but the whole dnf support seems to be broken in F26.
When I run taskotron locally (using tox from [1]), it shouts on me I need rpm-python: Version-Release number of selected component (if applicable): libtaskotron-0.4.20-1.fc26.noarch [libtaskotron] 14:33:46 INFO Execution started at: 2017-05-30 14:33:46 UTC [libtaskotron] 14:33:46 DEBUG Using libtaskotron 0.4.20 [libtaskotron] 14:33:46 DEBUG Parsed arguments: Namespace(arch='noarch', debug=False, item='python-twine-1.8.1-3.fc26', jobid='-1', libvirt=False, local=False, no_destroy=False, override=[], patch=None, ssh=None, ssh_privkey=None, task='runtask.yml', type='koji_build', uuid='20170530_143346_886492') [libtaskotron] 14:33:46 DEBUG Using config file: /etc/taskotron/taskotron.yaml [libtaskotron] 14:33:46 DEBUG Using config profile: development [libtaskotron] 14:33:46 INFO Task artifacts will be saved in: /var/lib/taskotron/artifacts/20170530_143346_886492 [libtaskotron] 14:33:46 INFO Checking installed state of 1 packages... [libtaskotron] 14:33:47 WARNING No DNF cache available, metadata will be need to be downloaded. If your DNF cache doesn't persist across task executions (e.g. on a disposable VM), consider creating the cache in your base system image to speed up execution. [libtaskotron] 14:33:47 CRITICAL Traceback (most recent call last): File "/usr/bin/runtask", line 11, in <module> load_entry_point('libtaskotron==0.4.20', 'console_scripts', 'runtask')() File "/usr/lib/python2.7/site-packages/libtaskotron/main.py", line 156, in main overlord.start() File "/usr/lib/python2.7/site-packages/libtaskotron/overlord.py", line 96, in start runner.execute() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 60, in execute self._prepare_task() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 77, in _prepare_task % " ".join([pipes.quote(rpm) for rpm in rpms])) TaskotronError: Some packages are not installed. Please run "dnf install rpm-python" to install all required packages. This is of course not true, as no such package exists on Fedora 26, but I have the thing that provides it: $ LANG=C.utf8 sudo dnf install rpm-python Package python2-rpm-4.13.0.1-4.fc26.x86_64 is already installed, skipping. Dependencies resolved. Nothing to do. Complete! The package is specified in the runtask section: environment: rpm: - rpm-python But when I change it to: environment: rpm: - python2-rpm It sill tracebacks: TaskotronError: Some packages are not installed. Please run "dnf install python2-rpm" to install all required packages. $ rpm -q python2-rpm python2-rpm-4.13.0.1-4.fc26.x86_64 I believe rpm_utils.is_installed() is broken. Also I was curious that sudo dnf install rpm-python actually fixed the problem for a while and I realized it's the sudo that fixes it. I.e. when I run sudo echo and enter the password, the code in rpm_utils.is_installed() goes into the os_utils.has_sudo() condition and uses sudo and that fixes the problem. I've examined the problem further. Running with --debug: [libtaskotron] 14:49:31 INFO Execution started at: 2017-05-30 14:49:31 UTC [libtaskotron] 14:49:31 DEBUG Using libtaskotron 0.4.20 [libtaskotron] 14:49:31 DEBUG Parsed arguments: Namespace(arch='noarch', debug=True, item='python-twine-1.8.1-3.fc26', jobid='-1', libvirt=False, local=False, no_destroy=False, override=[], patch=None, ssh=None, ssh_privkey=None, task='runtask.yml', type='koji_build', uuid='20170530_144931_095987') [libtaskotron] 14:49:31 DEBUG Using config file: /etc/taskotron/taskotron.yaml [libtaskotron] 14:49:31 DEBUG Using config profile: development [libtaskotron:logger.py:182] 2017-05-30 14:49:31 DEBUG Stream logging enabled with level: DEBUG [libtaskotron:overlord.py:38] 2017-05-30 14:49:31 DEBUG Found task: dist.python-versions [libtaskotron:overlord.py:90] 2017-05-30 14:49:31 INFO Task artifacts will be saved in: /var/lib/taskotron/artifacts/20170530_144931_095987 [libtaskotron:overlord.py:74] 2017-05-30 14:49:31 DEBUG Execution mode: local [libtaskotron:rpm_utils.py:184] 2017-05-30 14:49:31 INFO Checking installed state of 1 packages... [libtaskotron:os_utils.py:37] 2017-05-30 14:49:31 DEBUG Deciding whether we have a password-less sudo access. Running: sudo --validate --non-interactive Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128) Logged from file os_utils.py, line 43 [libtaskotron:rpm_utils.py:229] 2017-05-30 14:49:31 DEBUG Deciding whether DNF cache is available. Running: dnf --cacheonly repolist Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 19: ordinal not in range(128) Logged from file rpm_utils.py, line 234 [libtaskotron:rpm_utils.py:191] 2017-05-30 14:49:31 WARNING No DNF cache available, metadata will be need to be downloaded. If your DNF cache doesn't persist across task executions (e.g. on a disposable VM), consider creating the cache in your base system image to speed up execution. [libtaskotron:os_utils.py:37] 2017-05-30 14:49:31 DEBUG Deciding whether we have a password-less sudo access. Running: sudo --validate --non-interactive Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128) Logged from file os_utils.py, line 43 [libtaskotron:rpm_utils.py:198] 2017-05-30 14:49:31 DEBUG Running: dnf --assumeno --disableplugin=noroot install rpm-python Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 14: ordinal not in range(128) Logged from file rpm_utils.py, line 203 [libtaskotron:logger.py:88] 2017-05-30 14:49:32 CRITICAL Traceback (most recent call last): File "/usr/bin/runtask", line 11, in <module> load_entry_point('libtaskotron==0.4.20', 'console_scripts', 'runtask')() File "/usr/lib/python2.7/site-packages/libtaskotron/main.py", line 156, in main overlord.start() File "/usr/lib/python2.7/site-packages/libtaskotron/overlord.py", line 96, in start runner.execute() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 60, in execute self._prepare_task() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 77, in _prepare_task % " ".join([pipes.quote(rpm) for rpm in rpms])) TaskotronError: Some packages are not installed. Please run "dnf install rpm-python" to install all required packages. Now I have a pretty decent idea what's going on here. $ sudo --validate --non-interactive sudo: je vyžadováno heslo UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128) Char 11 is ž. $ \dnf --assumeno --disableplugin=noroot install rpm-python Chyba: Tento příkaz musí být spuštěn pod uživatelem root. UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 14: ordinal not in range(128) Char 14 is ř. I believe I've identified several problems here: 1) You are using Python 2 and thus you handle unicode in "I don't care" mode - be careful or use Python 3 instead to fix that :) 2) You are calling external commands and doing stuff with the output but you do not set locale to English 3) dnf --disableplugin=noroot does not work on Fedora 26 [1] https://github.com/fedora-python/taskotron-python-versions
When I run taskotron locally (using tox from [1]), it shouts on me I need rpm-python:
Version-Release number of selected component (if applicable): libtaskotron-0.4.20-1.fc26.noarch
[libtaskotron] 14:33:46 INFO Execution started at: 2017-05-30 14:33:46 UTC [libtaskotron] 14:33:46 DEBUG Using libtaskotron 0.4.20 [libtaskotron] 14:33:46 DEBUG Parsed arguments: Namespace(arch='noarch', debug=False, item='python-twine-1.8.1-3.fc26', jobid='-1', libvirt=False, local=False, no_destroy=False, override=[], patch=None, ssh=None, ssh_privkey=None, task='runtask.yml', type='koji_build', uuid='20170530_143346_886492') [libtaskotron] 14:33:46 DEBUG Using config file: /etc/taskotron/taskotron.yaml [libtaskotron] 14:33:46 DEBUG Using config profile: development [libtaskotron] 14:33:46 INFO Task artifacts will be saved in: /var/lib/taskotron/artifacts/20170530_143346_886492 [libtaskotron] 14:33:46 INFO Checking installed state of 1 packages... [libtaskotron] 14:33:47 WARNING No DNF cache available, metadata will be need to be downloaded. If your DNF cache doesn't persist across task executions (e.g. on a disposable VM), consider creating the cache in your base system image to speed up execution. [libtaskotron] 14:33:47 CRITICAL Traceback (most recent call last): File "/usr/bin/runtask", line 11, in <module> load_entry_point('libtaskotron==0.4.20', 'console_scripts', 'runtask')() File "/usr/lib/python2.7/site-packages/libtaskotron/main.py", line 156, in main overlord.start() File "/usr/lib/python2.7/site-packages/libtaskotron/overlord.py", line 96, in start runner.execute() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 60, in execute self._prepare_task() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 77, in _prepare_task % " ".join([pipes.quote(rpm) for rpm in rpms])) TaskotronError: Some packages are not installed. Please run "dnf install rpm-python" to install all required packages.
This is of course not true, as no such package exists on Fedora 26, but I have the thing that provides it:
$ LANG=C.utf8 sudo dnf install rpm-python Package python2-rpm-4.13.0.1-4.fc26.x86_64 is already installed, skipping. Dependencies resolved. Nothing to do. Complete!
The package is specified in the runtask section:
environment: rpm: - rpm-python
But when I change it to:
environment: rpm: - python2-rpm
It sill tracebacks:
TaskotronError: Some packages are not installed. Please run "dnf install python2-rpm" to install all required packages.
$ rpm -q python2-rpm python2-rpm-4.13.0.1-4.fc26.x86_64
I believe rpm_utils.is_installed() is broken. Also I was curious that sudo dnf install rpm-python actually fixed the problem for a while and I realized it's the sudo that fixes it. I.e. when I run sudo echo and enter the password, the code in rpm_utils.is_installed() goes into the os_utils.has_sudo() condition and uses sudo and that fixes the problem.
rpm_utils.is_installed()
sudo dnf install rpm-python
sudo echo
os_utils.has_sudo()
I've examined the problem further. Running with --debug:
[libtaskotron] 14:49:31 INFO Execution started at: 2017-05-30 14:49:31 UTC [libtaskotron] 14:49:31 DEBUG Using libtaskotron 0.4.20 [libtaskotron] 14:49:31 DEBUG Parsed arguments: Namespace(arch='noarch', debug=True, item='python-twine-1.8.1-3.fc26', jobid='-1', libvirt=False, local=False, no_destroy=False, override=[], patch=None, ssh=None, ssh_privkey=None, task='runtask.yml', type='koji_build', uuid='20170530_144931_095987') [libtaskotron] 14:49:31 DEBUG Using config file: /etc/taskotron/taskotron.yaml [libtaskotron] 14:49:31 DEBUG Using config profile: development [libtaskotron:logger.py:182] 2017-05-30 14:49:31 DEBUG Stream logging enabled with level: DEBUG [libtaskotron:overlord.py:38] 2017-05-30 14:49:31 DEBUG Found task: dist.python-versions [libtaskotron:overlord.py:90] 2017-05-30 14:49:31 INFO Task artifacts will be saved in: /var/lib/taskotron/artifacts/20170530_144931_095987 [libtaskotron:overlord.py:74] 2017-05-30 14:49:31 DEBUG Execution mode: local [libtaskotron:rpm_utils.py:184] 2017-05-30 14:49:31 INFO Checking installed state of 1 packages... [libtaskotron:os_utils.py:37] 2017-05-30 14:49:31 DEBUG Deciding whether we have a password-less sudo access. Running: sudo --validate --non-interactive Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128) Logged from file os_utils.py, line 43 [libtaskotron:rpm_utils.py:229] 2017-05-30 14:49:31 DEBUG Deciding whether DNF cache is available. Running: dnf --cacheonly repolist Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 19: ordinal not in range(128) Logged from file rpm_utils.py, line 234 [libtaskotron:rpm_utils.py:191] 2017-05-30 14:49:31 WARNING No DNF cache available, metadata will be need to be downloaded. If your DNF cache doesn't persist across task executions (e.g. on a disposable VM), consider creating the cache in your base system image to speed up execution. [libtaskotron:os_utils.py:37] 2017-05-30 14:49:31 DEBUG Deciding whether we have a password-less sudo access. Running: sudo --validate --non-interactive Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128) Logged from file os_utils.py, line 43 [libtaskotron:rpm_utils.py:198] 2017-05-30 14:49:31 DEBUG Running: dnf --assumeno --disableplugin=noroot install rpm-python Traceback (most recent call last): File "/usr/lib64/python2.7/logging/__init__.py", line 861, in emit msg = self.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 734, in format return fmt.format(record) File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format record.message = record.getMessage() File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 14: ordinal not in range(128) Logged from file rpm_utils.py, line 203 [libtaskotron:logger.py:88] 2017-05-30 14:49:32 CRITICAL Traceback (most recent call last): File "/usr/bin/runtask", line 11, in <module> load_entry_point('libtaskotron==0.4.20', 'console_scripts', 'runtask')() File "/usr/lib/python2.7/site-packages/libtaskotron/main.py", line 156, in main overlord.start() File "/usr/lib/python2.7/site-packages/libtaskotron/overlord.py", line 96, in start runner.execute() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 60, in execute self._prepare_task() File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 77, in _prepare_task % " ".join([pipes.quote(rpm) for rpm in rpms])) TaskotronError: Some packages are not installed. Please run "dnf install rpm-python" to install all required packages.
Now I have a pretty decent idea what's going on here.
$ sudo --validate --non-interactive sudo: je vyžadováno heslo
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 11: ordinal not in range(128)
Char 11 is ž.
$ \dnf --assumeno --disableplugin=noroot install rpm-python Chyba: Tento příkaz musí být spuštěn pod uživatelem root.
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 14: ordinal not in range(128)
Char 14 is ř.
I believe I've identified several problems here:
1) You are using Python 2 and thus you handle unicode in "I don't care" mode - be careful or use Python 3 instead to fix that :) 2) You are calling external commands and doing stuff with the output but you do not set locale to English 3) dnf --disableplugin=noroot does not work on Fedora 26
[1] https://github.com/fedora-python/taskotron-python-versions
This ticket had assigned some Differential requests: D1216
So, I looked into the brave new world in #939. As it seems, we'll likely remove sudo support, since it requires to run tasks as root. So that's first part "resolved". The current WIP code in D1195 also removes all code that uses dnf install and uses ansible for that instead. If that turns out of be the way we handle things, that's the second half of this "resolved", we'll see.
sudo
dnf install
I'm going to block this on #939. Once that is implemented, we can look at if we still need to fix something here, or if all of our relevant existing code had been thrown away.
This has been partially "fixed" in D1216 - figuring out whether packages are installed now works under a non-root account, as long as you use plain package names/file paths as requirements. That should cover the major issues until we replace this whole code.
Metadata Update from @kparal: - Issue marked as depending on: #376
This is now outdated since we moved to STI-based tasks.
Metadata Update from @kparal: - Issue close_status updated to: Invalid
Login to comment on this ticket.