koji-atomic-reactor-plugin

THIS IS A PROOF OF CONCEPT NOT MEANT FOR PRODUCTION USE

koji-atomic-reactor-plugin is a plugin to koji that enables builders to execute docker layered image builds using atomic-reactor.

The builders need to have both docker and atomic-reactor installed with the docker service enabled.

Place the atomicreactor.py in /usr/lib/koji-builder-plugins/, edit the koji builder config file, /etc/kojid/kojid.conf and enable it:

# A space-separated list of plugins to enable
plugins = atomicreactor

Restart the Koji Builder service, and your plugin will be enabled.

Now you can run the task with the koji client:

$ koji make-task atomicreactor \
    buildroot \
    "https://github.com/maxamillion/atomic-reactor-dockerfile-test.git" \
    test-image

# You can optionally pass in a git commit, branch, or tag
$ koji make-task atomicreactor \
    buildroot \
    "https://github.com/maxamillion/atomic-reactor-dockerfile-test.git" \
    test-image \
    my-git-topic-branch

Builder Pre-Reqs

On the builder, you need to have atomic-reactor installed which can be found for the time being in a COPR here: maxamillion-atomic-reactor

Currently you also need to alter python-requests version (EL7-requests and Fedora-requests)

# For RHEL7 or derivative distros such as CentOS7
$ yum -y install atomic-reactor
$ yum -y update python-requests

# For Fedora
$ dnf -y install atomic-reactor
$ dnf -y downgrade python-requests

Also need to have docker installed and running on the build system. Please note recomendations about docker-storage for production systems.

# For RHEL7 or derivative distros such as CentOS7
#
## NOTE: RHEL7 systems need the optional channel enabled in
##       subscription-manager
$ yum -y install docker
$ systemctl docker start

# For Fedora
$ dnf -y install docker
$ systemctl docker start

Then we need to build the buildroot image for atomic-reactor

$ atomic-reactor create-build-image --reactor-tarball-path /usr/share/atomic-reactor/atomic-reactor.tar.gz /usr/share/atomic-reactor/images/dockerhost-builder buildroot

We are now ready to build!

Known Issues

Currently in Koji version 1.10.x there is a bug in how it handles SSL errors as some of them aren't considered fatal.

If you see a build with the following failure:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1161, in runTask
    response = (handler.run(),)
  File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 158, in run
    return koji.util.call_with_argcheck(self.handler, self.params, self.opts)
  File "/usr/lib/python2.7/site-packages/koji/util.py", line 154, in call_with_argcheck
    return func(*args, **kwargs)
  File "/usr/lib/koji-builder-plugins/atomicreactor.py", line 135, in handler
    self.uploadFile(archive_name)
  File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 262, in uploadFile
    self.session.uploadWrapper(filename, uploadPath, remoteName)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2078, in uploadWrapper
    self.fastUpload(localfile, path, name, callback, blocksize, overwrite)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2021, in fastUpload
    result = self._callMethod('rawUpload', (chunk, ofs, path, name), {'overwrite':overwrite})
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1920, in _callMethod
    return self._sendCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1831, in _sendCall
    return self._sendOneCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1850, in _sendOneCall
    cnx.send(request)
  File "/usr/lib64/python2.7/httplib.py", line 805, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line 111, in sendall
    self.close()
  File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line 82, in close
    self.shutdown()
  File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line 53, in shutdown
    self.__dict__["conn"].shutdown()
Error: []

Then you will need to apply the following patch (not yet merged upstream, awaiting review and testing).

From 117324a8568d137bf2eaecdcafd0ed5bc66fbcfd Mon Sep 17 00:00:00 2001
From: Mathieu Bridon <bochecha@daitauha.fr>
Date: Thu, 23 Jul 2015 10:19:23 +0200
Subject: [PATCH] Better catch SSL errors

Commit 4de27c52de80596d256b059a67d10c7ed5e61238 made Koji to not retry
on SSL errors.

However, it turns out that some SSL errors are transient, and Koji
should still retry for them.

This commit changes that, so that we are more specific about which SSL
errors should be fatal: expired or revoked certificates.

https://bugzilla.redhat.com/show_bug.cgi?id=1207178
---
 koji/__init__.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/koji/__init__.py b/koji/__init__.py
index fadbada..e7a66f2 100644
--- a/koji/__init__.py
+++ b/koji/__init__.py
@@ -1940,11 +1940,15 @@ class ClientSession(object):
                 except (SystemExit, KeyboardInterrupt):
                     #(depending on the python version, these may or may not be subclasses of Exception)
                     raise
-                except OpenSSL.SSL.Error as e:
-                    # There's no point in retrying this
-                    raise
                 except Exception, e:
                     self._close_connection()
+                    if isinstance(e, OpenSSL.SSL.Error):
+                        for arg in e.args:
+                            for _, _, ssl_reason in arg:
+                                if ('certificate revoked' in ssl_reason or
+                                        'certificate expired' in ssl_reason):
+                                    # There's no point in retrying for this
+                                    raise
                     if not self.logged_in:
                         #in the past, non-logged-in sessions did not retry. For compatibility purposes
                         #this behavior is governed by the anon_retry opt.
--
2.4.3