#1043 Add OpenShift test template and Dockerfiles
Merged 6 months ago by mprahl. Opened 6 months ago by csomh.
csomh/fm-orchestrator openshift-test-template  into  master

file added
+69

@@ -0,0 +1,69 @@ 

+ Deploy MBS to OpenShift

+ =======================

+ 

+ ## Build the container image for MBS backend

+ 

+ ```bash

+ $ docker build openshift/backend \

+     --tag mbs-backend:latest \

+     --build-arg mbs_rpm=<MBS_RPM> \

+     --build-arg mbs_messaging_umb_rpm=<MBS_MESSAGING_UMB_RPM>

+ ```

+ 

+ where:

+ * MBS_RPM is a path or URL to the Module Build Service RPM. If not specified,

+   MBS [provided by

+   Fedora](https://apps.fedoraproject.org/packages/module-build-service) will be

+   installed in the image.

+ * MBS_MESSAGING_UMB_RPM is a path or URL to the [UMB Messaging

+   Plugin](https://github.com/release-engineering/mbs-messaging-umb) RPM. If not

+   provided, only `fedmsg` and `in_memory` will be available for messaging in the

+   image.

+ 

+ ## Build the container image for MBS frontend

+ 

+ The frontend container image is built on top of the backend image, which should

+ be available as `mbs-backend:latest`.

+ 

+ ```bash

+ $ docker build openshift/frontend \

+     --tag mbs-frontend:latest

+ ```

+ 

+ ## Deploy MBS

+ 

+ ```bash

+ $ oc process -f openshift/mbs-test-template.yaml \

+     -p TEST_ID=123 \

+     -p MBS_BACKEND_IMAGE=<MBS_BACKEND_IMAGE> \

+     -p MBS_FRONTEND_IMAGE=<MBS_FRONTEND_IMAGE> \

+     -p MESSAGING_CERT=$(base64 -w0 <messaging.crt>) \

+     -p MESSAGING_KEY=$(base64 -w0 <messaging.key>) \

+     -p KOJI_CERT=$(base64 -w0 <koji.crt>) \

+     -p KOJI_SERVERCA=$(base64 -w0 <koji_ca_cert.crt>) \

+     -p KOJI_URL=<KOJI_URL> \

+     -p STOMP_URI=<STOMP_URI> | oc apply -f -

+ ```

+ 

+ Use `oc process parameters` to learn more about template parameters:

+ 

+ ```bash

+ $ oc process --local -f openshift/mbs-test-template.yaml --parameters

+ NAME                 DESCRIPTION                                                                             GENERATOR           VALUE

+ TEST_ID              Short unique identifier for this test run (e.g. Jenkins job number)                                         

+ MBS_BACKEND_IMAGE    Image to be used for MBS backend deployment                                                                 172.30.1.1:5000/myproject/mbs-backend:latest

+ MBS_FRONTEND_IMAGE   Image to be used for MBS frontend deployment                                                                172.30.1.1:5000/myproject/mbs-frontend:latest

+ MESSAGING_CERT       base64 encoded SSL certificate for message bus authentication                                               

+ MESSAGING_KEY        base64 encoded SSL key for message bus authentication                                                       

+ KOJI_CERT            base 64 encoded client certificate used to authenticate with Koji                                           

+ KOJI_SERVERCA        base64 encoded certificate of the CA that issued the HTTP server certificate for Koji                       

+ DATABASE_PASSWORD                                                                                            expression          [\w]{32}

+ STOMP_URI            Messagebus URI                                                                                              

+ KOJI_URL             Top level URL of the Koji instance to use. Without a '/' at the end.                                        

+ ```

+ 

+ ## Delete MBS

+ 

+ ```bash

+ $ oc delete dc,deploy,pod,configmap,secret,svc,route -l app=mbs

+ ```

@@ -0,0 +1,32 @@ 

+ FROM fedora:28

+ LABEL \

+     name="Backend for the Module Build Service (MBS)" \

+     vendor="The Factory 2.0 Team" \

+     license="MIT" \

+     description="The MBS coordinates module builds. This image is to serve as the MBS backend." \

+     usage="https://pagure.io/fm-orchestrator" \

+     build-date=""

+ 

+ # The caller can chose to provide an already built module-build-service RPM.

+ ARG mbs_rpm=module-build-service

+ ARG mbs_messaging_umb_rpm

+ 

+ RUN dnf -y install \

+             $mbs_rpm \

+             $mbs_messaging_umb_rpm \

+             python2-psycopg2 \

+     && dnf -y clean all

+ 

+ # 1. Use latest stomp.py - hackish way for making this change, until there is

+ #    sufficient proof that newer versions don't break mbs_messaging_umb

+ #

+ # 2. Install python2-docopt - required by the latest version of stomp.py

+ #

+ # 3. Install python2-pungi - to make MBS RPMs built for RHEL work with a Fedora

+ #    base image

+ RUN sed -i 's/==3\.1\.6//g' /usr/lib/python2.7/site-packages/mbs_messaging_umb-*-py2.7.egg-info/requires.txt \

+     && dnf -y install python2-docopt python2-pungi \

+     && dnf -y clean all

+ 

+ VOLUME ["/etc/module-build-service", "/etc/fedmsg.d", "/etc/mbs-certs"]

+ ENTRYPOINT fedmsg-hub

@@ -0,0 +1,27 @@ 

+ # See `../backend/` for building `mbs-backend:latest`

+ FROM mbs-backend:latest

+ LABEL \

+     name="Frontend for the Module Build Service (MBS)" \

+     vendor="The Factory 2.0 Team" \

+     license="MIT" \

+     description="The MBS coordinates module builds. This image is to serve as the MBS frontend." \

+     usage="https://pagure.io/fm-orchestrator" \

+     build-date=""

+ 

+ RUN dnf -y install \

+             httpd \

+             mod_wsgi \

+     && dnf -y clean all

+ 

+ EXPOSE 8080/tcp 8443/tcp

+ VOLUME ["/etc/module-build-service", "/etc/fedmsg.d", "/etc/mbs-certs", "/etc/httpd/conf.d"]

+ ENTRYPOINT ["mod_wsgi-express", "start-server", "/usr/share/mbs/mbs.wsgi"]

+ CMD [\

+     "--user", "fedmsg", "--group", "fedmsg", \

+     "--port", "8080", "--threads", "1", \

+     "--include-file", "/etc/httpd/conf.d/mbs.conf", \

+     "--log-level", "info", \

+     "--log-to-terminal", \

+     "--access-log", \

+     "--startup-log" \

+ ]

@@ -0,0 +1,905 @@ 

+ ---

+ apiVersion: v1

+ kind: Template

+ metadata:

+   name: mbs-test-template

+ 

+ objects:

+ # frontend

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-frontend-fedmsg-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+   data:

+     logging.py: |

+       bare_format = "[%(asctime)s][%(name)10s %(levelname)7s] %(message)s"

+ 

+       config = dict(

+           logging=dict(

+               version=1,

+               formatters=dict(

+                   bare={

+                       "datefmt": "%Y-%m-%d %H:%M:%S",

+                       "format": bare_format

+                   },

+               ),

+               handlers=dict(

+                   console={

+                       "class": "logging.StreamHandler",

+                       "formatter": "bare",

+                       "level": "DEBUG",

+                       "stream": "ext://sys.stdout",

+                   },

+               ),

+               loggers=dict(

+                   fedmsg={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+                   moksha={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+               ),

+           ),

+       )

+     mbs-logging.py: |

+       config = dict(

+           logging=dict(

+               loggers=dict(

+                   # Quiet this guy down...

+                   requests={

+                       "level": "WARNING",

+                       "propagate": True,

+                   },

+                   module_build_service={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+                   mbs_messaging_umb={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+               ),

+               root=dict(

+                   handlers=["console"],

+                   level="DEBUG",

+               ),

+           ),

+       )

+     mbs-fedmsg.py: |

+       config = {

+           'zmq_enabled': False,

+           'validate_signatures': False,

+           'endpoints': {},

+           'stomp_uri': '${STOMP_URI}',

+           'stomp_heartbeat': 5000,

+           'stomp_ssl_crt': '/etc/mbs-certs/messaging.crt',

+           'stomp_ssl_key': '/etc/mbs-certs/messaging.key',

+           'stomp_ack_mode': 'auto',

+       }

+     mbs-scheduler.py: |

+       config = {

+           # The frontend should have these turned off in perpetuity.

+           'mbsconsumer': False,

+           'mbspoller': False,

+       }

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-frontend-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+   data:

+     config.py: |

+       class ProdConfiguration(object):

+           DEBUG = False

+ 

+           SECRET_KEY = ''

+ 

+           SQLALCHEMY_DATABASE_URI = 'postgresql://mbs:${DATABASE_PASSWORD}@mbs-database:5432/mbs'

+           SQLALCHEMY_TRACK_MODIFICATIONS = True

+ 

+           # Global network-related values, in seconds

+           NET_TIMEOUT = 120

+           NET_RETRY_INTERVAL = 30

+ 

+           SYSTEM = 'koji'

+           MESSAGING = 'umb'

+           MESSAGING_TOPIC_PREFIX = ['']

+           KOJI_CONFIG = '/etc/module-build-service/koji.conf'

+           KOJI_PROFILE = 'test'

+           KOJI_ARCHES = ['x86_64']

+           KOJI_PROXYUSER = False

+           KOJI_REPOSITORY_URL = ''

+           PDC_URL = ''

+           PDC_INSECURE = True

+           PDC_DEVELOP = True

+           SCMURLS = []

+ 

+           RESOLVER = 'db'

+ 

+           # This is a whitelist of prefixes of koji tags we're allowed to manipulate

+           KOJI_TAG_PREFIXES = ["module"]

+ 

+           DEFAULT_DIST_TAG_PREFIX = 'module'

+ 

+           # Use the same priority as all other builds

+           KOJI_BUILD_PRIORITY = 0

+ 

+           # Control where modules get tagged post-build.

+           BASE_MODULE_NAMES = ['platform']

+           KOJI_CG_BUILD_TAG_TEMPLATE = ''

+           KOJI_CG_DEFAULT_BUILD_TAG = ''

+ 

+           # yes, we want everyone to authenticate

+           NO_AUTH = False

+ 

+           YAML_SUBMIT_ALLOWED = False

+ 

+           # Allow maintainers to specify something that differs from the git branch.

+           ALLOW_NAME_OVERRIDE_FROM_SCM = False

+           ALLOW_STREAM_OVERRIDE_FROM_SCM = False

+ 

+           # How often should we resort to polling, in seconds

+           # Set to zero to disable polling

+           POLLING_INTERVAL = 600

+ 

+           # Determines how many builds that can be submitted to the builder

+           # and be in the build state at a time. Set this to 0 for no restrictions

+           NUM_CONCURRENT_BUILDS = 2

+ 

+           RPMS_DEFAULT_REPOSITORY = ''

+           RPMS_ALLOW_REPOSITORY = False

+           RPMS_DEFAULT_CACHE = ''

+           RPMS_ALLOW_CACHE = False

+           MODULES_DEFAULT_REPOSITORY = ''

+           MODULES_ALLOW_REPOSITORY = False

+ 

+           # Our per-build logs for the Koji content generator go here.

+           # CG imports are controlled by KOJI_ENABLE_CONTENT_GENERATOR

+           BUILD_LOGS_DIR = '/var/tmp'

+ 

+           # Time after which MBS will delete koji targets it created.

+           KOJI_TARGET_DELETE_TIME = 86400

+ 

+           # Whether or not to import modules back to koji.

+           KOJI_ENABLE_CONTENT_GENERATOR = False

+ 

+           # Available backends are: console, file.

+           LOG_BACKEND = 'console'

+ 

+           # Available log levels are: debug, info, warn, error.

+           LOG_LEVEL = 'debug'

+ 

+           REBUILD_STRATEGY_ALLOW_OVERRIDE = True

+           REBUILD_STRATEGY = 'only-changed'

+ 

+           # Settings for Kerberos + LDAP auth

+           AUTH_METHOD = 'oidc'

+           # These groups are allowed to submit builds.

+           ALLOWED_GROUPS = []

+           # These groups are allowed to cancel the builds of other users.

+           ADMIN_GROUPS = []

+     koji.conf: |

+       [test]

+       server = ${KOJI_URL}/kojihub

+       weburl = ${KOJI_URL}/koji/

+       topurl = ${KOJI_URL}/kojiroot/

+       authtype = ssl

+       ;client certificate

+       cert = /etc/koji-certs/kojiadmin.crt

+       ;certificate of the CA that issued the client certificate

+       ;ca = /etc/koji-certs/clientca.crt

+       ;certificate of the CA that issued the HTTP server certificate

+       serverca = /etc/koji-certs/koji_ca_cert.crt

+     mock.cfg: |

+       config_opts['root'] = '$root'

+       config_opts['target_arch'] = '$arch'

+       config_opts['legal_host_arches'] = ('$arch',)

+       config_opts['chroot_setup_cmd'] = 'install $group'

+       config_opts['dist'] = ''

+       config_opts['extra_chroot_dirs'] = [ '/run/lock', ]

+       config_opts['releasever'] = ''

+       config_opts['package_manager'] = 'dnf'

+       config_opts['nosync'] = True

+       config_opts['use_bootstrap_container'] = False

+ 

+       config_opts['yum.conf'] = """

+       $yum_conf

+       """

+     yum.conf: |

+       [main]

+       keepcache=1

+       debuglevel=2

+       reposdir=/dev/null

+       logfile=/var/log/yum.log

+       retries=20

+       obsoletes=1

+       gpgcheck=0

+       assumeyes=1

+       syslog_ident=mock

+       syslog_device=

+       install_weak_deps=0

+       metadata_expire=3600

+       mdpolicy=group:primary

+ 

+       # repos

+     platform.yaml: |

+       document: modulemd

+       version: 1

+       data:

+         description: Fedora 28 traditional base

+         name: platform

+         license:

+           module: [MIT]

+         profiles:

+           buildroot:

+             rpms: [bash, bzip2, coreutils, cpio, diffutils, fedora-release, findutils, gawk,

+               gcc, gcc-c++, grep, gzip, info, make, patch, redhat-rpm-config, rpm-build,

+               sed, shadow-utils, tar, unzip, util-linux, which, xz]

+           srpm-buildroot:

+             rpms: [bash, fedora-release, fedpkg-minimal, gnupg2, redhat-rpm-config, rpm-build,

+               shadow-utils]

+         stream: f28

+         summary: Fedora 28 traditional base

+         version: 3

+         context: 00000000

+         xmd:

+           mbs:

+             buildrequires: {}

+             commit: virtual

+             requires: {}

+             mse: true

+             koji_tag: module-f28-build

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-httpd-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+   data:

+     mbs.conf: |

+       <Location />

+           Require all granted

+       </Location>

+ 

+       RedirectMatch ^/$ /module-build-service/1/module-builds/

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-wsgi-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+   data:

+     mbs.wsgi: |

+       #-*- coding: utf-8 -*-

+ 

+       import logging

+       logging.basicConfig(level='DEBUG')

+ 

+       from module_build_service import app as application

+ - apiVersion: v1

+   kind: Secret

+   metadata:

+     name: "mbs-frontend-certificates"

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: frontend

+   data:

+     messaging.crt: |-

+       ${MESSAGING_CERT}

+     messaging.key: |-

+       ${MESSAGING_KEY}

+ - apiVersion: v1

+   kind: Service

+   metadata:

+     name: "mbs-frontend"

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: frontend

+   spec:

+     selector:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+     ports:

+       - name: https

+         port: 443

+         targetPort: https

+       - name: http

+         port: 80

+         targetPort: http

+ - apiVersion: v1

+   kind: Route

+   metadata:

+     name: mbs-api

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: frontend

+   spec:

+     to:

+       kind: Service

+       name: mbs-frontend

+     tls:

+       termination: edge

+       insecureEdgeTerminationPolicy: Redirect

+ - apiVersion: v1

+   kind: DeploymentConfig

+   metadata:

+     name: "mbs-frontend"

+     labels:

+       environment: "test-${TEST_ID}"

+       service: frontend

+       app: mbs

+   spec:

+     replicas: 1

+     strategy:

+       type: Recreate

+     selector:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: frontend

+     strategy:

+       type: Rolling

+     template:

+       metadata:

+         labels:

+           environment: "test-${TEST_ID}"

+           service: frontend

+           app: mbs

+       spec:

+         containers:

+         - name: frontend

+           image: "${MBS_FRONTEND_IMAGE}"

+           imagePullPolicy: Always

+           ports:

+           - containerPort: 8080

+             protocol: TCP

+             name: http

+           - containerPort: 8443

+             protocol: TCP

+             name: https

+           volumeMounts:

+           - name: fedmsg-config

+             mountPath: /etc/fedmsg.d

+             readOnly: true

+           - name: frontend-certificates

+             mountPath: /etc/mbs-certs

+             readOnly: true

+           - name: mbs-config

+             mountPath: /etc/module-build-service

+             readOnly: true

+           - name: httpd-config

+             mountPath: /etc/httpd/conf.d

+             readOnly: true

+           - name: wsgi-config

+             mountPath: /usr/share/mbs

+             readOnly: true

+           - name: koji-certificates

+             mountPath: /etc/koji-certs

+             readOnly: true

+           resources:

+             limits:

+               memory: 400Mi

+               cpu: 300m

+         volumes:

+         - name: fedmsg-config

+           configMap:

+             name: mbs-frontend-fedmsg-config

+         - name: frontend-certificates

+           secret:

+             secretName: mbs-frontend-certificates

+         - name: mbs-config

+           configMap:

+             name: mbs-frontend-config

+         - name: httpd-config

+           configMap:

+             name: mbs-httpd-config

+         - name: wsgi-config

+           configMap:

+             name: mbs-wsgi-config

+         - name: koji-certificates

+           secret:

+             secretName: mbs-koji-secrets

+       triggers:

+       - type: ConfigChange

+ # backend

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-backend-fedmsg-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: backend

+   data:

+     logging.py: |

+       bare_format = "[%(asctime)s][%(name)10s %(levelname)7s] %(message)s"

+ 

+       config = dict(

+           logging=dict(

+               version=1,

+               formatters=dict(

+                   bare={

+                       "datefmt": "%Y-%m-%d %H:%M:%S",

+                       "format": bare_format

+                   },

+               ),

+               handlers=dict(

+                   console={

+                       "class": "logging.StreamHandler",

+                       "formatter": "bare",

+                       "level": "DEBUG",

+                       "stream": "ext://sys.stdout",

+                   },

+               ),

+               loggers=dict(

+                   fedmsg={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+                   moksha={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+               ),

+           ),

+       )

+     mbs-logging.py: |

+       config = dict(

+           logging=dict(

+               loggers=dict(

+                   # Quiet this guy down...

+                   requests={

+                       "level": "WARNING",

+                       "propagate": True,

+                   },

+                   module_build_service={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+                   mbs_messaging_umb={

+                       "level": "DEBUG",

+                       "propagate": True,

+                   },

+               ),

+               root=dict(

+                   handlers=["console"],

+                   level="DEBUG",

+               ),

+           ),

+       )

+     mbs-fedmsg.py: |

+       config = {

+           'zmq_enabled': False,

+           'validate_signatures': False,

+           'endpoints': {},

+           'stomp_uri': '${STOMP_URI}',

+           'stomp_heartbeat': 5000,

+           'stomp_ssl_crt': '/etc/mbs-certs/messaging.crt',

+           'stomp_ssl_key': '/etc/mbs-certs/messaging.key',

+           'stomp_ack_mode': 'auto',

+       }

+     mbs-scheduler.py: |

+       config = {

+           'mbsconsumer': True,

+           'mbspoller': True,

+       }

+ - apiVersion: v1

+   kind: ConfigMap

+   metadata:

+     name: "mbs-backend-config"

+     labels:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: backend

+   data:

+     config.py: |

+       class ProdConfiguration(object):

+           DEBUG = False

+ 

+           SECRET_KEY = ''

+ 

+           SQLALCHEMY_DATABASE_URI = 'postgresql://mbs:${DATABASE_PASSWORD}@mbs-database:5432/mbs'

+           SQLALCHEMY_TRACK_MODIFICATIONS = True

+ 

+           # Global network-related values, in seconds

+           NET_TIMEOUT = 120

+           NET_RETRY_INTERVAL = 30

+ 

+           SYSTEM = 'koji'

+           MESSAGING = 'umb'

+           MESSAGING_TOPIC_PREFIX = ['']

+           KOJI_CONFIG = '/etc/module-build-service/koji.conf'

+           KOJI_PROFILE = 'test'

+           KOJI_ARCHES = ['x86_64']

+           KOJI_PROXYUSER = False

+           KOJI_REPOSITORY_URL = ''

+           PDC_URL = ''

+           PDC_INSECURE = True

+           PDC_DEVELOP = True

+           SCMURLS = []

+ 

+           RESOLVER = 'db'

+ 

+           # This is a whitelist of prefixes of koji tags we're allowed to manipulate

+           KOJI_TAG_PREFIXES = ["module"]

+ 

+           DEFAULT_DIST_TAG_PREFIX = 'module'

+ 

+           # Use the same priority as all other builds

+           KOJI_BUILD_PRIORITY = 0

+ 

+           # Control where modules get tagged post-build.

+           BASE_MODULE_NAMES = ['platform']

+           KOJI_CG_BUILD_TAG_TEMPLATE = ''

+           KOJI_CG_DEFAULT_BUILD_TAG = ''

+ 

+           # yes, we want everyone to authenticate

+           NO_AUTH = False

+ 

+           YAML_SUBMIT_ALLOWED = False

+ 

+           # Allow maintainers to specify something that differs from the git branch.

+           ALLOW_NAME_OVERRIDE_FROM_SCM = False

+           ALLOW_STREAM_OVERRIDE_FROM_SCM = False

+ 

+           # How often should we resort to polling, in seconds

+           # Set to zero to disable polling

+           POLLING_INTERVAL = 20

+ 

+           # Determines how many builds that can be submitted to the builder

+           # and be in the build state at a time. Set this to 0 for no restrictions

+           NUM_CONCURRENT_BUILDS = 2

+ 

+           RPMS_DEFAULT_REPOSITORY = ''

+           RPMS_ALLOW_REPOSITORY = False

+           RPMS_DEFAULT_CACHE = ''

+           RPMS_ALLOW_CACHE = False

+           MODULES_DEFAULT_REPOSITORY = ''

+           MODULES_ALLOW_REPOSITORY = False

+ 

+           # Our per-build logs for the Koji content generator go here.

+           # CG imports are controlled by KOJI_ENABLE_CONTENT_GENERATOR

+           BUILD_LOGS_DIR = '/var/tmp'

+ 

+           # Time after which MBS will delete koji targets it created.

+           KOJI_TARGET_DELETE_TIME = 86400

+ 

+           # Whether or not to import modules back to koji.

+           KOJI_ENABLE_CONTENT_GENERATOR = False

+ 

+           # Available backends are: console, file.

+           LOG_BACKEND = 'console'

+ 

+           # Available log levels are: debug, info, warn, error.

+           LOG_LEVEL = 'debug'

+ 

+           REBUILD_STRATEGY_ALLOW_OVERRIDE = True

+           REBUILD_STRATEGY = 'only-changed'

+     koji.conf: |

+       [test]

+       server = ${KOJI_URL}/kojihub

+       weburl = ${KOJI_URL}/koji/

+       topurl = ${KOJI_URL}/kojiroot/

+       authtype = ssl

+       ;client certificate

+       cert = /etc/koji-certs/kojiadmin.crt

+       ;certificate of the CA that issued the client certificate

+       ;ca = /etc/koji-certs/clientca.crt

+       ;certificate of the CA that issued the HTTP server certificate

+       serverca = /etc/koji-certs/koji_ca_cert.crt

+     mock.cfg: |

+       config_opts['root'] = '$root'

+       config_opts['target_arch'] = '$arch'

+       config_opts['legal_host_arches'] = ('$arch',)

+       config_opts['chroot_setup_cmd'] = 'install $group'

+       config_opts['dist'] = ''

+       config_opts['extra_chroot_dirs'] = [ '/run/lock', ]

+       config_opts['releasever'] = ''

+       config_opts['package_manager'] = 'dnf'

+       config_opts['nosync'] = True

+       config_opts['use_bootstrap_container'] = False

+ 

+       config_opts['yum.conf'] = """

+       $yum_conf

+       """

+     yum.conf: |

+       [main]

+       keepcache=1

+       debuglevel=2

+       reposdir=/dev/null

+       logfile=/var/log/yum.log

+       retries=20

+       obsoletes=1

+       gpgcheck=0

+       assumeyes=1

+       syslog_ident=mock

+       syslog_device=

+       install_weak_deps=0

+       metadata_expire=3600

+       mdpolicy=group:primary

+ 

+       # repos

+     platform.yaml: |

+       document: modulemd

+       version: 1

+       data:

+         description: Fedora 28 traditional base

+         name: platform

+         license:

+           module: [MIT]

+         profiles:

+           buildroot:

+             rpms: [bash, bzip2, coreutils, cpio, diffutils, fedora-release, findutils, gawk,

+               gcc, gcc-c++, grep, gzip, info, make, patch, redhat-rpm-config, rpm-build,

+               sed, shadow-utils, tar, unzip, util-linux, which, xz]

+           srpm-buildroot:

+             rpms: [bash, fedora-release, fedpkg-minimal, gnupg2, redhat-rpm-config, rpm-build,

+               shadow-utils]

+         stream: f28

+         summary: Fedora 28 traditional base

+         version: 3

+         context: 00000000

+         xmd:

+           mbs:

+             buildrequires: {}

+             commit: virtual

+             requires: {}

+             mse: true

+             koji_tag: module-f28-build

+ - apiVersion: v1

+   kind: Secret

+   metadata:

+     name: mbs-backend-secrets

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: backend

+   data:

+     messaging.crt: |-

+       ${MESSAGING_CERT}

+     messaging.key: |-

+       ${MESSAGING_KEY}

+ - apiVersion: v1

+   kind: Secret

+   metadata:

+     name: mbs-koji-secrets

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+   data:

+     kojiadmin.crt: |-

+       ${KOJI_CERT}

+     koji_ca_cert.crt: |-

+       ${KOJI_SERVERCA}

+ - apiVersion: v1

+   kind: DeploymentConfig

+   metadata:

+     name: "mbs-backend"

+     labels:

+       environment: "test-${TEST_ID}"

+       service: backend

+       app: mbs

+   spec:

+     replicas: 1

+     strategy:

+       type: Recreate

+     selector:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: backend

+     strategy:

+       type: Rolling

+       rollingParams:

+         pre:

+           failurePolicy: Abort

+           execNewPod:

+             containerName: backend

+             command:

+             - /bin/sh

+             - -i

+             - -c

+             - |

+               # try for 10 minutes (600 seconds)

+               e=$(( $(date +%s) + 600 ))

+               i=0

+               while [ $(date +%s) -lt $e ]; do

+                 echo 'TRY #'$((++i))

+                 if mbs-upgradedb ; then

+                   mbs-manager import_module /etc/module-build-service/platform.yaml

+                   exit 0

+                 fi

+               done

+               exit 1

+             volumes:

+             - mbs-config

+     template:

+       metadata:

+         labels:

+           environment: "test-${TEST_ID}"

+           service: backend

+           app: mbs

+       spec:

+         containers:

+         - name: backend

+           image: "${MBS_BACKEND_IMAGE}"

+           imagePullPolicy: Always

+           volumeMounts:

+           - name: fedmsg-config

+             mountPath: /etc/fedmsg.d

+             readOnly: true

+           - name: mbs-config

+             mountPath: /etc/module-build-service

+             readOnly: true

+           - name: backend-certificates

+             mountPath: /etc/mbs-certs

+             readOnly: true

+           - name: koji-certificates

+             mountPath: /etc/koji-certs

+             readOnly: true

+           resources:

+             limits:

+               memory: 400Mi

+               cpu: 300m

+         volumes:

+         - name: fedmsg-config

+           configMap:

+             name: mbs-backend-fedmsg-config

+         - name: mbs-config

+           configMap:

+             name: mbs-backend-config

+         - name: backend-certificates

+           secret:

+             secretName: mbs-backend-secrets

+         - name: koji-certificates

+           secret:

+             secretName: mbs-koji-secrets

+       triggers:

+       - type: ConfigChange

+ # postgresql

+ - apiVersion: v1

+   kind: Secret

+   metadata:

+     name: "mbs-database-secret"

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: database

+   stringData:

+     database-password: "${DATABASE_PASSWORD}"

+ - apiVersion: v1

+   kind: Service

+   metadata:

+     name: "mbs-database"

+     labels:

+       environment: "test-${TEST_ID}"

+       app: mbs

+       service: database

+   spec:

+     selector:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: database

+     ports:

+     - name: postgresql

+       port: 5432

+       targetPort: 5432

+ - apiVersion: v1

+   kind: DeploymentConfig

+   metadata:

+     name: "mbs-database"

+     labels:

+       environment: "test-${TEST_ID}"

+       service: database

+       app: mbs

+   spec:

+     replicas: 1

+     strategy:

+       type: Recreate

+     selector:

+       app: mbs

+       environment: "test-${TEST_ID}"

+       service: database

+     template:

+       metadata:

+         labels:

+           environment: "test-${TEST_ID}"

+           service: database

+           app: mbs

+       spec:

+         containers:

+         - name: postgresql

+           image: registry.access.redhat.com/rhscl/postgresql-95-rhel7:latest

+           imagePullPolicy: Always

+           ports:

+           - containerPort: 5432

+             protocol: TCP

+           resources:

+             limits:

+               memory: 512Mi

+               cpu: 0.4

+           readinessProbe:

+             timeoutSeconds: 1

+             initialDelaySeconds: 5

+             exec:

+               command: [ /bin/sh, -i, -c, "psql -h 127.0.0.1 -U $POSTGRESQL_USER -q -d $POSTGRESQL_DATABASE -c 'SELECT 1'" ]

+           livenessProbe:

+             timeoutSeconds: 1

+             initialDelaySeconds: 30

+             tcpSocket:

+               port: 5432

+           env:

+           - name: POSTGRESQL_USER

+             value: mbs

+           - name: POSTGRESQL_PASSWORD

+             valueFrom:

+               secretKeyRef:

+                 name: "mbs-database-secret"

+                 key: database-password

+           - name: POSTGRESQL_DATABASE

+             value: mbs

+     triggers:

+     - type: ConfigChange

+ 

+ # template parameters

+ parameters:

+ - name: TEST_ID

+   displayName: Test id

+   description: Short unique identifier for this test run (e.g. Jenkins job number)

+   required: true

+ - name: MBS_BACKEND_IMAGE

+   displayName: Container image for MBS backend

+   description: Image to be used for MBS backend deployment

+   value: 172.30.1.1:5000/myproject/mbs-backend:latest

+   required: true

+ - name: MBS_FRONTEND_IMAGE

+   displayName: Container image for MBS frontend

+   description: Image to be used for MBS frontend deployment

+   value: 172.30.1.1:5000/myproject/mbs-frontend:latest

+   required: true

+ - name: MESSAGING_CERT

+   displayName: SSL certificate for messaging

+   description: base64 encoded SSL certificate for message bus authentication

+   required: true

+ - name: MESSAGING_KEY

+   displayName: SSL key for messaging

+   description: base64 encoded SSL key for message bus authentication

+   required: true

+ - name: KOJI_CERT

+   displayName: Koji client certificate

+   description: base 64 encoded client certificate used to authenticate with Koji

+   required: true

+ - name: KOJI_SERVERCA

+   displayName: Koji server CA

+   description: >-

+     base64 encoded certificate of the CA

+     that issued the HTTP server certificate for Koji

+   required: true

+ - name: DATABASE_PASSWORD

+   displayName: Database password

+   generate: expression

+   from: "[\\w]{32}"

+ - name: STOMP_URI

+   displayName: Messagebus URI

+   description: Messagebus URI

+   required: true

+ - name: KOJI_URL

+   displayName: Top level URL of the Koji instance to use

+   description: Top level URL of the Koji instance to use. Without a '/' at the end.

+   default: https://mbs-brew-hub.usersys.redhat.com

+   required: true

Add Dockerfiles to build images for the backend and frontend.

Add an OpenShift template to deploy an MBS test instance, and connect it
to a message bus and Koji instance.

Signed-off-by: Hunor Csomortáni csomh@redhat.com

It looks good, +1. I have not tried to deploy this myself anywhere, but it seems to do sane things and the configuration looks OK.

Not a big deal, but this is no longer a set in the latest version of MBS. Also, making this ['platform'] would be a nice default.

Why are these hardcoded?

Can some of these not be reused from above?

What cert does https end up using? Are they autogenerated by mod_wsgi-express.

What is the correct type? Ansible role templates still use this as a set().

Why ['platform']?

For convenience :)

I'll make them template parameters and update the docs accordingly.

Afaik a volume mount point can take only one configmap or secret. It's not possible to layer multiple ones.

Most of the frontend and backend configuration is identical, but due to the small differences that do exist, separate configmaps had to be created.

Now, this is a good questions, but I couldn't find a full answer for it.

At some point I thought using --https-port is important, but it seems it does not matter, at least when deploying in UpShift - it seems, that https in that cluster will use the cluster certificate.

This probably is controlled by the Route being set to use edge termination policy.

I agree. I say just remove the HTTPS listener here.

It was changed at:
https://pagure.io/fm-orchestrator/c/54c1ed716629c319fd5178409bb85b356332321d?branch=master

['platform'] is the base module used in all the environments that MBS is deployed in that I'm aware of.

rebased onto 65300a5

6 months ago

@mprahl addressed all your comments and rebased it to master, ptal. Thanks!

Pull-Request has been merged by mprahl

6 months ago