#1467 use C3I-as-a-Service to allocate a dynamic project for building and testing
Merged 4 years ago by mprahl. Opened 4 years ago by mikeb.
mikeb/fm-orchestrator c3iaas  into  master

@@ -135,6 +135,24 @@ 

      causes all available testcases to run. The value "skip" causes no testcases to be run.

    required: false

    value: ""

+ - name: USE_C3IAAS

Does this need to be a parameter? It seems like it wouldn't work if this were false.

+   displayName: >-

+     Use C3I-as-a-Service to dynamically allocate a temporary OpenShift project for building

+     and testing.

+   required: true

+   value: "true"

+ - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE

+   displayName: The namespace where the C3I-as-a-Service project request BuildConfig has been defined

+   required: false

+   value: c3i

+ - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME

+   displayName: The name of the C3I-as-a-Service project request BuildConfig

+   required: false

+   value: c3iaas-request-project

+ - name: C3IAAS_LIFETIME

+   displayName: The lifetime of the OpenShift project allocated by C3I-as-a-Service.

+   required: true

+   value: "120"

  - name: CLEANUP

    displayName: Cleanup objects after the pipeline is complete

    required: true
@@ -203,7 +221,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial"

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      source:

        git:
@@ -267,6 +285,14 @@ 

            value: "${EXTRA_RPMS}"

          - name: TESTCASES

            value: "${TESTCASES}"

+         - name: USE_C3IAAS

+           value: "${USE_C3IAAS}"

+         - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE

+           value: "${C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE}"

+         - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME

+           value: "${C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME}"

+         - name: C3IAAS_LIFETIME

+           value: "${C3IAAS_LIFETIME}"

          - name: CLEANUP

            value: "${CLEANUP}"

          jenkinsfilePath: openshift/integration/koji/pipelines/templates/mbs-build.Jenkinsfile

@@ -42,7 +42,6 @@ 

      timestamps()

      timeout(time: 120, unit: 'MINUTES')

      buildDiscarder(logRotator(numToKeepStr: '10'))

-     disableConcurrentBuilds()

      skipDefaultCheckout()

    }

    environment {
@@ -55,25 +54,7 @@ 

      stage('Prepare') {

        steps {

          script {

-           // check out specified branch/commit

-           def srcRef = env.MBS_GIT_REF.startsWith('pull/') ? env.MBS_GIT_REF : "heads/${env.MBS_GIT_REF}"

-           retry(5) {

-             checkout([$class: 'GitSCM',

-               branches: [[name: params.MBS_GIT_REF]],

-               userRemoteConfigs: [

-                 [

-                   name: 'origin',

-                   url: params.MBS_GIT_REPO,

-                   refspec: "+refs/${srcRef}:refs/remotes/origin/${env.MBS_GIT_REF}",

-                 ],

-               ],

-               extensions: [

-                 [$class: 'CleanBeforeCheckout'],

-                 [$class: 'CloneOption', noTags: true, shallow: true, depth: 2, honorRefspec: true],

-               ],

-             ])

-           }

- 

+           c3i.clone(repo: params.MBS_GIT_REPO, branch: params.MBS_GIT_REF)

            // get current commit ID

            // FIXME: Due to a bug discribed in https://issues.jenkins-ci.org/browse/JENKINS-45489,

            // the return value of checkout() is unreliable.
@@ -167,6 +148,42 @@ 

  	}

        }

      }

+     stage('Allocate C3IaaS project') {

+       when {

+         expression {

+           return params.USE_C3IAAS == 'true' &&

+                  params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE &&

+                  params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME

+         }

+       }

+       steps {

+         script {

+           if (env.PR_NO) {

+             env.C3IAAS_NAMESPACE = "c3i-mbs-pr-${env.PR_NO}-git${env.MBS_GIT_COMMIT.take(8)}"

+           } else {

+             env.C3IAAS_NAMESPACE = "c3i-mbs-${params.MBS_GIT_REF}-git${env.MBS_GIT_COMMIT.take(8)}"

+           }

+           echo "Requesting new OpenShift project ${env.C3IAAS_NAMESPACE}..."

+           openshift.withCluster() {

+             openshift.withProject(params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE) {

+               c3i.buildAndWait(script: this, objs: "bc/${params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME}",

+                 '-e', "PROJECT_NAME=${env.C3IAAS_NAMESPACE}",

+                 '-e', "ADMIN_GROUPS=system:serviceaccounts:${PIPELINE_NAMESPACE}",

+                 '-e', "LIFETIME_IN_MINUTES=${params.C3IAAS_LIFETIME}"

+               )

+             }

+           }

+         }

+       }

+       post {

+         success {

+           echo "Allocated project ${env.C3IAAS_NAMESPACE}"

+         }

+         failure {

+           echo "Failed to allocate ${env.C3IAAS_NAMESPACE} project"

+         }

+       }

+     }

      stage('Build backend image') {

        environment {

          BACKEND_BUILDCONFIG_ID = "mbs-backend-build-${currentBuild.id}-${UUID.randomUUID().toString().take(7)}"
@@ -174,33 +191,34 @@ 

        steps {

          script {

            openshift.withCluster() {

-             // OpenShift BuildConfig doesn't support specifying a tag name at build time.

-             // We have to create a new BuildConfig for each image build.

-             echo 'Creating a BuildConfig for mbs-backend build...'

-             def created = new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone('UTC'))

-             def template = readYaml file: 'openshift/backend/mbs-backend-build-template.yaml'

-             def processed = openshift.process(template,

-               '-p', "NAME=${env.BACKEND_BUILDCONFIG_ID}",

-               '-p', "MBS_GIT_REPO=${params.MBS_GIT_REPO}",

-               // A pull-request branch, like pull/123/head, cannot be built with commit ID

-               // because refspec cannot be customized in an OpenShift build.

-               '-p', "MBS_GIT_REF=${env.PR_NO ? params.MBS_GIT_REF : env.MBS_GIT_COMMIT}",

-               '-p', "MBS_BACKEND_IMAGESTREAM_NAME=${params.MBS_BACKEND_IMAGESTREAM_NAME}",

-               '-p', "MBS_BACKEND_IMAGESTREAM_NAMESPACE=${env.PIPELINE_NAMESPACE}",

-               '-p', "MBS_IMAGE_TAG=${env.TEMP_TAG}",

-               '-p', "EXTRA_RPMS=${params.EXTRA_RPMS}",

-               '-p', "CREATED=${created}"

-             )

-             def buildname = c3i.buildAndWait(processed, '--from-dir=.')

-             def build = openshift.selector(buildname)

-             def ocpBuild = build.object()

-             env.BACKEND_IMAGE_DIGEST = ocpBuild.status.output.to.imageDigest

-             def ref = ocpBuild.status.outputDockerImageReference

-             def repo = ref.tokenize(':')[0..-2].join(':')

-             env.BACKEND_IMAGE_REPO = repo

-             env.BACKEND_IMAGE_REF = repo + '@' + env.BACKEND_IMAGE_DIGEST

-             env.BACKEND_IMAGE_TAG = env.TEMP_TAG

-             echo "Built image ${env.BACKEND_IMAGE_REF}, digest: ${env.BACKEND_IMAGE_DIGEST}, tag: ${env.BACKEND_IMAGE_TAG}"

+             openshift.withProject(env.C3IAAS_NAMESPACE ?: env.PIPELINE_NAMESPACE) {

+               // OpenShift BuildConfig doesn't support specifying a tag name at build time.

+               // We have to create a new BuildConfig for each image build.

+               echo 'Creating a BuildConfig for mbs-backend build...'

+               def created = new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone('UTC'))

+               def template = readYaml file: 'openshift/backend/mbs-backend-build-template.yaml'

+               def processed = openshift.process(template,

+                 '-p', "NAME=${env.BACKEND_BUILDCONFIG_ID}",

+                 '-p', "MBS_GIT_REPO=${params.MBS_GIT_REPO}",

+                 // A pull-request branch, like pull/123/head, cannot be built with commit ID

+                 // because refspec cannot be customized in an OpenShift build.

+                 '-p', "MBS_GIT_REF=${env.PR_NO ? params.MBS_GIT_REF : env.MBS_GIT_COMMIT}",

+                 '-p', "MBS_BACKEND_IMAGESTREAM_NAME=${params.MBS_BACKEND_IMAGESTREAM_NAME}",

+                 '-p', "MBS_BACKEND_IMAGESTREAM_NAMESPACE=${env.C3IAAS_NAMESPACE ?: env.PIPELINE_NAMESPACE}",

+                 '-p', "MBS_IMAGE_TAG=${env.TEMP_TAG}",

+                 '-p', "EXTRA_RPMS=${params.EXTRA_RPMS}",

+                 '-p', "CREATED=${created}"

+               )

+               def build = c3i.buildAndWait(script: this, objs: processed, '--from-dir=.')

+               def ocpBuild = build.object()

+               env.BACKEND_IMAGE_DIGEST = ocpBuild.status.output.to.imageDigest

+               def ref = ocpBuild.status.outputDockerImageReference

+               def repo = ref.tokenize(':')[0..-2].join(':')

+               env.BACKEND_IMAGE_REPO = repo

+               env.BACKEND_IMAGE_REF = repo + '@' + env.BACKEND_IMAGE_DIGEST

+               env.BACKEND_IMAGE_TAG = env.TEMP_TAG

+               echo "Built image ${env.BACKEND_IMAGE_REF}, digest: ${env.BACKEND_IMAGE_DIGEST}, tag: ${env.BACKEND_IMAGE_TAG}"

+             }

            }

          }

        }
@@ -210,12 +228,14 @@ 

          }

          cleanup {

            script {

-             openshift.withCluster() {

-               echo 'Tearing down...'

-               openshift.selector('bc', [

-                 'app': env.BACKEND_BUILDCONFIG_ID,

-                 'template': 'mbs-backend-build-template',

-                 ]).delete()

+             if (!env.C3IAAS_NAMESPACE) {

Could we just require C3IAAS and remove this branch of code?

+               openshift.withCluster() {

+                 echo 'Tearing down...'

+                 openshift.selector('bc', [

+                   'app': env.BACKEND_BUILDCONFIG_ID,

+                   'template': 'mbs-backend-build-template',

+                   ]).delete()

+               }

              }

            }

          }
@@ -228,34 +248,35 @@ 

        steps {

          script {

            openshift.withCluster() {

-             // OpenShift BuildConfig doesn't support specifying a tag name at build time.

-             // We have to create a new BuildConfig for each image build.

-             echo 'Creating a BuildConfig for mbs-frontend build...'

-             def created = new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone('UTC'))

-             def template = readYaml file: 'openshift/frontend/mbs-frontend-build-template.yaml'

-             def processed = openshift.process(template,

-               '-p', "NAME=${env.FRONTEND_BUILDCONFIG_ID}",

-               '-p', "MBS_GIT_REPO=${params.MBS_GIT_REPO}",

-               // A pull-request branch, like pull/123/head, cannot be built with commit ID

-               // because refspec cannot be customized in an OpenShift build.

-               '-p', "MBS_GIT_REF=${env.PR_NO ? params.MBS_GIT_REF : env.MBS_GIT_COMMIT}",

-               '-p', "MBS_FRONTEND_IMAGESTREAM_NAME=${params.MBS_FRONTEND_IMAGESTREAM_NAME}",

-               '-p', "MBS_FRONTEND_IMAGESTREAM_NAMESPACE=${env.PIPELINE_NAMESPACE}",

-               '-p', "MBS_IMAGE_TAG=${env.TEMP_TAG}",

-               '-p', "MBS_BACKEND_IMAGESTREAM_NAME=${params.MBS_BACKEND_IMAGESTREAM_NAME}",

-               '-p', "MBS_BACKEND_IMAGESTREAM_NAMESPACE=${env.PIPELINE_NAMESPACE}",

-               '-p', "CREATED=${created}"

-             )

-             def buildname = c3i.buildAndWait(processed, '--from-dir=.')

-             def build = openshift.selector(buildname)

-             def ocpBuild = build.object()

-             env.FRONTEND_IMAGE_DIGEST = ocpBuild.status.output.to.imageDigest

-             def ref = ocpBuild.status.outputDockerImageReference

-             def repo = ref.tokenize(':')[0..-2].join(':')

-             env.FRONTEND_IMAGE_REPO = repo

-             env.FRONTEND_IMAGE_REF = repo + '@' + env.FRONTEND_IMAGE_DIGEST

-             env.FRONTEND_IMAGE_TAG = env.TEMP_TAG

-             echo "Built image ${env.FRONTEND_IMAGE_REF}, digest: ${env.FRONTEND_IMAGE_DIGEST}, tag: ${env.FRONTEND_IMAGE_TAG}"

+             openshift.withProject(env.C3IAAS_NAMESPACE ?: env.PIPELINE_NAMESPACE) {

+               // OpenShift BuildConfig doesn't support specifying a tag name at build time.

+               // We have to create a new BuildConfig for each image build.

+               echo 'Creating a BuildConfig for mbs-frontend build...'

+               def created = new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone('UTC'))

+               def template = readYaml file: 'openshift/frontend/mbs-frontend-build-template.yaml'

+               def processed = openshift.process(template,

+                 '-p', "NAME=${env.FRONTEND_BUILDCONFIG_ID}",

+                 '-p', "MBS_GIT_REPO=${params.MBS_GIT_REPO}",

+                 // A pull-request branch, like pull/123/head, cannot be built with commit ID

+                 // because refspec cannot be customized in an OpenShift build.

+                 '-p', "MBS_GIT_REF=${env.PR_NO ? params.MBS_GIT_REF : env.MBS_GIT_COMMIT}",

+                 '-p', "MBS_FRONTEND_IMAGESTREAM_NAME=${params.MBS_FRONTEND_IMAGESTREAM_NAME}",

+                 '-p', "MBS_FRONTEND_IMAGESTREAM_NAMESPACE=${env.C3IAAS_NAMESPACE ?: env.PIPELINE_NAMESPACE}",

+                 '-p', "MBS_IMAGE_TAG=${env.TEMP_TAG}",

+                 '-p', "MBS_BACKEND_IMAGESTREAM_NAME=${params.MBS_BACKEND_IMAGESTREAM_NAME}",

+                 '-p', "MBS_BACKEND_IMAGESTREAM_NAMESPACE=${env.C3IAAS_NAMESPACE ?: env.PIPELINE_NAMESPACE}",

+                 '-p', "CREATED=${created}"

+               )

+               def build = c3i.buildAndWait(script: this, objs: processed, '--from-dir=.')

+               def ocpBuild = build.object()

+               env.FRONTEND_IMAGE_DIGEST = ocpBuild.status.output.to.imageDigest

+               def ref = ocpBuild.status.outputDockerImageReference

+               def repo = ref.tokenize(':')[0..-2].join(':')

+               env.FRONTEND_IMAGE_REPO = repo

+               env.FRONTEND_IMAGE_REF = repo + '@' + env.FRONTEND_IMAGE_DIGEST

+               env.FRONTEND_IMAGE_TAG = env.TEMP_TAG

+               echo "Built image ${env.FRONTEND_IMAGE_REF}, digest: ${env.FRONTEND_IMAGE_DIGEST}, tag: ${env.FRONTEND_IMAGE_TAG}"

+             }

            }

          }

        }
@@ -265,12 +286,14 @@ 

          }

          cleanup {

            script {

-             openshift.withCluster() {

-               echo 'Tearing down...'

-               openshift.selector('bc', [

-                 'app': env.FRONTEND_BUILDCONFIG_ID,

-                 'template': 'mbs-frontend-build-template',

-                 ]).delete()

+             if (!env.C3IAAS_NAMESPACE) {

+               openshift.withCluster() {

+                 echo 'Tearing down...'

+                 openshift.selector('bc', [

+                   'app': env.FRONTEND_BUILDCONFIG_ID,

+                   'template': 'mbs-frontend-build-template',

+                   ]).delete()

+               }

              }

            }

          }
@@ -281,13 +304,15 @@ 

          script {

            openshift.withCluster() {

              openshift.withProject(params.MBS_INTEGRATION_TEST_BUILD_CONFIG_NAMESPACE) {

-               def build = c3i.buildAndWait("bc/${params.MBS_INTEGRATION_TEST_BUILD_CONFIG_NAME}",

+               def build = c3i.buildAndWait(script: this, objs: "bc/${params.MBS_INTEGRATION_TEST_BUILD_CONFIG_NAME}",

                    '-e', "MBS_GIT_REPO=${params.MBS_GIT_REPO}",

                    '-e', "MBS_GIT_REF=${env.PR_NO ? params.MBS_GIT_REF : env.MBS_GIT_COMMIT}",

                    '-e', "MBS_BACKEND_IMAGE=${env.BACKEND_IMAGE_REF}",

                    '-e', "MBS_FRONTEND_IMAGE=${env.FRONTEND_IMAGE_REF}",

-                   '-e', "TEST_IMAGES='${env.BACKEND_IMAGE_REF} ${env.FRONTEND_IMAGE_REF}'",

+                   '-e', "TEST_IMAGES=${env.BACKEND_IMAGE_REF},${env.FRONTEND_IMAGE_REF}",

                    '-e', "IMAGE_IS_SCRATCH=${params.MBS_GIT_REF != params.MBS_MAIN_BRANCH}",

+                   // If env.C3IAAS_NAMESPACE has not been defined, tests will be run in the current namespace

+                   '-e', "TEST_NAMESPACE=${env.C3IAAS_NAMESPACE ?: ''}",

                    '-e', "TESTCASES='${params.TESTCASES}'",

                    '-e', "CLEANUP=${params.CLEANUP}"

                )
@@ -377,7 +402,8 @@ 

        when {

          expression {

            return "${params.MBS_DEV_IMAGE_TAG}" && params.TAG_INTO_IMAGESTREAM == "true" &&

-             (params.FORCE_PUBLISH_IMAGE == "true" || params.MBS_GIT_REF == params.MBS_MAIN_BRANCH)

+             (params.FORCE_PUBLISH_IMAGE == "true" || params.MBS_GIT_REF == params.MBS_MAIN_BRANCH) &&

+             !env.C3IAAS_NAMESPACE

          }

        }

        steps {
@@ -408,7 +434,7 @@ 

    post {

      cleanup {

        script {

-         if (params.CLEANUP == 'true') {

+         if (params.CLEANUP == 'true' && !env.C3IAAS_NAMESPACE) {

            openshift.withCluster() {

              if (env.BACKEND_IMAGE_TAG) {

                echo "Removing tag ${env.BACKEND_IMAGE_TAG} from the ${params.MBS_BACKEND_IMAGESTREAM_NAME} ImageStream..."
@@ -494,7 +520,6 @@ 

      }

    }

  }

- @NonCPS

  def getPrNo(branch) {

    def prMatch = branch =~ /^(?:.+\/)?pull\/(\d+)\/head$/

    return prMatch ? prMatch[0][1] : ''

@@ -1,7 +1,6 @@ 

  // Use scripted syntax because CIBuildTrigger currently doesn't support the declarative syntax

  properties([

    buildDiscarder(logRotator(numToKeepStr: '10')),

-   disableConcurrentBuilds(),

    pipelineTriggers([

      // example: https://github.com/jenkinsci/jms-messaging-plugin/blob/9b9387c3a52f037ba0d019c2ebcf2a2796fc6397/src/test/java/com/redhat/jenkins/plugins/ci/integration/AmqMessagingPluginIntegrationTest.java

      [$class: 'CIBuildTrigger',

@@ -72,7 +72,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial"

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      source:

        git:

@@ -85,7 +85,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial" # FIXME: Parallel is supported, but we have limited quota in UpShift.

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      source:

        git:

@@ -39,7 +39,6 @@ 

      timestamps()

      timeout(time: 30, unit: 'MINUTES')

      buildDiscarder(logRotator(numToKeepStr: '10'))

-     disableConcurrentBuilds()

      skipDefaultCheckout()

    }

    environment {

@@ -48,14 +48,15 @@ 

    displayName: Container image for Jenkins slave pods

    required: true

    value: quay.io/factory2/mbs-jenkins-slave:latest

- - name: CONTAINER_REGISTRY_CREDENTIALS

-   displayName: Secret name of container registries used for pulling and pushing images

-   value: factory2-pipeline-registry-credentials

-   required: false

  - name: JENKINS_AGENT_CLOUD_NAME

    displayName: Name of OpenShift cloud in Jenkins master configuration

    required: true

    value: openshift

+ - name: TEST_NAMESPACE

+   displayName: >-

+     Namespace where the Jenkins agent for this test will run, and where

+     test resources will be created.

+   required: false

  - name: ENVIRONMENT

    displayName: environment name (dev/stage/prod)

    required: true
@@ -98,7 +99,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial" # FIXME: Parallel is supported, but we have limited quota in UpShift.

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      source:

        git:
@@ -126,12 +127,12 @@ 

            value: "true"

          - name: "TEST_ID"

            value: ""

-         - name: CONTAINER_REGISTRY_CREDENTIALS

-           value: "${CONTAINER_REGISTRY_CREDENTIALS}"

          - name: JENKINS_AGENT_IMAGE

            value: "${JENKINS_AGENT_IMAGE}"

          - name: JENKINS_AGENT_CLOUD_NAME

            value: "${JENKINS_AGENT_CLOUD_NAME}"

+         - name: TEST_NAMESPACE

+           value: "${TEST_NAMESPACE}"

          - name: ENVIRONMENT

            value: "${ENVIRONMENT}"

          - name: MESSAGING_PROVIDER

@@ -1,5 +1,6 @@ 

  library identifier: 'c3i@master', changelog: false,

-   retriever: modernSCM([$class: "GitSCMSource", remote: "https://pagure.io/c3i-library.git"])

+   retriever: modernSCM([$class: 'GitSCMSource', remote: 'https://pagure.io/c3i-library.git'])

+ def deployments

  pipeline {

    agent {

      kubernetes {
@@ -21,12 +22,6 @@ 

            image: "${params.JENKINS_AGENT_IMAGE}"

            imagePullPolicy: Always

            tty: true

-           env:

-           - name: REGISTRY_CREDENTIALS

-             valueFrom:

-               secretKeyRef:

-                 name: "${params.CONTAINER_REGISTRY_CREDENTIALS}"

-                 key: ".dockerconfigjson"

            resources:

              requests:

                memory: 512Mi
@@ -41,12 +36,11 @@ 

      timestamps()

      timeout(time: 60, unit: 'MINUTES')

      buildDiscarder(logRotator(numToKeepStr: '10'))

-     disableConcurrentBuilds()

      skipDefaultCheckout()

    }

    environment {

      // Jenkins BUILD_TAG could be too long (> 63 characters) for OpenShift to consume

-     TEST_ID = "${params.TEST_ID ?: 'jenkins-' + currentBuild.id + '-' + UUID.randomUUID().toString().substring(0,7)}"

+     TEST_ID = "${params.TEST_ID ?: UUID.randomUUID().toString().substring(0,7)}"

    }

    stages {

      stage('Prepare') {
@@ -54,44 +48,36 @@ 

          script {

            // Don't set ENVIRONMENT_LABEL in the environment block! Otherwise you will get 2 different UUIDs.

            env.ENVIRONMENT_LABEL = "test-${env.TEST_ID}"

- 

-           def srcRef = params.MBS_GIT_REF.startsWith('pull/') ? params.MBS_GIT_REF : 'heads/master'

-           def localRef = params.MBS_GIT_REF.startsWith('pull/') ? params.MBS_GIT_REF : 'master'

-           def cloneDepth = params.MBS_GIT_REF.startsWith('pull/') ? 2 : 10

-           retry(5) {

-             // check out specified branch/commit

-             checkout([$class: 'GitSCM',

-               branches: [[name: params.MBS_GIT_REF]],

-               userRemoteConfigs: [

-                 [

-                   name: 'origin',

-                   url: params.MBS_GIT_REPO,

-                   refspec: "+refs/${srcRef}:refs/remotes/origin/${localRef}",

-                 ],

-               ],

-               extensions: [

-                 [$class: 'CleanBeforeCheckout'],

-                 [$class: 'CloneOption', noTags: true, shallow: true, depth: cloneDepth, honorRefspec: true],

-               ],

-             ])

-           }

- 

+           // MBS_GIT_REF can be either a regular branch (in the heads/ namespace), a pull request

+           // branch (in the pull/ namespace), or a full 40-character sha1, which is assumed to

+           // exist on the master branch.

+           def branch = params.MBS_GIT_REF ==~ '[0-9a-f]{40}' ? 'master' : params.MBS_GIT_REF

+           c3i.clone(repo: params.MBS_GIT_REPO, branch: branch, rev: params.MBS_GIT_REF)

            // get current commit ID

            // FIXME: Due to a bug discribed in https://issues.jenkins-ci.org/browse/JENKINS-45489,

            // the return value of checkout() is unreliable.

            // Not working: env.MBS_GIT_COMMIT = scmVars.GIT_COMMIT

            env.MBS_GIT_COMMIT = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()

-           echo "Running integration tests for ${params.MBS_GIT_REF}, commit=${env.MBS_GIT_COMMIT}"

- 

-           currentBuild.displayName = "${params.MBS_GIT_REF}: ${env.MBS_GIT_COMMIT.take(7)}"

+           echo "Running integration tests for ${branch}, commit=${env.MBS_GIT_COMMIT}"

+           currentBuild.displayName = "${branch}: ${env.MBS_GIT_COMMIT.take(7)}"

          }

        }

      }

-     stage('Call cleanup routine') {

+     stage('Cleanup') {

+       when {

+         expression {

+           // Only run cleanup if we're running tests in the default namespace.

+           return !params.TEST_NAMESPACE

+         }

+       }

        steps {

          script {

-           // Cleanup all test environments that were created 1 hour ago in case of failures of previous cleanups.

-           c3i.cleanup('umb', 'koji', 'mbs')

+           openshift.withCluster() {

+             openshift.withProject() {

+               // Cleanup all test environments that were created 1 hour ago in case of failures of previous cleanups.

+               c3i.cleanup(script: this, 'umb', 'koji', 'mbs')

+             }

+           }

          }

        }

        post {
@@ -100,59 +86,149 @@ 

          }

        }

      }

-     stage('Call UMB deployer') {

+     stage('Route suffix') {

        steps {

          script {

-           def keystore = ca.get_keystore("umb-${TEST_ID}-broker", 'mbskeys')

-           def truststore = ca.get_truststore('mbstrust')

-           umb.deploy(env.TEST_ID, keystore, 'mbskeys', truststore, 'mbstrust',

-                      params.UMB_IMAGE)

+           openshift.withCluster() {

+             openshift.withProject(params.TEST_NAMESPACE) {

+               def testroute = openshift.create('route', 'edge', 'test',  '--service=test', '--port=8080')

+               def testhost = testroute.object().spec.host

+               // trim off the test- prefix

+               env.ROUTE_SUFFIX = testhost.drop(5)

+               testroute.delete()

+             }

+           }

+         }

+       }

+       post {

+         success {

+           echo "Routes end with ${env.ROUTE_SUFFIX}"

+         }

+       }

+     }

+     stage('Generate CA') {

+       steps {

+         script {

+           ca.gen_ca()

+         }

+       }

+     }

+     stage('Deploy UMB') {

+       steps {

+         script {

+           openshift.withCluster() {

+             openshift.withProject(params.TEST_NAMESPACE) {

+               // The extact hostname doesn't matter, (as long as it resolves to the cluster) because traffic will

+               // be routed to the pod via the NodePort.

+               // However, the hostname we use to access the service must be a subjectAltName of the certificate

+               // being served by the service.

+               env.UMB_HOST = "umb-${TEST_ID}-${env.ROUTE_SUFFIX}"

+               ca.gen_ssl_cert("umb-${TEST_ID}-broker", env.UMB_HOST)

+               def keystore = ca.get_keystore("umb-${TEST_ID}-broker", 'mbskeys')

+               def truststore = ca.get_truststore('mbstrust')

+               deployments = umb.deploy(script: this, test_id: env.TEST_ID,

+                                        keystore_data: keystore, keystore_password: 'mbskeys',

+                                        truststore_data: truststore, truststore_password: 'mbstrust',

+                                        broker_image: params.UMB_IMAGE)

+               def ports = openshift.selector('service', "umb-${TEST_ID}-broker").object().spec.ports

+               env.UMB_AMQPS_PORT = ports.find { it.name == 'amqps' }.nodePort

+               env.UMB_STOMP_SSL_PORT = ports.find { it.name == 'stomp-ssl' }.nodePort

+             }

+           }

          }

        }

        post {

+         success {

+           echo "UMB deployed: amqps: ${env.UMB_HOST}:${env.UMB_AMQPS_PORT} stomp-ssl: ${env.UMB_HOST}:${env.UMB_STOMP_SSL_PORT}"

+         }

          failure {

            echo "UMB deployment FAILED"

          }

        }

      }

-     stage('Call Koji deployer') {

+     stage('Deploy Koji') {

        steps {

          script {

-           koji.deploy(env.TEST_ID, ca.get_ca_cert(),

-                       ca.get_ssl_cert("koji-${TEST_ID}-hub"),

-                       "amqps://umb-${TEST_ID}-broker",

-                       ca.get_ssl_cert("koji-${TEST_ID}-msg"),

-                       "mbs-${TEST_ID}-koji-admin",

-                       params.KOJI_IMAGE)

+           openshift.withCluster() {

+             openshift.withProject(params.TEST_NAMESPACE) {

+               env.KOJI_SSL_HOST = "koji-${TEST_ID}-hub-${env.ROUTE_SUFFIX}"

+               def hubcert = ca.get_ssl_cert("koji-${TEST_ID}-hub", env.KOJI_SSL_HOST)

+               env.KOJI_ADMIN = "mbs-${TEST_ID}-koji-admin"

+               env.KOJI_MSG_CERT = "koji-${TEST_ID}-msg"

+               def deployed = koji.deploy(script: this, test_id: env.TEST_ID,

+                                          hubca: ca.get_ca_cert(), hubcert: hubcert,

+                                          brokerurl: "amqps://${env.UMB_HOST}:${env.UMB_AMQPS_PORT}",

+                                          brokercert: ca.get_ssl_cert(env.KOJI_MSG_CERT),

+                                          admin_user: env.KOJI_ADMIN,

+                                          hub_image: params.KOJI_IMAGE)

+               deployments = deployments.union(deployed)

+             }

+           }

          }

        }

        post {

+         success {

+           echo "Koji deployed: hub: https://${env.KOJI_SSL_HOST}/"

+         }

          failure {

            echo "Koji deployment FAILED"

          }

        }

      }

-     stage('Call MBS deployer') {

+     stage('Deploy MBS') {

        steps {

          script {

+           env.MBS_SSL_HOST = "mbs-${TEST_ID}-frontend-${env.ROUTE_SUFFIX}"

+           def frontendcert = ca.get_ssl_cert("mbs-${TEST_ID}-frontend", env.MBS_SSL_HOST)

            // Required for accessing src.fedoraproject.org

            def digicertca = readFile file: 'openshift/integration/koji/resources/certs/DigiCertHighAssuranceEVRootCA.pem'

            def cabundle = ca.get_ca_cert().cert + digicertca

            def msgcert = ca.get_ssl_cert("mbs-${TEST_ID}-msg")

-           def frontendcert = ca.get_ssl_cert("mbs-${TEST_ID}-frontend")

-           def kojicert = ca.get_ssl_cert("mbs-${TEST_ID}-koji-admin")

-           mbs.deploy(env.TEST_ID, kojicert, ca.get_ca_cert(), msgcert, frontendcert, ca.get_ca_cert(), cabundle,

-                      "https://koji-${TEST_ID}-hub",

-                      "umb-${TEST_ID}-broker:61612",

-                      params.MBS_BACKEND_IMAGE, params.MBS_FRONTEND_IMAGE)

+           def kojicert = ca.get_ssl_cert(env.KOJI_ADMIN)

+           openshift.withCluster() {

+             openshift.withProject(params.TEST_NAMESPACE) {

+               def deployed = mbs.deploy(script: this, test_id: env.TEST_ID,

+                                         kojicert: kojicert, kojica: ca.get_ca_cert(),

+                                         brokercert: msgcert,

+                                         frontendcert: frontendcert, frontendca: ca.get_ca_cert(),

+                                         cacerts: cabundle,

+                                         kojiurl: "https://${env.KOJI_SSL_HOST}",

+                                         stompuri: "${env.UMB_HOST}:${env.UMB_STOMP_SSL_PORT}",

+                                         backend_image: params.MBS_BACKEND_IMAGE,

+                                         frontend_image: params.MBS_FRONTEND_IMAGE)

+               deployments = deployments.union(deployed)

+             }

+           }

          }

        }

        post {

+         success {

+           echo "MBS deployed: frontend: https://${env.MBS_SSL_HOST}/"

+         }

          failure {

            echo "MBS deployment FAILED"

          }

        }

      }

+     stage('Wait for deployments') {

+       steps {

+         script {

+           openshift.withCluster() {

+             openshift.withProject(params.TEST_NAMESPACE) {

+               c3i.waitForDeployment(script: this, objs: deployments)

+             }

+           }

+         }

+       }

+       post {

+         success {

+           echo "Deployments complete"

+         }

+         failure {

+           echo 'Deployments FAILED'

+         }

+       }

+     }

      stage('Run tests') {

        steps {

          script {
@@ -177,10 +253,12 @@ 

              def test = load "openshift/integration/koji/pipelines/tests/${testcase}.groovy"

              test.runTests()

            }

-           echo "Tests complete"

          }

        }

        post {

+         success {

+           echo "All tests successful"

+         }

          failure {

            echo "Testcase ${env.CURRENT_TESTCASE} FAILED"

          }
@@ -190,32 +268,37 @@ 

    post {

      success {

        script {

-         params.TEST_IMAGES.split().each {

+         params.TEST_IMAGES.split(',').each {

            sendToResultsDB(it, 'passed')

          }

        }

      }

      failure {

        script {

-         params.TEST_IMAGES.split().each {

+         params.TEST_IMAGES.split(',').each {

            sendToResultsDB(it, 'failed')

          }

          openshift.withCluster() {

-           echo 'Getting logs from all deployments...'

-           def sel = openshift.selector('dc', ['environment': env.ENVIRONMENT_LABEL])

-           sel.logs('--tail=100')

+           openshift.withProject(params.TEST_NAMESPACE) {

+             if (deployments) {

+               echo 'Getting logs from all deployments...'

+               deployments.logs('--tail=100')

+             }

+           }

          }

        }

      }

      cleanup {

        script {

-         if (params.CLEANUP == 'true') {

+         if (params.CLEANUP == 'true' && !params.TEST_NAMESPACE) {

            openshift.withCluster() {

              /* Tear down everything we just created */

              echo 'Tearing down test resources...'

              openshift.selector('all,pvc,configmap,secret',

                                 ['environment': env.ENVIRONMENT_LABEL]).delete('--ignore-not-found=true')

            }

+         } else {

+           echo 'Skipping cleanup'

          }

        }

      }

@@ -83,7 +83,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial"

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      strategy:

        type: JenkinsPipeline
@@ -127,7 +127,6 @@ 

                timestamps()

                timeout(time: 60, unit: 'MINUTES')

                buildDiscarder(logRotator(numToKeepStr: '10'))

-               disableConcurrentBuilds()

                skipDefaultCheckout()

              }

              environment {
@@ -148,26 +147,10 @@ 

                  agent { label 'master' }

                  steps {

                    script {

-                     // checking out the polled branch

-                     def polledBranch = env.PAGURE_POLLING_FOR_PR == 'true' ? 'origin/pull/*/head' : "origin/${PAGURE_POLLED_BRANCH}"

-                     def srcRef = env.PAGURE_POLLING_FOR_PR == 'true' ? 'pull/*/head' : "heads/${PAGURE_POLLED_BRANCH}"

-                     def scmVars

-                     retry(5) {

-                       scmVars = checkout([$class: 'GitSCM',

-                         branches: [[name: polledBranch]],

-                         userRemoteConfigs: [

-                           [

-                             name: 'origin',

-                             url: env.GIT_URL,

-                             refspec: "+refs/${srcRef}:refs/remotes/${polledBranch}",

-                           ],

-                         ],

-                         extensions: [

-                           [$class: 'CleanBeforeCheckout'],

-                           [$class: 'CloneOption', noTags: true, shallow: true, depth: 2, honorRefspec: true],

-                         ],

-                       ])

-                     }

+                     def polled = env.PAGURE_POLLING_FOR_PR == 'true' ? 'pull/*/head' : "${PAGURE_POLLED_BRANCH}"

+                     // Need to prefix the rev with origin/ for pollSCM to work correctly

+                     def rev = "origin/${polled}"

+                     def scmVars = c3i.clone(repo: env.GIT_URL, branch: polled, rev: rev)

                      env.MBS_GIT_COMMIT = scmVars.GIT_COMMIT

                      // setting build display name

                      def prefix = 'origin/'
@@ -207,23 +190,8 @@ 

                    }

                  }

                  steps {

-                   retry(5) {

-                     checkout([$class: 'GitSCM',

-                       branches: [[name: env.MBS_GIT_BRANCH]],

-                       userRemoteConfigs: [

-                         [

-                           name: 'origin',

-                           url: env.GIT_URL,

-                           refspec: "+refs/heads/${env.MBS_GIT_BRANCH}:refs/remotes/origin/${env.MBS_GIT_BRANCH}",

-                         ],

-                       ],

-                       extensions: [

-                         [$class: 'CleanBeforeCheckout'],

-                         [$class: 'CloneOption', noTags: true, shallow: true, depth: 2, honorRefspec: true],

-                       ],

-                     ])

-                   }

                    script {

+                     c3i.clone(repo: env.GIT_URL, branch: env.MBS_GIT_BRANCH)

                      dir('openshift/integration/koji/pipelines') {

                        sh '''

                        make install JOBS_DIR="${PIPELINE_UPDATE_JOBS_DIR}"
@@ -241,15 +209,12 @@ 

                  steps {

                    script {

                      openshift.withCluster() {

-                       def bcSelector = openshift.selector('bc',

-                         env.PAGURE_POLLING_FOR_PR == 'true' ? env.PREMERGE_JOB_NAME : env.POSTMERGE_JOB_NAME)

                        echo 'Starting a MBS build run...'

-                       def devBuild = bcSelector.startBuild(

-                         '-e', "MBS_GIT_REF=${env.MBS_GIT_BRANCH}",

+                       def devBuild = c3i.build(script: this,

+                         objs: "bc/${env.PAGURE_POLLING_FOR_PR == 'true' ? env.PREMERGE_JOB_NAME : env.POSTMERGE_JOB_NAME}",

+                         '-e', "MBS_GIT_REF=${env.MBS_GIT_BRANCH}"

                        )

-                       devBuild.watch {

-                         return !(it.object().status.phase in ["New", "Pending"])

-                       }

+                       c3i.waitForBuildStart(script: this, build: devBuild)

                        def devBuildInfo = devBuild.object()

                        def downstreamBuildName = devBuildInfo.metadata.name

                        def downstreamBuildUrl = devBuildInfo.metadata.annotations['openshift.io/jenkins-build-uri']

@@ -1,7 +1,6 @@ 

  // Use scripted syntax because CIBuildTrigger currently doesn't support the declarative syntax

  properties([

    buildDiscarder(logRotator(numToKeepStr: '10')),

-   disableConcurrentBuilds(),

    pipelineTriggers([

      // example: https://github.com/jenkinsci/jms-messaging-plugin/blob/9b9387c3a52f037ba0d019c2ebcf2a2796fc6397/src/test/java/com/redhat/jenkins/plugins/ci/integration/AmqMessagingPluginIntegrationTest.java

      [$class: 'CIBuildTrigger',
@@ -18,13 +17,13 @@ 

    echo 'This build is not started by a CI message. Only configurations were done.'

    return

  }

- 

+ library identifier: 'c3i@master', changelog: false,

+   retriever: modernSCM([$class: 'GitSCMSource', remote: 'https://pagure.io/c3i-library.git'])

  def label = "jenkins-slave-${UUID.randomUUID().toString()}"

  podTemplate(

    cloud: "${params.JENKINS_AGENT_CLOUD_NAME}",

    label: label,

    serviceAccount: "${env.JENKINS_AGENT_SERVICE_ACCOUNT}",

-   defaultContainer: 'jnlp',

    yaml: """

      apiVersion: v1

      kind: Pod
@@ -55,6 +54,27 @@ 

      """

  ) {

    node(label) {

+     stage('Allocate C3IaaS project') {

+       if (!(params.USE_C3IAAS == 'true' &&

+             params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE &&

+             params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME)) {

+         echo "Not configured to use C3IaaS"

+         return

+       }

+       env.PIPELINE_NAMESPACE = readFile("/run/secrets/kubernetes.io/serviceaccount/namespace").trim()

+       env.C3IAAS_NAMESPACE = "c3i-mbs-tr-${params.TRACKED_TAG}-${env.BUILD_NUMBER}"

+       echo "Requesting new OpenShift project ${env.C3IAAS_NAMESPACE}..."

+       openshift.withCluster() {

+         openshift.withProject(params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE) {

+           c3i.buildAndWait(script: this, objs: "bc/${params.C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME}",

+             '-e', "PROJECT_NAME=${env.C3IAAS_NAMESPACE}",

+             '-e', "ADMIN_GROUPS=system:serviceaccounts:${env.PIPELINE_NAMESPACE}",

+             '-e', "LIFETIME_IN_MINUTES=${params.C3IAAS_LIFETIME}"

+           )

+         }

+       }

+       echo "Allocated project ${env.C3IAAS_NAMESPACE}"

+     }

      stage('Trigger tests') {

        def message = readJSON text: params.CI_MESSAGE

        echo "Tag :${message.tag} is ${message.action} in ${message.repo}. New digest: ${message.digest}"
@@ -73,17 +93,15 @@ 

        echo "Triggering a job to test if ${frontendImage} and ${backendImage} meet all criteria of desired tag"

        openshift.withCluster() {

          openshift.withProject(params.TEST_JOB_NAMESPACE) {

-           def testBcSelector = openshift.selector('bc', params.TEST_JOB_NAME)

-           def buildSelector = testBcSelector.startBuild(

+           def build = c3i.build(script: this, objs: "bc/${params.TEST_JOB_NAME}",

              '-e', "MBS_BACKEND_IMAGE=${backendImage}",

              '-e', "MBS_FRONTEND_IMAGE=${frontendImage}",

-             '-e', "TEST_IMAGES='${backendImage} ${frontendImage}'",

-             '-e', "IMAGE_IS_SCRATCH=false"

+             '-e', "TEST_IMAGES=${backendImage},${frontendImage}",

+             '-e', "IMAGE_IS_SCRATCH=false",

+             '-e', "TEST_NAMESPACE=${env.C3IAAS_NAMESPACE ?: ''}",

            )

-           buildSelector.watch {

-             return !(it.object().status.phase in ["New", "Pending"])

-           }

-           buildInfo = buildSelector.object()

+           c3i.waitForBuildStart(script: this, build: build)

+           buildInfo = build.object()

            echo "Build ${buildInfo.metadata.annotations['openshift.io/jenkins-build-uri'] ?: buildInfo.metadata.name} started."

          }

        }

@@ -51,6 +51,24 @@ 

  - name: TEST_JOB_NAMESPACE

    displayName: Namespace in which to trigger the integration test job

    required: false

+ - name: USE_C3IAAS

+   displayName: >-

+     Use C3I-as-a-Service to dynamically allocate a temporary OpenShift project for building

+     and testing.

+   required: true

+   value: "true"

+ - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE

+   displayName: The namespace where the C3I-as-a-Service project request BuildConfig has been defined

+   required: false

+   value: c3i

+ - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME

+   displayName: The name of the C3I-as-a-Service project request BuildConfig

+   required: false

+   value: c3iaas-request-project

+ - name: C3IAAS_LIFETIME

+   displayName: The lifetime of the OpenShift project allocated by C3I-as-a-Service.

+   required: true

+   value: "120"

  objects:

  - kind: ServiceAccount

    apiVersion: v1
@@ -76,7 +94,7 @@ 

      labels:

        app: "${NAME}"

    spec:

-     runPolicy: "Serial"

+     runPolicy: "Parallel"

      completionDeadlineSeconds: 1800

      source:

        git:
@@ -104,6 +122,14 @@ 

            value: "${TEST_JOB_NAME}"

          - name: TEST_JOB_NAMESPACE

            value: "${TEST_JOB_NAMESPACE}"

+         - name: USE_C3IAAS

+           value: "${USE_C3IAAS}"

+         - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE

+           value: "${C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAMESPACE}"

+         - name: C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME

+           value: "${C3IAAS_REQUEST_PROJECT_BUILD_CONFIG_NAME}"

+         - name: C3IAAS_LIFETIME

+           value: "${C3IAAS_LIFETIME}"

          - name: MESSAGING_PROVIDER

            value: "${MESSAGING_PROVIDER}"

          - name: MESSAGING_TOPIC

@@ -2,8 +2,8 @@ 

  import groovy.json.JsonOutput

  

  def runTests() {

-   def clientcert = ca.get_ssl_cert("mbs-${TEST_ID}-koji-admin")

-   koji.setConfig("https://koji-${TEST_ID}-hub/kojihub", "https://koji-${TEST_ID}-hub/kojifiles",

+   def clientcert = ca.get_ssl_cert(env.KOJI_ADMIN)

+   koji.setConfig("https://${env.KOJI_SSL_HOST}/kojihub", "https://${env.KOJI_SSL_HOST}/kojifiles",

                   clientcert.cert, clientcert.key, ca.get_ca_cert().cert)

    def tags = koji.callMethod("listTags")

    if (!tags.any { it.name == "module-f28" }) {
@@ -15,9 +15,9 @@ 

    try {

      // There's currently no way to query whether a given user has CG access, so just add it

      // and hope no one else has already done it.

-     koji.runCmd("grant-cg-access", "mbs-${TEST_ID}-koji-admin", "module-build-service", "--new")

+     koji.runCmd("grant-cg-access", env.KOJI_ADMIN, "module-build-service", "--new")

    } catch (ex) {

-     echo "Granting cg-access to mbs-${TEST_ID}-koji-admin failed, assuming it was already provided in a previous test"

+     echo "Granting cg-access to ${env.KOJI_ADMIN} failed, assuming it was already provided in a previous test"

    }

  

    if (!koji.callMethod("listBTypes").any { it.name == "module" }) {
@@ -44,10 +44,10 @@ 

        documentation: https://fedoraproject.org/wiki/Fedora_Packaging_Guidelines_for_Modules

    """

  

-   def buildparams = groovy.json.JsonOutput.toJson([modulemd: testmodule, owner: "mbs-${TEST_ID}-koji-admin"])

+   def buildparams = groovy.json.JsonOutput.toJson([modulemd: testmodule, owner: env.KOJI_ADMIN])

    def resp = httpRequest(

          httpMode: "POST",

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/",

          acceptType: "APPLICATION_JSON",

          contentType: "APPLICATION_JSON",

          requestBody: buildparams,
@@ -61,7 +61,7 @@ 

    timeout(10) {

      waitUntil {

        resp = httpRequest(

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/${buildinfo.id}",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/${buildinfo.id}",

          ignoreSslErrors: true,

        )

        if (resp.status != 200) {

@@ -1,8 +1,8 @@ 

  // Build an empty module and verify that the CGImport works correctly

  

  def runTests() {

-   def clientcert = ca.get_ssl_cert("mbs-${TEST_ID}-koji-admin")

-   koji.setConfig("https://koji-${TEST_ID}-hub/kojihub", "https://koji-${TEST_ID}-hub/kojifiles",

+   def clientcert = ca.get_ssl_cert(env.KOJI_ADMIN)

+   koji.setConfig("https://${env.KOJI_SSL_HOST}/kojihub", "https://${env.KOJI_SSL_HOST}/kojifiles",

                   clientcert.cert, clientcert.key, ca.get_ca_cert().cert)

    def tags = koji.callMethod("listTags")

    if (!tags.any { it.name == "module-f28" }) {
@@ -14,9 +14,9 @@ 

    try {

      // There's currently no way to query whether a given user has CG access, so just add it

      // and hope no one else has already done it.

-     koji.runCmd("grant-cg-access", "mbs-${TEST_ID}-koji-admin", "module-build-service", "--new")

+     koji.runCmd("grant-cg-access", env.KOJI_ADMIN, "module-build-service", "--new")

    } catch (ex) {

-     echo "Granting cg-access to mbs-${TEST_ID}-koji-admin failed, assuming it was already provided in a previous test"

+     echo "Granting cg-access to ${env.KOJI_ADMIN} failed, assuming it was already provided in a previous test"

    }

  

    if (!koji.callMethod("listBTypes").any { it.name == "module" }) {
@@ -26,11 +26,11 @@ 

    def buildparams = """

          {"scmurl": "https://src.fedoraproject.org/forks/mikeb/modules/testmodule.git?#8b3fb16160f899ce10905faf570f110d52b91154",

           "branch": "empty-f28",

-          "owner":  "mbs-${TEST_ID}-koji-admin"}

+          "owner":  "${env.KOJI_ADMIN}"}

        """

    def resp = httpRequest(

          httpMode: "POST",

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/",

          acceptType: "APPLICATION_JSON",

          contentType: "APPLICATION_JSON",

          requestBody: buildparams,
@@ -44,7 +44,7 @@ 

    timeout(10) {

      waitUntil {

        resp = httpRequest(

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/${buildinfo.id}",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/${buildinfo.id}",

          ignoreSslErrors: true,

        )

        if (resp.status != 200) {

@@ -1,8 +1,8 @@ 

  // Submit a build to MBS and verify that it initializes Koji correctly

  

  def runTests() {

-   def clientcert = ca.get_ssl_cert("mbs-${TEST_ID}-koji-admin")

-   koji.setConfig("https://koji-${TEST_ID}-hub/kojihub", "https://koji-${TEST_ID}-hub/kojifiles",

+   def clientcert = ca.get_ssl_cert(env.KOJI_ADMIN)

+   koji.setConfig("https://${env.KOJI_SSL_HOST}/kojihub", "https://${env.KOJI_SSL_HOST}/kojifiles",

                   clientcert.cert, clientcert.key, ca.get_ca_cert().cert)

    def tags = koji.callMethod("listTags")

    if (!tags.any { it.name == "module-f28" }) {
@@ -14,11 +14,11 @@ 

    def buildparams = """

          {"scmurl": "https://src.fedoraproject.org/modules/testmodule.git?#9c589780e1dd1698dc64dfa28d30014ad18cad32",

           "branch": "f28",

-          "owner":  "mbs-${TEST_ID}-koji-admin"}

+          "owner":  "${env.KOJI_ADMIN}"}

        """

    def resp = httpRequest(

          httpMode: "POST",

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/",

          acceptType: "APPLICATION_JSON",

          contentType: "APPLICATION_JSON",

          requestBody: buildparams,
@@ -33,7 +33,7 @@ 

    timeout(10) {

      waitUntil {

        resp = httpRequest(

-         url: "https://mbs-${TEST_ID}-frontend/module-build-service/1/module-builds/${buildinfo.id}",

+         url: "https://${env.MBS_SSL_HOST}/module-build-service/1/module-builds/${buildinfo.id}",

          ignoreSslErrors: true,

        )

        if (resp.status != 200) {

The C3I-as-a-Service system
enables creation of new OpenShift projects, which will be deleted after a
configurable period of time, and are suitable for building and testing proposed
changes. The MBS-Koji integration tests now make use of C3IaaS, to reduce resource
contention and avoid false failures due to quota limitations.

This PR also enables parallel job execution (possible now because of C3IaaS) which
will reduce the lag time for feedback on PRs.

Jenkins pipelines are now making use of the new c3i-library
API to reduce code duplication.

Build #481 failed (commit: 78ee32b).
Rebase or make new commits to rebuild.

CI failures are expected because it's running the old pipelines, which are not compatible with the new c3i-library API.

Does this need to be a parameter? It seems like it wouldn't work if this were false.

Could we just require C3IAAS and remove this branch of code?

Could we just require C3IAAS and remove this branch of code?

We could. The intent was to still make it possible to run the pipelines in a more standard OpenShift environment, where C3I wasn't available. This would facilitate (for example) local testing.

Does this need to be a parameter? It seems like it wouldn't work if this were false.

It's checked in the when expression of the Allocate C3iaaS project stage. If it's not the string true then no C3IaaS project is allocated, and all services are deployed in the current project, as they always have been.

@mikeb I took a quick scan and it looks okay. My only suggestion to only support C3IAAS to reduce the code since we won't ever not use it.

Commit 5fc3e0a fixes this pull-request

Pull-Request has been merged by mprahl

4 years ago

Pull-Request has been merged by mprahl

4 years ago