PR#295: CI/CD: repotracker trigger enhancement - waiverdb

waiverdb

#295 CI/CD: repotracker trigger enhancement

Merged 5 years ago by mikeb. Opened 5 years ago by rayson.

rayson/waiverdb cicd-customize-message-provider into master

CI/CD: repotracker trigger enhancement

Yuxiang Zhu • 5 years ago

3f0d0d8

openshift/pipelines/jobs/waiverdb-repotracker-trigger.env

file removed

-1

		`@@ -1,1 +0,0 @@`
		`- NAME=waiverdb-repotracker-trigger`

openshift/pipelines/jobs/waiverdb-trigger-on-latest-tag.env

file added

		`@@ -0,0 +1,4 @@`
		`+ NAME=waiverdb-trigger-on-latest-tag`
		`+ MESSAGING_TOPIC=Consumer.rh-jenkins-ci-plugin.c3i-waiverdb-trigger-on-latest-tag.VirtualTopic.eng.repotracker.container.tag.>`
		`+ TEST_JOB_NAME=waiverdb-stage-integration-test`
		`+ TRACKED_TAG=latest`

openshift/pipelines/jobs/waiverdb-trigger-on-latest-tag.tmpl ~~openshift/pipelines/jobs/waiverdb-repotracker-trigger.tmpl~~

file renamed

file was moved with no change to the file

openshift/pipelines/jobs/waiverdb-trigger-on-stage-tag.env

file added

		`@@ -0,0 +1,4 @@`
		`+ NAME=waiverdb-trigger-on-stage-tag`
		`+ MESSAGING_TOPIC=Consumer.rh-jenkins-ci-plugin.c3i-waiverdb-trigger-on-stage-tag.VirtualTopic.eng.repotracker.container.tag.>`
		`+ TEST_JOB_NAME=waiverdb-prod-integration-test`
		`+ TRACKED_TAG=stage`

openshift/pipelines/jobs/waiverdb-trigger-on-stage-tag.tmpl

file added

		`@@ -0,0 +1,1 @@`
		`+ waiverdb-repotracker-trigger.yaml`

openshift/pipelines/templates/waiverdb-repotracker-trigger.Jenkinsfile

file modified

+6 -30

		`@@ -5,27 +5,12 @@`
		`// example: https://github.com/jenkinsci/jms-messaging-plugin/blob/9b9387c3a52f037ba0d019c2ebcf2a2796fc6397/src/test/java/com/redhat/jenkins/plugins/ci/integration/AmqMessagingPluginIntegrationTest.java`
		`[$class: 'CIBuildTrigger',`
		`providerData: [$class: 'ActiveMQSubscriberProviderData',`
		`- name: 'Red Hat UMB',`
		`- overrides: [topic: "Consumer.rh-jenkins-ci-plugin.c3i-${env.JOB_BASE_NAME}.VirtualTopic.eng.repotracker.container.tag.>"],`
		`- selector: "repo='${TRACKED_CONTAINER_REPO}'",`
		`- checks: [`
		`- [field: '$.action', expectedValue: 'added\|updated'],`
		`- ],`
		`+ name: params.MESSAGING_PROVIDER,`
		`+ overrides: [topic: params.MESSAGING_TOPIC],`
		`+ selector: "repo = '${params.TRACKED_CONTAINER_REPO}' AND action IN ('added', 'updated') AND tag = '${params.TRACKED_TAG}'",`
		`],`
		`],`
		`]),`
		`- parameters([`
		`- stringParam(`
		`- defaultValue: '',`
		`- description: '(Used by CIBuildTrigger internally)',`
		`- name: 'CI_MESSAGE'`
		`- ),`
		`- stringParam(`
		`- defaultValue: '',`
		`- description: '(Used by CIBuildTrigger internally)',`
		`- name: 'MESSAGE_HEADERS'`
		`- ),`
		`- ]),`
		`])`

		`if (!params.CI_MESSAGE) {`
		`@@ -64,22 +49,13 @@`
		`) {`
		`node(label) {`
		`stage('trigger test') {`
		`- def promotionMap = [`
		`- 'latest': 'stage',`
		`- 'stage': 'prod',`
		`- ]`
		`def message = readJSON text: params.CI_MESSAGE`
		`- def changedTag = message.tag`
		`- echo "Tag :$changedTag is ${message.action} in ${message.repo}. New digest: ${message.digest}"`
		`- def promoteTo = promotionMap[changedTag]`
		`- if (!promoteTo) {`
		`- error("I don't know where to promote :$changedTag. Aborting...")`
		`- }`
		`+ echo "Tag :${message.tag} is ${message.action} in ${message.repo}. New digest: ${message.digest}"`
		`def image = "${message.repo}@${message.digest}"`
		`- echo "Triggering a job to test if $image meets all criteria of desired tag :${promoteTo}"`
		`+ echo "Triggering a job to test if $image meets all criteria of desired tag :${message.tag}"`
		`def buildInfo = null`
		`openshift.withCluster() {`
		`- def testBcSelector = openshift.selector('bc', "waiverdb-${promoteTo}-integration-test")`
		`+ def testBcSelector = openshift.selector('bc', params.TEST_JOB_NAME)`
		`def buildSelector = testBcSelector.startBuild(`
		`'-e', "IMAGE=${image}",`
		`)`

openshift/pipelines/templates/waiverdb-repotracker-trigger.yaml

file modified

+25 -4

		`@@ -23,12 +23,24 @@`
		`- name: TRACKED_CONTAINER_REPO`
		`displayName: Container repo to be tracked`
		`value: "quay.io/factory2/waiverdb"`
		`+ - name: TRACKED_TAG`
		`+ displayName: Name of tag to be tracked`
		`+ required: true`
		`- name: JENKINS_AGENT_IMAGE`
		`displayName: Container image for Jenkins slave pods`
		`value: docker-registry.engineering.redhat.com/factory2/waiverdb-jenkins-slave:latest`
		`- name: OPENSHIFT_CLOUD_NAME`
		`displayName: Name of OpenShift cloud in Jenkins master configuration`
		`value: openshift`
		`+ - name: MESSAGING_PROVIDER`
		`+ displayName: Name of the JMS messaging provider`
		`+ value: Red Hat UMB`
		`+ - name: MESSAGING_TOPIC`
		`+ displayName: Name of the topic that the trigger subscribes to`
		`+ value: "Consumer.rh-jenkins-ci-plugin.c3i-waiverdb-repotracker-trigger.VirtualTopic.eng.repotracker.container.tag.>"`
		`+ - name: TEST_JOB_NAME`
		`+ displayName: Name of integration test job to trigger`
		`+ required: true`
		`objects:`
		`- kind: ServiceAccount`
		`apiVersion: v1`
		`@@ -64,10 +76,6 @@`
		`type: JenkinsPipeline`
		`jenkinsPipelineStrategy:`
		`env:`
		`- - name: "WAIVERDB_GIT_REPO"`
		`- value: "${WAIVERDB_GIT_REPO}"`
		`- - name: "WAIVERDB_GIT_REF"`
		`- value: "${WAIVERDB_GIT_REF}"`
		`- name: "OPENSHIFT_CLOUD_NAME"`
		`value: "${OPENSHIFT_CLOUD_NAME}"`
		`- name: "JENKINS_AGENT_IMAGE"`
		`@@ -76,4 +84,17 @@`
		`value: "${NAME}-jenkins-slave"`
		`- name: "TRACKED_CONTAINER_REPO"`
		`value: "${TRACKED_CONTAINER_REPO}"`
		`+ - name: "TRACKED_TAG"`
		`+ value: "${TRACKED_TAG}"`
		`+ - name: "TEST_JOB_NAME"`
		`+ value: "${TEST_JOB_NAME}"`
		`+ - name: MESSAGING_PROVIDER`
		`+ value: "${MESSAGING_PROVIDER}"`
		`+ - name: MESSAGING_TOPIC`
		`+ value: "${MESSAGING_TOPIC}"`
		`+ # CI_MESSAGE and MESSAGE_HEADERS are used internally by JMS messaging plugin`
		`+ - name: CI_MESSAGE`
		`+ value:`
		`+ - name: MESSAGE_HEADERS`
		`+ value:`
		`jenkinsfilePath: openshift/pipelines/templates/waiverdb-repotracker-trigger.Jenkinsfile`

rayson commented 5 years ago

Split waiverdb-repotracker-trigger into 2 jobs: waiverdb-trigger-on-latest-tag and waiverdb-trigger-on-stage-tag. (Following the practice of #284)
Allow to customize messaging provider (for pipeline debugging).
Fix an error about TRACKED_CONTAINER_REPO.

@mikeb PTAL

rebased onto 346079d9cbbb14199bd828f7855b1309a6089346

5 years ago

mikeb commented on line 5 of openshift/pipelines/jobs/waiverdb-trigger-on-latest-tag.env 5 years ago

Should be TRACKED_TAG_REGEX

mikeb commented on line 5 of openshift/pipelines/templates/waiverdb-repotracker-trigger.yaml 5 years ago

Should be TRACKED_TAG_REGEX

mikeb commented on line 13 of openshift/pipelines/templates/waiverdb-repotracker-trigger.Jenkinsfile 5 years ago

The action and tag checks could all be moved into the selector, which is more efficient than message checks. Something like:

selector: "repo='${params.TRACKED_CONTAINER_REPO}' AND action IN ('added', 'updated') AND tag LIKE '${params.TRACKED_TAG_REGEX}'"

mikeb commented on line 6 of openshift/pipelines/templates/waiverdb-repotracker-trigger.yaml 5 years ago

Does this really need to be a regex? It might be simpler to just make this a string that's matched against.

rayson commented 5 years ago

The action and tag checks could all be moved into the selector, which is more efficient than message checks. Something like:
selector: "repo='${params.TRACKED_CONTAINER_REPO}' AND action IN ('added', 'updated') AND tag LIKE '${params.TRACKED_TAG_REGEX}'"

Sounds good. Regex is used because in the existing job checks for updates of 2 tags. Since we are going to split into 2 jobs, each job should focus only on one tag.

Does this really need to be a regex? It might be simpler to just make this a string that's matched against.
Regex is not needed because each new job focuses only on one tag.

rebased onto 3f0d0d8

5 years ago

mikeb commented 5 years ago

+1 This looks great!

mikeb commented 5 years ago

Commit 5f84e1e fixes this pull-request