| |
@@ -0,0 +1,213 @@
|
| |
+ # C3I ansible role
|
| |
+
|
| |
+ C3I role gives you control over building, testing, deploying, and promoting custom service on OpenShift.
|
| |
+
|
| |
+ It uses [OpenShift Pipeline][], a technology powered by OpenShift, using a combination of the [Jenkins Pipeline Build Strategy][], [Jenkinsfiles][], and the OpenShift Domain Specific Language (DSL) powered by the [OpenShift Jenkins Pipeline (DSL) Plugin][] (previously called the OpenShift Jenkins Client Plugin).
|
| |
+
|
| |
+ When starting a new pipeline build, the Jenkins slave and the whole testing environment will be dynamically created as OpenShift pods. And after the build is complete, all of them will be destroyed.
|
| |
+
|
| |
+ C3I role is providing C3I workflows deployed to Openshift Jenkins master:
|
| |
+
|
| |
+ * premerge
|
| |
+ * postmerge
|
| |
+ * image promotions
|
| |
+
|
| |
+ Role is configurable with role variables starting `c3i_`. Component developers are expected to provide Jenkinsfile snippets with stages for building their component container and tests for their promotion workflows.
|
| |
+
|
| |
+ ## Requirements
|
| |
+
|
| |
+ ### Jenkins master
|
| |
+ [Installation and requirements for Jenkins master](jenkins-master-install.md)
|
| |
+
|
| |
+ ### Integration with pagure.io
|
| |
+ Premerge and Postmerge workflows can be triggered by FedMsg hook from Pagure. To use this integration set your Pagure project:
|
| |
+
|
| |
+ * Pagure projects have to set 'Fedmsg notifications' in 'Project Options'
|
| |
+ * Pagure projects have to activate 'Hooks' -> 'Fedmsg'
|
| |
+ * OpenShift namespace with Jenkins master contains secret named `c3i_pagure_api_key_secret` with Pagure API Token
|
| |
+
|
| |
+ #### Pagure API token
|
| |
+ Generate token in Pagure project setting -> API Keys, add these permissions to token:
|
| |
+
|
| |
+ * Flag a commit
|
| |
+ * Comment on a pull-request
|
| |
+ * Flag a pull-request
|
| |
+
|
| |
+ Secret creation:
|
| |
+ ```
|
| |
+ oc delete secret pagure-api-key --ignore-not-found=true
|
| |
+ oc create secret generic pagure-api-key --from-literal=secrettext=${KEY}
|
| |
+ oc label secret pagure-api-key credential.sync.jenkins.openshift.io=true
|
| |
+ ```
|
| |
+
|
| |
+ #### Listen for Fedmsg
|
| |
+ Jenkins master is listening on Fedmsg.
|
| |
+
|
| |
+ * Click on 'Manage Jenkins'
|
| |
+ * Go to 'Configure System'
|
| |
+ * In Section 'JMS Messaging Providers' add FedMsg
|
| |
+ * Name: fedmsg
|
| |
+ * Hub Address: tcp://hub.fedoraproject.org:9940
|
| |
+ * Publish Address: tcp://hub.fedoraproject.org:9940
|
| |
+
|
| |
+ ### Integration with Ansible tower
|
| |
+ Postmerge and Promotion workflows can trigger Tower job when new container image is created, tested and uploaded. To enable this feature your Openshift namespace has to contain secret `tower-promotion-secret` with username and password of Tower user who has permission to run requested Tower job templates.
|
| |
+ Secret creation:
|
| |
+ ```
|
| |
+ oc create secret generic tower-promotion-secret --from-literal=USERNAME=<toweruser> --from-literal=PASSWORD=<toweruserpass>
|
| |
+ ```
|
| |
+
|
| |
+ ## Workflows
|
| |
+
|
| |
+ ### Pre-merge
|
| |
+ Jenkins Job `premerge` is listening on PR changes - new, update, rebase, reopen. New changes are checkouted into current directory. New C3IaaS project is requested and provided via `PIPELINE_ID` variable. Pagure PR is marked as `Pending` in pagure. Snippet `c3i_build_and_test_snippet` is executed. Test results should be sent as part of `c3i_build_and_test_snippet`. Pagure PR state is updated. Temporary images are deleted.
|
| |
+
|
| |
+ ### Post-merge
|
| |
+ Jenkins Job named `job-updater` is listening on commits in master branch. For every new commit in master branch the `job-updater` updates all workflow jobs and triggers `post-merge` job.
|
| |
+
|
| |
+ `post-merge` Jenkins Job clones new changes into current directory. Snippet `c3i_build_and_test_snippet` is executed. Test results should be sent as part of `c3i_build_and_test_snippet`. Temporary images are promoted to final destination with `latest` tag.
|
| |
+
|
| |
+ ### Promotion
|
| |
+ Jenkins job `trigger-on-[latest|stage]-tag` is triggered on on image change for specified tag (latest or stage) by [Repotracker][]. The job deploys environment and run tests defined in `c3i_integration_test_snippet`. Test results should be sent as part of `c3i_integration_test_snippet`.
|
| |
+
|
| |
+ Jenkins job `greenwave-promote-to-[stage|prod]` listens on container tests results. Job is promoting image from latest to stage or from stage to prod.
|
| |
+
|
| |
+ ## Role Variables
|
| |
+
|
| |
+ For all variables check [roles/c3i/defaults/main.yml](../roles/c3i/defaults/main.yml)
|
| |
+
|
| |
+ ### Global variables
|
| |
+
|
| |
+ * c3i\_component - default identificator used in job naming and in default values.
|
| |
+ * c3i\_mail\_address - email address for report, by default it's empty
|
| |
+
|
| |
+ ### Git Repository setting
|
| |
+
|
| |
+ * c3i\_git\_repo - Git repository to be tracked and built
|
| |
+ * default: https://pagure.io/{{ c3i\_component }}.git
|
| |
+ * c3i\_git\_main\_branch - Main branch used for triggering post merge workflow
|
| |
+ * default: master
|
| |
+
|
| |
+ ### Docker Repository setting
|
| |
+
|
| |
+ * c3i\_container\_registry\_credentials - OpenShift secret name with dockerconfigjson data
|
| |
+ * default: factory2-pipeline-registry-credentials
|
| |
+
|
| |
+ ### Jenkins agent variables
|
| |
+ When `c3i_jenkins_agent_buildconfig_dockerfile` is available the build config for Jenkins agent is created.
|
| |
+
|
| |
+ * c3i\_jenkins\_agent\_buildconfig\_dockerfile - Docker file path for build Jenkins agent.
|
| |
+ * c3i\_jenkins\_agent\_buildconfig\_contextdir - Context dir path for Docker build.
|
| |
+
|
| |
+ Jenkins agents can be customized:
|
| |
+
|
| |
+ * c3i\_default\_agent\_snippet - Agent definition snippet, default [default-agent.groovy](../roles/c3i/templates/default-agent.groovy)
|
| |
+ * c3i\_jenkins\_agent\_image - image for Jenking agent
|
| |
+ * default: "{{ c3i\_quay\_address }}/{{ c3i\_quay\_namespace }}/{{ c3i\_component }}-jenkins-slave:latest"
|
| |
+
|
| |
+ ### Ansible tower
|
| |
+
|
| |
+ * c3i\_tower\_instance - url of ansible tower
|
| |
+ * c3i\_tower\_secret: Secret in Openshift namespace containing credentials - USERNAME and PASSWORD
|
| |
+ * default: tower-promotion-secret
|
| |
+ * c3i\_tower\_template\_id\_dev - Ansible tower job template id for dev promotion
|
| |
+ * c3i\_tower\_template\_id\_stage - Ansible tower job template id for stage promotion
|
| |
+ * c3i\_tower\_template\_id\_prod - Ansible tower job template id for prod promotion
|
| |
+
|
| |
+
|
| |
+ ## Stage snippets
|
| |
+ Snippets are used for defining custom parts of Jenkins files. It's expected `stage` part in declarative syntax. This snippet is included in final Jenkins file. Purpose of snippet is to provide way to define custom code for a component, eg. way how to build and test the component.
|
| |
+
|
| |
+ ### c3i\_build\_and\_test\_snippet
|
| |
+ Path to snippet for Build container and test acceptance for 'latest' tag.
|
| |
+
|
| |
+ Git repository is checkout into current working directory. New openshift project is created and available in `PIPELINE_ID` variable.
|
| |
+
|
| |
+ Input parameters are defined in [roles/c3i/templates/build.yml](../roles/c3i/templates/build.yml). Input parameters can be extended by `c3i_build_custom_parameters`, eg.:
|
| |
+ ```yaml
|
| |
+ c3i_build_custom_parameters:
|
| |
+ - name: MBS_BACKEND_IMAGESTREAM_NAME
|
| |
+ value: mbs-backend
|
| |
+ - name: MBS_FRONTEND_IMAGESTREAM_NAME
|
| |
+ value: mbs-frontend
|
| |
+ - name: MBS_SPEC_FILE
|
| |
+ value: https://src.fedoraproject.org/rpms/module-build-service/raw/master/f/module-build-service.spec
|
| |
+ ```
|
| |
+
|
| |
+ Expected output variables for Post merge workflow:
|
| |
+
|
| |
+ * RESULTING\_IMAGE\_REPOS - provides list of repos containing generated images separated by ','.
|
| |
+ * RESULTING\_TAG - provides tag used on generated images.
|
| |
+
|
| |
+ Snippet could contain multiple stages. Snippet should:
|
| |
+
|
| |
+ * Prepare environment for build
|
| |
+ * Build image
|
| |
+ * Run unit tests
|
| |
+ * Deploy test environment
|
| |
+ * Run integration tests
|
| |
+ * Send result
|
| |
+
|
| |
+ Examples:
|
| |
+
|
| |
+ * [Waiverdb build and test snippet](https://pagure.io/waiverdb/blob/master/f/openshift/pipelines/snippets/waiverdb-build-and-test.groovy)
|
| |
+ * [MBS build and test snippet](https://pagure.io/fm-orchestrator/blob/master/f/openshift/integration/koji/pipelines/snippets/mbs-build-and-test.groovy)
|
| |
+
|
| |
+ ### c3i\_integration\_test\_snippet
|
| |
+ Path to snippet for building testing environment and running tests for promotion workflow.
|
| |
+
|
| |
+ Input parameters are defined in [roles/c3i/templates/trigger-on-tag.yml](../roles/c3i/templates/trigger-on-tag.yml). Input parameters can be extended by `c3i_integration_test_custom_parameters`.
|
| |
+
|
| |
+ Snippet should:
|
| |
+
|
| |
+ * Deploy testing environment
|
| |
+ * Run tests
|
| |
+ * Report results
|
| |
+
|
| |
+ Examples:
|
| |
+
|
| |
+ * [Waiverdb integration test snippet](https://pagure.io/waiverdb/blob/master/f/openshift/pipelines/snippets/waiverdb-full-integration-test.groovy)
|
| |
+ * [MBS integration test snippet](https://pagure.io/fm-orchestrator/blob/master/f/openshift/integration/koji/pipelines/snippets/mbs-integration-test.groovy)
|
| |
+
|
| |
+
|
| |
+ ## Example Playbook
|
| |
+
|
| |
+ ```yaml
|
| |
+ - name: Deplomyent playbook
|
| |
+ hosts: localhost
|
| |
+ vars_files:
|
| |
+ - c3i-role-vars.yml
|
| |
+ tasks:
|
| |
+ - git:
|
| |
+ repo: "{{ c3i_lib_url }}"
|
| |
+ dest: c3i-library
|
| |
+ version: "{{ c3i_lib_branch }}"
|
| |
+ tags:
|
| |
+ - validate
|
| |
+ - file:
|
| |
+ src: c3i-library/roles
|
| |
+ dest: roles
|
| |
+ state: link
|
| |
+ tags:
|
| |
+ - validate
|
| |
+ - include_role:
|
| |
+ name: c3i
|
| |
+ tags:
|
| |
+ - validate
|
| |
+ ```
|
| |
+
|
| |
+ ## Example projects
|
| |
+
|
| |
+ * [Waiverdb](https://pagure.io/waiverdb/blob/master/f/openshift/pipelines)
|
| |
+ * [Module Build System](https://pagure.io/fm-orchestrator/blob/master/f/openshift/integration/koji/pipelines)
|
| |
+
|
| |
+ ## Validation
|
| |
+ Generated Jenkins files can be validated by running playbook with `--tags validate`.
|
| |
+
|
| |
+ [OpenShift Pipeline]: https://docs.okd.io/3.11/dev_guide/openshift_pipeline.html
|
| |
+ [Jenkins Pipeline Build Strategy]: https://docs.openshift.com/container-platform/3.11/dev_guide/dev_tutorials/openshift_pipeline.html
|
| |
+ [Jenkinsfiles]: https://jenkins.io/doc/book/pipeline/jenkinsfile/
|
| |
+ [OpenShift Jenkins Pipeline (DSL) Plugin]: https://github.com/openshift/jenkins-client-plugin
|
| |
+ [OpenShift secret for registries]:https://docs.openshift.com/container-platform/3.11/dev_guide/builds/build_inputs.html#using-docker-credentials-for-private-registries
|
| |
+ [OpenShift secret for SSH key authentication]: https://docs.openshift.com/container-platform/3.11/dev_guide/builds/build_inputs.html#source-secrets-ssh-key-authentication
|
| |
+ [Repotracker]: https://github.com/release-engineering/repotracker
|
| |
mkovarik
commented 4 years ago
Update of ansible role documentation. Description of parameters and deployment.