This is also a thought to share most of pipeline logic among nearly all factory 2.0 services.
Many services in Factory 2.0 have the followings in common:

• hosted on Pagure
• written in Python
• having OpenShift templates for dynamic test environment creations
• using similar tools to test and build

I can imagine if we build the CI/CD for other services using this way, most of the logic in the pipelines can be reused.

@rayson Before we write our own Factory 2.0 shared library, we should take a look at what we can leverage from the H-DSL/contra work being done by the Continuous Productization team. Their shared libraries will (soon) support provisioning OpenShift resources, running tests, and reporting back to resultsdb. They've also expressed interest in writing libraries to interface with Greenwave for gating. Let's see how we can collaborate with that team to reuse and extend the code they already have.

@mikeb That could be a direction.

At the level of Jenkinfile code, current WaiverDB pipelines use sub-jobs to reuse some common procedures. For example, in many workflows we should run an integration test, so we have a Jenkins job named waiverdb-integration-test and start that job as a build step in other jobs. This is still not flexible enough especially you want to reuse it cross project. The intention of creating a shared lib is to abstract those common steps. Hence, I think a shared library, at least shared among Factory 2.0 services, is still needed.

I heard Contra H-DSL before (I never used it but I will try). From my understanding, it is a high level DSL which focus the test infrastructure creation, deployment and some high-level workflows. It doesn't cover some concrete routines like MBS/ResultsDB/Greenwave/Pagure integration, testing environment setup, image push, etc.

Anyway, I will try H-DSL in the next few weeks to see what and how we can benefit from it. Do you know where I can get started and who can help me with this?

Here's some information about H-DSL: https://docs.engineering.redhat.com/display/CONTRA/Higher+Level+DSL

Unfortunately it does not support deploying infrastructure in OpenShift yet, but that is being actively worked on and should be supported in the next few weeks.

When it does, I think we can work on extending or wrapping it to handle the concrete routines you mention, like deploying and configuring MBS/WaiverDB, etc. If we could drive a whole end-to-end workflow using their yaml syntax, I think that would be a big win for reusability and maintainability. This is a direction I'm very interested in investigating.

You can ask questions about it on the internal cp-devel mailing list, and find people working on it on #contra on IRC.

Based on email conversations, I think it makes sense to start development of our own Jenkins shared library, independent of H-DSL/contra.

https://pagure.io/c3i-library is created for this purpose.