koji

Clone

#2754 doc: jenkins fedora -> centos migration
Merged a year ago by tkopecek. Opened a year ago by tkopecek.
tkopecek/koji issue2745  into  master

file modified
+54 -89 
@@ -1,15 +1,14 @@ 
 

  Unit tests in Fedora's Jenkins
 

  ==============================
 

  

- We're using Fedora's `Jenkins <https://jenkins.fedorainfracloud.org/job/koji>`_
 

- infrastructure for automatically running unit tests for new commits in
 

- master branch.
 

+ We're using CentOS's `Jenkins
 

+ <https://jenkins-fedora-infra.apps.ocp.ci.centos.org/job/koji>`_ infrastructure
 

+ for automatically running unit tests for new commits in master branch.
 

  

- Current setup uses periodical checking of koji's git and when new commits are
 

- seen, tests are run. Normal developer needn't to care about his workflow, but can
 

- use it for his advantage as it can run tests for him with platforms he has no
 

- access to. Currently are tests run on Fedora 24, Fedora 25 and CentOS 6
 

- platforms.
 

+ Pagure triggers tests for any new pull request. It can be also triggered
 

+ manually by pressing "Rerun CI" button in PR.
 

+ 

+ Currently we run tests on Fedora 33/34/rawhide and CentOS 7/8 platforms.
 

  

  Usage
 

  -----
 
@@ -18,31 +17,18 @@ 
 

  of solving issue, this documentation must be updated to reflect current state
 

  in jenkins.
 

  

- If you want to run tests against specific repo (your or official branch), log
 

- in to jenkins (via FAS unified login), navigate to `Build with parameters
 

- <https://jenkins.fedorainfracloud.org/job/koji/build?delay=0sec>`_ and put
 

- your repository url to ``REPO`` and name of branch to ``BRANCH``.
 

- ``BRANCH_TO`` could be left blank as it is default set to *master*. Pressing
 

- ``BUILD`` should give you link to running build in all supported
 

- environments.
 

- 

- 

  Configuration
 

  -------------
 

  

  - Public access is for everyone with no need to log in to jenkins.
 

- - Admin access is via same interface and currently tkopecek and mikem have
 

+ - Admin access is via same interface and currently tkopecek have
 

     access there. If you need access for yourself (you're probably part of
 

-    brew/koji team) create jira in BREW project requesting this.
 

-    Prerequisite for this is Fedora account (probably same one you are using
 

-    for work in pagure).
 

+    brew/koji team) create ticket in https://pagure.io/centos-infra requesting this.
 

+    Prerequisite for this is CentOS account.
 

  

  - Setup - Following items are set-up via
 

-    https://jenkins.fedorainfracloud.org/job/koji/configure
 

+    https://jenkins-fedora-infra.apps.ocp.ci.centos.org/job/koji/configure
 

  

- - Please don't change access rules (*Enable project-based security*
 

-    fields) unless you've a corresponding jira for that, so every change of
 

-    access is tracked there.
 

  - *Job notifications* - when jenkins job finishes, it will send some info to
 

     pagure. Such call will add a comment to PR. For this REST hook needs to
 

     be configured:
 
@@ -70,8 +56,10 @@ 
 

  

  - *Disable build* - it could be used if a lot of failing build happens with
 

      no vision of early recovery - temporarily suspend jenkins jobs
 

- - *Execute concurrent builds if necessary* - We currently have no need of this
 

- - *Restrict where this project can be run* - Fedora 24 is used for now which means 'F24' value
 

+ - *Restrict where this project can be run* - F33 is the only used agent now.
 

+     Basically we need an agent which supports podman as all the tests are run in
 

+     docker images managed by podman.
 

+ 

  - *Source Code Management*
 

  

    * Git
 
@@ -91,70 +79,47 @@ 
 

  

      * Schedule: H/5 * * * *
 

  

- - *Build* - most important part - script which runs tests itself. Here you can also add missing requirements which will get installed via pip (not via rpms from F24!)
 

+ - *Build* - most important part - script which runs tests itself. Basically we
 

+   run tests in containers, last one also run flake8 and is used for coverage
 

+   report.
 

  

  

  .. code-block:: shell
 

- 

-   # setup virtual environment
 

-   rm -rf kojienv
 

-   if [ -x /usr/bin/python3 ] ; then
 

-       python3 -m venv --system-site-packages kojienv
 

-   else
 

-       virtualenv --system-site-packages kojienv
 

-   fi
 

-   source kojienv/bin/activate
 

- 

-   # install python requirements via pip, you can also specify exact versions here
 

-   if [ $NODE_NAME == "EL6" ] ; then
 

-       pip install "psycopg2<2.7" "urllib3<1.24" "requests<2.20" "requests-mock<1.5" \
 

-                   "Markdown<3.1" "mock<3.0.0" nose python-qpid-proton coverage \
 

-                   python-multilib Cheetah --upgrade --ignore-installed
 

-   else
 

-       pip install pip packaging --upgrade --ignore-installed
 

-       pip install setuptools --upgrade --ignore-installed
 

-       pip install psycopg2 requests-mock nose python-qpid-proton mock coverage \
 

-                   python-multilib flake8 --upgrade --ignore-installed
 

-       if [ -x /usr/bin/python3 ] ; then
 

-           pip install Cheetah3 nose-cover3 --upgrade --ignore-installed
 

-       else
 

-           pip install Cheetah --upgrade --ignore-installed
 

-       fi
 

-   fi
 

- 

- 

-   # rehash package to be sure updated versions are used
 

-   hash -r
 

- 

-   # merge PR into main repository
 

-   if [ -n "$REPO" -a -n "$BRANCH" ]; then
 

-       git config --global user.email "test@example.com"
 

-       git config --global user.name "Tester"
 

-       git remote rm proposed || true
 

-       git remote add proposed "$REPO"
 

-       git fetch proposed
 

-       git checkout "origin/${BRANCH_TO:-master}"
 

-       git merge --no-ff "proposed/$BRANCH" -m "Merge PR"
 

-   fi
 

- 

-   # remove possible coverage output and run tests
 

-   coverage erase
 

-   PYTHONPATH=hub/.:plugins/hub/.:plugins/builder/.:cli/plugins/cli/.:cli/.:www/lib/. nosetests --with-coverage --cover-package .
 

-   coverage xml --omit 'kojienv/*'
 

- 

-   # run additional tests if configured
 

-   #pylint . > pylint_report.txt
 

-   #pep8 . > pep8_report.txt
 

- 

-   if [ $NODE_NAME == "EL6" ] ; then
 

-       # fake report, flake8 is not available for old python
 

-       echo > flake8_report.txt
 

-   else
 

-       flake8 cli hub builder plugins koji util www vm devtools tests --output-file flake8_report.txt
 

-   fi
 

- 

-   # kill virtual environment
 

-   deactivate
 

+     # merge PR into main repository  

+     if [ -n "$REPO" -a -n "$BRANCH" ]; then  

+         git config --global user.email "test@example.com"  

+         git config --global user.name "Tester"
 

+         git remote rm proposed || true  

+         git remote add proposed "$REPO"  

+         git fetch proposed   

+         git checkout "origin/${BRANCH_TO:-master}"  

+         git merge --no-ff "proposed/$BRANCH" -m "Merge PR"  

+     fi  

+ 

+     # centos:7
 

+     rm -rf .tox
 

+     podman run --rm --pull=always -v $PWD:/koji --security-opt label=disable --name koji-centos-test-7 quay.io/tkopecek/koji-centos-test:7 bash -c "cd /koji && tox -e py2"
 

+     podman rmi quay.io/tkopecek/koji-centos-test:7
 

+ 

+     # centos:8
 

+     rm -rf .tox
 

+     podman run --rm --pull=always -v $PWD:/koji --security-opt label=disable --name koji-centos-test-8 quay.io/tkopecek/koji-centos-test:8 bash -c "cd /koji && tox -e py3"
 

+     podman rmi quay.io/tkopecek/koji-centos-test:8
 

+ 

+     # fedora:32
 

+     rm -rf .tox
 

+     podman run --rm --pull=always -v $PWD:/koji --security-opt label=disable --name koji-fedora-test-32 quay.io/tkopecek/koji-fedora-test:32 bash -c "cd /koji && tox -e py3"
 

+     podman rmi quay.io/tkopecek/koji-fedora-test:32
 

+ 

+     # fedora:33
 

+     rm -rf .tox
 

+     podman run --rm --pull=always -v $PWD:/koji --security-opt label=disable --name koji-fedora-test-33 quay.io/tkopecek/koji-fedora-test:33 bash -c "cd /koji && tox -e py3"
 

+     podman rmi quay.io/tkopecek/koji-fedora-test:33
 

+ 

+     # fedora:rawhide
 

+     rm -rf .tox
 

+     podman run --rm --pull=always -v $PWD:/koji --security-opt label=disable --name koji-fedora-test-rawhide quay.io/tkopecek/koji-fedora-test:rawhide bash -c "cd /koji && tox -e flake8,py3"
 

+     podman rmi quay.io/tkopecek/koji-fedora-test:rawhide
 

  

  - *Post-build actions*
 

  
@@ -162,7 +127,7 @@ 
 

    * *Report Violations* - *pep8*: flake8_report.txt
 

    * *E-mail notification*:
 

  

-     * Recipients: tkopecek@redhat.com brew-devel@redhat.com
 

+     * Recipients: tkopecek@redhat.com exd-sp-rhel-build-alerts@redhat.com
 

      * Send separate e-mails to individuals who broke the build
 

  

  - *Send messages to fedmsg*

:thumbsup:

BTW, I just registered my centos account, and I'm able to manipulate it

Metadata Update from @tkopecek:
- Pull-request tagged with: doc, no_qe
a year ago

Cool, I would like to add automatic triggering of image rebuilds on commits here, but it looks needs more data than pagure's webhook is able to provide. So for now I'll just rebuild them manually.

Commit 8c9a347 fixes this pull-request

Pull-Request has been merged by tkopecek
a year ago
Metadata
None

no_qe doc

Changes Summary 1
+54 -89
file changed
docs/source/configuring_jenkins.rst
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.