PR#366: Build container from official fedora - waiverdb

waiverdb

#366 Build container from official fedora

Merged 4 years ago by mikeb. Opened 4 years ago by mkovarik.

mkovarik/waiverdb jenkins-slave-direct into master

Build container from official fedora

Michal Kovarik • 4 years ago

5095acc

openshift/containers/jenkins-slave/Dockerfile

file modified

+66 -24

		`@@ -1,35 +1,77 @@`
		`- #FIXME: The Pipeline is currently using this Dockerfile to produce images. The one located at the project root is not changed in case of breaking anything.`
		`- FROM docker-registry.upshift.redhat.com/devops-automation/rad-slave-fedora:latest`
		`- LABEL name="waiverdb-jenkins-slave" \`
		`- description="Jenkins slave for WaiverDB dev tests" \`
		`- vendor="WaiverDB Developers" \`
		`- license="GPLv2+"`
		`+ # Based on the rad-jenkins image, which is in turn based on:`
		`+ # https://github.com/jenkinsci/docker-jnlp-slave/blob/master/Dockerfile`
		`+ # https://github.com/jenkinsci/docker-slave/blob/master/Dockerfile`

		`- USER root`
		`+ FROM fedora:29`
		`+ LABEL \`
		`+ org.opencontainers.image.title="Jenkins agent image for WaiverDB" \`
		`+ org.opencontainers.image.description="Jenkins slave for WaiverDB dev tests" \`
		`+ org.opencontainers.image.vendor="WaiverDB Developers" \`
		`+ org.opencontainers.image.licenses="GPLv2+"`

		`+ ARG USER=jenkins`
		`+ ARG UID=10000`
		`+ ARG HOME_DIR=/home/jenkins`
		`+ ARG REMOTING_VERSION=3.36`
		`ARG TINI_VERSION=0.18.0`
		`+ ARG DNF_CMD="dnf -y --setopt=deltarpm=0 --setopt=install_weak_deps=false --setopt=tsflags=nodocs"`
		`+ ARG CA_URLS=""`
		`+`
		`+ USER root`

		`- RUN dnf -y install 'dnf-command(builddep)' dnf-utils git mock-core-configs tar gzip skopeo \`
		`- wget postgresql make rpmdevtools rpmlint \`
		`+ RUN ${DNF_CMD} install -y \`
		`+ java-1.8.0-openjdk-headless gettext nss_wrapper git-core \`
		`+ tar gzip skopeo wget make bind-utils \`
		`+ origin-clients \`
		`+ # Jenkins pipeline 'sh' steps seem to require ps`
		`+ procps-ng \`
		`+ # Tools to build and test waiverdb`
		`+ 'dnf-command(builddep)' dnf-utils git mock-core-configs \`
		`+ postgresql rpmdevtools rpmlint \`
		`python3-flake8 python3-pylint python3-pytest \`
		`python3-sphinx python3-sphinxcontrib-httpdomain \`
		`- python3-ldap \`
		`- origin-clients \`
		`- # install tini, a tiny but valid init for containers`
		`- && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini" \`
		`- && chmod +rx /usr/local/bin/tini \`
		`- # install wait-for-it.sh, to allow containers to wait for other services to come up`
		`- && wget -O /usr/local/bin/wait-for-it "https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh" \`
		`- && chmod +rx /usr/local/bin/tini /usr/local/bin/wait-for-it \`
		`- # clean up`
		`- && dnf clean all`
		`+ python3-ldap`
		`+`
		`+ # CA Certs`
		`+ WORKDIR /etc/pki/ca-trust/source/anchors/`
		`+ RUN for ca_url in ${CA_URLS}; do curl -skO ${ca_url}; done && \`
		`+ update-ca-trust`

		`# install build dependencies for WaiverDB`
		`COPY waiverdb.spec /usr/local/src/waiverdb/waiverdb.spec`
		`RUN cd /usr/local/src/waiverdb \`
		`- && dnf -y builddep waiverdb.spec \`
		`- && dnf clean all \`
		`+ && ${DNF_CMD} builddep waiverdb.spec \`
		`+ && ${DNF_CMD} clean all \`
		`&& cd / && rm -rf /usr/local/src/waiverdb`
		`- WORKDIR /var/lib/jenkins/`
		`- ENTRYPOINT ["/usr/local/bin/tini", "--", "jenkins-slave"]`
		`- USER 1000`
		`+`
		`+ # Setup the user for non-arbitrary UIDs with OpenShift`
		`+ # https://docs.openshift.org/latest/creating_images/guidelines.html#openshift-origin-specific-guidelines`
		`+ RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} && \`
		`+ chmod -R g+rwx ${HOME_DIR}`
		`+`
		`+ # Make /etc/passwd writable for root group`
		`+ # so we can add dynamic user to the system in entrypoint script`
		`+ RUN chmod g+rw /etc/passwd`
		`+`
		`+ # Retrieve jenkins agent client`
		`+ RUN curl --create-dirs -sSLo /usr/share/jenkins/agent.jar \`
		`+ https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/${REMOTING_VERSION}/remoting-${REMOTING_VERSION}.jar && \`
		`+ chmod 755 /usr/share/jenkins && \`
		`+ chmod 644 /usr/share/jenkins/agent.jar`
		`+`
		`+ # Entry point script to run jenkins agent client`
		`+ COPY openshift/containers/jenkins-slave/jenkins-agent /usr/local/bin/jenkins-agent`
		`+ RUN chmod 755 /usr/local/bin/jenkins-agent`
		`+`
		`+ # install tini, a tiny but valid init for containers`
		`+ # install wait-for-it.sh, to allow containers to wait for other services to come up`
		`+ RUN curl -L -o /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini" \`
		`+ && chmod +rx /usr/local/bin/tini \`
		`+ && curl -L -o /usr/local/bin/wait-for-it "https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh" \`
		`+ && chmod +rx /usr/local/bin/tini /usr/local/bin/wait-for-it \`
		`+ && ${DNF_CMD} clean all`
		`+`
		`+ # For OpenShift we MUST use the UID of the user and not the name.`
		`+ USER ${UID}`
		`+ WORKDIR ${HOME_DIR}`
		`+ ENTRYPOINT ["/usr/local/bin/tini", "--", "jenkins-agent"]`

openshift/containers/jenkins-slave/jenkins-agent

file added

+111

		`@@ -0,0 +1,111 @@`
		`+ #!/usr/bin/env bash`
		`+`
		`+ # The MIT License`
		`+ #`
		`+ # Copyright (c) 2015, CloudBees, Inc.`
		`+ #`
		`+ # Permission is hereby granted, free of charge, to any person obtaining a copy`
		`+ # of this software and associated documentation files (the "Software"), to deal`
		`+ # in the Software without restriction, including without limitation the rights`
		`+ # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell`
		`+ # copies of the Software, and to permit persons to whom the Software is`
		`+ # furnished to do so, subject to the following conditions:`
		`+ #`
		`+ # The above copyright notice and this permission notice shall be included in`
		`+ # all copies or substantial portions of the Software.`
		`+ #`
		`+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR`
		`+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,`
		`+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE`
		`+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER`
		`+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,`
		`+ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN`
		`+ # THE SOFTWARE.`
		`+`
		`+ # Usage jenkins-agent.sh [options] -url http://jenkins [SECRET] [AGENT_NAME]`
		`+ # Optional environment variables :`
		`+ # * JENKINS_TUNNEL : HOST:PORT for a tunnel to route TCP traffic to jenkins host, when jenkins can't be directly accessed over network`
		`+ # * JENKINS_URL : alternate jenkins URL`
		`+ # * JENKINS_SECRET : agent secret, if not set as an argument`
		`+ # * JENKINS_AGENT_NAME : agent name, if not set as an argument`
		`+ # * JENKINS_JAR_CACHE : directory for cached jar files`
		`+ #`
		`+ # Credentials are also supported for authentication to jenkins. If desired,`
		`+ # create the directory /etc/jenkins/credentials with "username" and "password"`
		`+ # files within.`
		`+ #`
		`+ # This script was originally adopted from:`
		`+ # https://github.com/jenkinsci/docker-jnlp-slave/blob/master/jenkins-slave`
		`+`
		`+ # Dynamically create a passwd file for non-arbitrary UIDs.`
		`+ # Taken from: https://docs.openshift.org/latest/creating_images/guidelines.html#openshift-origin-specific-guidelines`
		`+ export USER_ID=$(id -u)`
		`+ export GROUP_ID=$(id -g)`
		`+`
		`+ # Skip for root user`
		`+ if [ x"$USER_ID" != x"0" ]; then`
		`+ cp /etc/passwd $NSS_WRAPPER_PASSWD`
		`+ echo "jenkins:x:${USER_ID}:${GROUP_ID}:jenkins:${HOME}:/bin/bash" >> $NSS_WRAPPER_PASSWD`
		`+ fi`
		`+`
		`+ if [ $# -eq 1 ]; then`
		`+`
		+ # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
		`+ exec "$@"`
		`+`
		`+ else`
		`+`
		`+ # if -tunnel is not provided try env vars`
		`+ if [[ "$@" != "-tunnel " ]]; then`
		`+ if [ ! -z "$JENKINS_TUNNEL" ]; then`
		`+ TUNNEL="-tunnel $JENKINS_TUNNEL"`
		`+ fi`
		`+ fi`
		`+`
		`+ if [ -n "$JENKINS_URL" ]; then`
		`+ URL="-url $JENKINS_URL"`
		`+ fi`
		`+`
		`+ if [ -n "$JENKINS_NAME" ]; then`
		`+ JENKINS_AGENT_NAME="$JENKINS_NAME"`
		`+ fi`
		`+`
		`+ if [ -n "$JENKINS_JAR_CACHE" ]; then`
		`+ JAR_CACHE="-jar-cache $JENKINS_JAR_CACHE"`
		`+ fi`
		`+`
		`+ if [ -d "/etc/jenkins/credentials" ]; then`
		`+ USERNAME="$(cat /etc/jenkins/credentials/username)"`
		`+ PASSWORD="$(cat /etc/jenkins/credentials/password)"`
		`+ CREDENTIALS="-credentials ${USERNAME}:${PASSWORD}"`
		`+ fi`
		`+`
		`+ if [ -z "$JNLP_PROTOCOL_OPTS" ]; then`
		`+ echo "Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior"`
		`+ JNLP_PROTOCOL_OPTS="-Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true"`
		`+ fi`
		`+`
		`+ # If both required options are defined, do not pass the parameters`
		`+ OPT_JENKINS_SECRET=""`
		`+ if [ -n "$JENKINS_SECRET" ]; then`
		`+ if [[ "$@" != "${JENKINS_SECRET}" ]]; then`
		`+ OPT_JENKINS_SECRET="${JENKINS_SECRET}"`
		`+ else`
		`+ echo "Warning: SECRET is defined twice in command-line arguments and the environment variable"`
		`+ fi`
		`+ fi`
		`+`
		`+ OPT_JENKINS_AGENT_NAME=""`
		`+ if [ -n "$JENKINS_AGENT_NAME" ]; then`
		`+ if [[ "$@" != "${JENKINS_AGENT_NAME}" ]]; then`
		`+ OPT_JENKINS_AGENT_NAME="${JENKINS_AGENT_NAME}"`
		`+ else`
		`+ echo "Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable"`
		`+ fi`
		`+ fi`
		`+`
		`+ #TODO: Handle the case when the command-line and Environment variable contain different values.`
		`+ #It is fine it blows up for now since it should lead to an error anyway.`
		`+`
		`+ exec java $JAVA_OPTS $JNLP_PROTOCOL_OPTS -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless $CREDENTIALS $JAR_CACHE $TUNNEL $URL $OPT_JENKINS_SECRET $OPT_JENKINS_AGENT_NAME "$@"`
		`+ fi`

openshift/pipelines/templates/waiverdb-build-template.yaml

file modified

		`@@ -128,6 +128,9 @@`
		`completionDeadlineSeconds: 1800`
		`strategy:`
		`dockerStrategy:`
		`+ buildArgs:`
		`+ - name: CA_URLS`
		`+ value: https://password.corp.redhat.com/RH-IT-Root-CA.crt`
		`forcePull: true`
		`dockerfilePath: openshift/containers/jenkins-slave/Dockerfile`
		`resources:`

mkovarik commented 4 years ago

Dependency on external container removes flexibility.
Old container has been updated to fedora31 which caused issue:
https://pagure.io/waiverdb/issue/365

mkovarik commented 4 years ago

@mikeb PTAL - going to remove dependency on NSS_WRAPPER in next PR. Current slave was broken since rad-jenkins image was updated to fedora 31 and unit tests are supporting only fedora 29.