#123 Update Dockerfile to conform C3I
Closed 5 years ago by jskladan. Opened 5 years ago by rayson.
taskotron/ rayson/resultsdb update-dockerfile  into  master

file modified
+2
@@ -8,6 +8,7 @@ 

      WSGIScriptReloading On

      Order deny,allow

      Allow from all

+     Require all granted

  </Directory>

  

  #Alias /resultsdb/static /var/www/resultsdb/resultsdb/static
@@ -15,4 +16,5 @@ 

  #<Directory /var/www/resultsdb/resultsdb/static>

  #Order allow,deny

  #Allow from all

+ #Require all granted

  #</Directory>

file modified
+1 -2
@@ -44,6 +44,7 @@ 

  

  # OIDC Configuration

  OIDC_ADMINS = []

+ import os

  OIDC_CLIENT_SECRETS = os.getcwd() + '/conf/oauth2_client_secrets.json'

  OIDC_AUD = 'My-Client-ID'

  OIDC_SCOPE = 'https://pagure.io/taskotron/resultsdb/access'
@@ -83,5 +84,3 @@ 

  

  # Publish Taskotron-compatible fedmsgs on the 'taskotron' topic

  MESSAGE_BUS_PUBLISH_TASKOTRON = False

- 

- 

file modified
+28 -19
@@ -1,10 +1,9 @@ 

  # This will produce an image to be used in Openshift

  # Build should be triggered from repo root like:

  # docker build -f openshift/Dockerfile \

- #              --tag <IMAGE_TAG> \

- #              --build-arg resultsdb_rpm=<RESULTSDB_RPM> ./

+ #              --tag <IMAGE_TAG>

  

- FROM fedora:28

+ FROM registry.fedoraproject.org/f29/httpd:latest

  LABEL \

      name="ResultsDB application" \

      vendor="ResultsDB developers" \
@@ -13,24 +12,33 @@ 

      usage="https://pagure.io/taskotron/resultsdb/blob/develop/f/openshift/README.md" \

      build-date=""

  

- # The caller should build a resultsdb RPM package used and then pass it in this arg.

- # Accept both a URL or a local path relative to the build context.

- ARG resultsdb_rpm

- ADD $resultsdb_rpm /tmp

+ USER root

+ COPY ./resultsdb.spec /opt/app-root/src/resultsdb/resultsdb.spec

  

- RUN dnf -y update \

-     && dnf -y install --setopt=tsflags=nodocs \

-         python-psycopg2 \

-         httpd \

-         mod_wsgi \

-         /tmp/$(basename $resultsdb_rpm) \

-     && dnf clean all \

-     && rm -f /tmp/$(basename $resultsdb_rpm)

+ # install dependencies defined in RPM spec file

+ RUN dnf -y install findutils rpm-build python3-pip python3-mod_wsgi python3-psycopg2 \

+     && rpm --query --requires --specfile ./resultsdb/resultsdb.spec | xargs -d '\n' dnf -y install

  

- EXPOSE 5001/tcp

- VOLUME ["/etc/resultsdb", "/etc/httpd/conf.d"]

- ENTRYPOINT ["mod_wsgi-express", "start-server", "/usr/share/resultsdb/resultsdb.wsgi"]

- CMD [\

+ COPY . /opt/app-root/src/resultsdb/

+ # install using --no-deps option to ensure nothing comes from PyPi

+ RUN pip3 install --no-deps ./resultsdb

+ 

+ # config files

+ RUN install -d /usr/share/resultsdb/conf \

+     && install -p -m 0644 ./resultsdb/conf/resultsdb.conf /usr/share/resultsdb/conf/ \

+     && install -p -m 0644 ./resultsdb/conf/resultsdb.wsgi /usr/share/resultsdb/ \

+     && install -d /etc/resultsdb \

+     && install -p -m 0644 ./resultsdb/conf/settings.py.example /etc/resultsdb/settings.py \

+     && install -p -m 0644 ./resultsdb/conf/resultsdb.conf /etc/httpd/conf.d/

+ 

+ # clean up

+ RUN rm -rf /opt/app-root/src/resultsdb \

+     && dnf -y autoremove findutils rpm-build \

+     && dnf clean all

+ 

+ # EXPOSE 5001/tcp

+ EXPOSE 5001

+ CMD ["mod_wsgi-express-3", "start-server", "/usr/share/resultsdb/resultsdb.wsgi", \

      "--user", "apache", "--group", "apache", \

      "--port", "5001", "--threads", "5", \

      "--include-file", "/etc/httpd/conf.d/resultsdb.conf", \
@@ -39,3 +47,4 @@ 

      "--access-log", \

      "--startup-log" \

  ]

+ USER 1001:0

file modified
+1 -6
@@ -6,16 +6,11 @@ 

  

  ```bash

  $ docker build -f openshift/Dockerfile \

-                --tag <IMAGE_TAG> \

-                --build-arg resultsdb_rpm=<RESULTSDB_RPM> ./

+                --tag <IMAGE_TAG>

  ```

  

  `IMAGE_TAG` is the tag to be applied on the image built.

  

- `RESULTSDB_RPM` is either the URL of the ResultsDB rpm, or the relative path of

- that rpm *within* the build context.

- 

- 

  Using the container image

  =========================

  

  1. Install app from source without building RPM to make it easier to build an image on OpenShift.
  2. Install dependencies from official Fedora repo rather than PyPI
    to comply with requirements from the Fedora community.
  3. Bump up Fedora version to 29.

rebased onto a58f4162bbbca39c3747c89385eb1753bfd7b4c8

5 years ago

@rayson Ideally, find somebody to test/review the changes. I'm not really that informed about the openshift requirements, nor do I have enough knowledge to test the changes properly.

@ralph @mikeb ?

rebased onto 5d30513f1bc2ab120890b91eaad5c0bfa81b7660

5 years ago

@jskladan Thank you for letting me know your confusion.

background

We are looking to leverage Greenwave for gating a container image between environments (dev,stage,prod,etc)[1]. To make that happen, new CI message types[2], like container-image.test.complete and container-image.gate.complete, are defined. Now we are in the progress of the deploying a cloned environment on UpShift (PnT's internal OpenShift cluster) where ResultsDB, ResultsDB-updater, WaiverDB, Greenwave, UMB, and all related Factory 2.0 services are modified to support gating containers.

purpose of making this PR

  1. I am hooking ResultsDB to debug and inspect the dataflow between services. But I find it's troublesome to make some changes then deploy it to UpShift.
  2. The current Dockerfile requires external RPMs before building, which make it harder to use OpenShift internal build system. This pull-request intends to build the container image from source.
  3. I'd like to deploy containerized ResultsDB to OpenShift during our integration test procedure. Changing the way of launching resultsdb from mod_wsgi-express to httpd will make the containerized ResultsDB more like production.

About test the changes

I will follow up later.

[1] https://pagure.io/greenwave/issue/333
[2] https://pagure.io/fedora-ci/messages/pull-request/18

rebased onto a2c878d17650aa0edb09afa79d54236c370bd63a

5 years ago

@mikeb could you comment here please?

Hey, guys. Just checking in on the progress - is somebody going to sign off on the changes? I could easily merge the PR, but as I said earlier, I can't really determine whether it is what you need.

@mprahl @mikeb @rayson ?

+1 This looks good to me.

thank you @mikeb

@rayson - could please rebase to current develop, so the PR can be merged? Thanks!

@jskladan Hi, sorry for my late following up. I was so busy recently.

Since there has been a lot of changes since the PR was created, especially the latest RPM package for fedora based on Python 3, I need to migrate the dockerfile to Python 3 too and deploy it to OpenShift to see if it works. I will do the rebase in the next week.

rebased onto 75fac9a53e5fea67608a2921f72902751e68d3f5

5 years ago

Hi all, I've updated this PR.

To test this PR:

Build a container image locally

The following command will build an image and tag it as local/resultsdb:test

# in the repo root
docker build -f openshift/Dockerfile --tag local/resultsdb:test .

Test the container locally with docker-compose

Prepare config files

conf/settings.py

SECRET_KEY = 'my-some-secret-key'
SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://resultsdb:resultsdb@db:5432/resultsdb'
FILE_LOGGING = False
LOGFILE = '/var/log/resultsdb/resultsdb.log'
SYSLOG_LOGGING = False
STREAM_LOGGING = True
RUN_HOST= '0.0.0.0'
RUN_PORT = 5001
MESSAGE_BUS_PUBLISH = False
MESSAGE_BUS_PLUGIN = 'dummy'

conf/resultsdb-openshift.conf

<Location />
</Location>

docker-compose.yaml

version: "2.2"
services:
  app:
    image: local/resultsdb:test
    build:
      context: .
      dockerfile: openshift/Dockerfile
    volumes:
      - ./conf/settings.py:/etc/resultsdb/settings.py:ro
      - ./conf/resultsdb-openshift.conf:/etc/httpd/conf.d/resultsdb.conf:ro
    ports:
      - 15001:5001/tcp
  db:
    image: registry.access.redhat.com/rhscl/postgresql-95-rhel7:latest
    environment:
      POSTGRESQL_USER: resultsdb
      POSTGRESQL_PASSWORD: resultsdb
      POSTGRESQL_DATABASE: resultsdb

Start

Start containers with:

docker-compose up

Import database schema and load dummy data:

docker-compose exec app resultsdb init_db
docker-compose exec app resultsdb mock_data

Browse all test results with a browser: http://localhost:15001/api/v2.0/results

Test this container on OpenShift

Push the built image to a remote registry, like quay.io/yuxzhu/resultsdb:test:

docker tag local/resultsdb:test quay.io/yuxzhu/resultsdb:test
docker push quay.io/yuxzhu/resultsdb:test

Deploy to OpenShift:

$ oc process -f openshift/resultsdb-test-template.yaml \
             -p TEST_ID=<TEST_ID> \
             -p RESULTSDB_IMAGE=<RESULTSDB_IMAGE> | oc apply -f -

rebased onto 0793d23

5 years ago

Thank you @rayson!

Still, could you please submit the PR against develop branch, instead of master? We only merge into master from the develop branch. Thanks again.

@jskladan It doesn't seem to that Pagure allows to change the target branch.
I opened a new PR replacing this https://pagure.io/taskotron/resultsdb/pull-request/133

Thank you! I'll close this one, then.

Pull-Request has been closed by jskladan

5 years ago